Slow, but Steady Progress

Phew, another 6 months went by. Is it possible that time is running so quickly? I spent time grinding through different tasks. I finished the automated map extraction that will magically extract all data that is required to control the background swapping of charsets that I described in the last post. The algorithm more or less starts with a map area that is about as big as one screen. It then increases the width and the height of the area until more than 256 characters are used. When that happens, it backs up a bit and starts a new area, until the complete map is covered. The screenshot below shows that the village level has 292 characters and is therefore split into 3 submaps.

I also added the handling of trigger areas (areas that will trigger a script when they are entered by the player) and command areas (areas that will add a command to the menu when it is opened while the player is inside) graphically in the editor, which makes creating them MUCH easier compared to the manual assembly source file hacking it was before.

A bit has happened on the graphic side, too. I created some more nature tiles to design a map in the woods. Designing trees is really difficult for me, but at least they look a bit better than the ones I had before. This is how it looks like:

Last but not least, I reworked the in-game menu and in particular the inventory handling. While the inventory was only informative until now, it is now fully functional and allows the equipment of items. I once again realize that I am no graphic wizard and it looks a bit pragmatic, but I still think it is an improvement to before and allows for much more complex quests. You might notice the attribute display in the right top of the screenshot below: yes, there will be more role playing elements coming.

 I guess the next big thing will be combat, and I am really excited to experiment with that. A convincing round based combat with a single player instead of a party is a challenging thing to do and I guess there will be a lot experimentation needed until it really works.

Automatic Charset Extraction

I mentioned in an earlier post already, that the ToS 2 game engine on the C64 is pretty flexible regarding the handling of character sets. The character mode on the C64, which is more or less required to create games with scrolling, makes use of one set of 256 characters at any given time. In ToS 1, every level consisted of two graphic sets, one for the outside world and one for all inside rooms. These two graphic sets had each a completely independent character set.

Now with the ToS 2 engine, I decided to allow to use a different graphics set for each room (with the outside world being a “room”, too). This allows for much more flexibility when designing rooms, e.g. rooms inside wooden huts, caves or palaces all in the same level. Obviously, defining the related character sets completely independently would need a huge amount of memory. Every full character set needs 2kb, so a dozen rooms would already use more than 20kb of charset data, which is by far too much. Therefore, the character sets are assembled from segments of characters, which can be combined in a mostly arbitrary manner. The picture below shows the principle.

Originally I thought that one could place these segments of characters at any location in the character sets, but using arbitrary positions for the segments creates a problem: The tiles used in the engine reference the characters in the charset by index. If the same segment appears at different places in different charsets, all the characters in the segment suddenly get different indices. So if a tile that uses a character from such a segment is used in different rooms, the index in the tile definition becomes ambigious. This would mean that the tile has to be defined several times, for each of the different character sets. As tile definitions are comparably expensive (16 bytes per tile), this did not seem a good thing to do. So all segments have a fixed position that is valid for all charsets in which they are used.

Managing the segments and character sets looks complicated, and it is. I did not want to spend time thinking about these things while creating levels, because that can really hamper creativity. So I decided to design an automatic mechanism in the editor, that would take care of deciding about which character to put into which segment, and how to combine the segments into character sets so that they fit for the rooms and need the smallest possible amount of memory at the same time. I did not find a suitable algorithm on the Internet, although it might well be that there are some. But I had a hard time coming up with suitable search terms, as the problem to solve is rather peculiar. In the end I came up with something myself after lots of head-scratching.

Unfortunately, a brute force approach that simply tries out all possible solutions and selects the best one does not work, because even for 256 characters the possible number of permutations is in the order of 10506. And that is only all possibilities for one single segment, the number gets far bigger when all potential combinations of segments are taken into account. Even for modern PCs this is far from doable in a reasonable amount of time. Instead, the algorithm that I came up with starts with a trivial solution: every single character is put in its own segment. After that, two segments are merged to a larger segment and it is verified, if the resulting segments can still be combined to character sets such that they cover every existing room, while still staying in the limit of 256 characters. The two segments that are chosen for merging are selected after evaluating, by how many rooms they both are needed. The two segments that are used together by the highest number of rooms are the best candidates. If merging them does not result in character sets that work for all rooms, the pair of segments with the next highest number of rooms that use both of them is tried. The merging is done again and again with the best candidates, until finally no further merge can take place because the resulting character sets get too large. The picture below shows the first three iterations in an example.Having this algorithm in the editor allows for a very memory efficient storage of flexible character sets, so that I can design the rooms in a level much more interesting. This solves one aspect of the handling of more than 256 characters. The other aspect is to identify areas on the map of a single room, for which the character sets can be adapted on the fly in the background while moving on the map in the game. Obviously, I also did not want to handle this manually, so I wrote another algorithm. But more on this in the next post…

Protovision, Cartridges and a Village!

So many things have happened since the last update! First of all, I am proud to be now a member of Protovision! This solved a couple of questions that were bugging me regarding the publishing of ToS2 on physical media: e.g. how to deal with the German FSK or or US MPAA or whatever legal age restrictions are relevant for computer games in all the countries around the world that ToS2 might be delivered to. Also, how to deal with payment and delivery for international customers etc. So I was pretty happy when Jakob from Protovision contacted me about joining them!

Now, the really cool thing is, that Protovision is ramping up on a new game cartridge developed with Individual Computers. The fantastic Sam’s Journey developed by Knights of Bytes will be released on this cartridge and I was able to get my hands on a test cartridge (unfortunately without a Sam’s Journey pre-release though). So I started integrating it into my development work flow and am happy to say, that I am really confident that there will be a cartridge version of ToS2! I would love to provide a disk version in addition, because I am a huge fan of good ol’ 5,25″ floppy disks, so potentially ToS2 will be released on both. Let’s see… In the course of working with the cartridge I also had the chance to chat quite a bit with Chester Kollschen from Knight of Bytes, which was a most interesting information exchange by itself!

This all means that I spent quite some time with the cartridge and with writing tools to make it work with the current development version of ToS2. Nevertheless I did some more work on the game itself. I am still in the process of finalizing the implementation of the new, highly memory efficient data structures for the game. But in order to learn efficient working with the editor I did a draft for a new level. Despite the general gloomy mood of the game, there will be a couple of light and friendly moments, too. One of them will take place in a scenic German village with pretty half timbered houses. I am currently really unhappy with the trees, so expect some changes there (and probably elsewhere, too). But without further ado, here comes a first screenshot of a ToS2 level!


Milestones and Kanban

In the last month I was able to finish version 1.0 of my game data editor “TOSEdit”. This is the first actually useful version, because it can create tiles, charsets and maps and export them in the same binary format as the wonderful CharPad. This will be the base for the further required features, that are needed to support the more special properties of the TOS2 C64 engine (like e.g. swapping parts of the charset on the fly). Interestingly, the implementation of the editing itself went pretty quickly (did I already mention that I love Java?). But what really needed time was a reasonable implementation of cut, copy and paste and of undo. I never implemented an undo mechanism before and it is actually quite a challenge. You need to do it completely for all actions that change the underlying data somehow, or it will fail miserably. Well, I was able to finish it and thought that this is worth a “release”. Where release in this context only means that I give it the satisfying version number 1.0 and write a blog post about it!


I think working towards milestones and releases is so much more satisfying! When starting the work on the editor I decided to even go one step further and apply some agile methods to plan and track my progress. As I am the only person working on the project, Kanban seemed a reasonable method and what can I say: I love it! Kanban is a simple way to organize tasks (or rather “stories”, which are features that are needed, described from a stakeholder point of view). In my specific case there are 4 columns with such stories, a “Backlog” of things that I want to do at some time in the future, but did not think enough about that I could immediately start work on. After I spent enough thoughts about it, the story is moved to the “Selected for Development” column. When I actually start working on it, it gets moved to “In Progress” and when I finally finish it, it gets moved to “Done”. There is one specific rule that is typical in Kanban, which is that you limit the number of stories that can be in a certain column. In my case I decided to say that I never want more than 3 stories in progress. This helps to focus and get things done instead of being distracted too much.  I am using the wonderful Atlassian Jira (which is free for small projects and nicely combines agile methods with bug tracking) and below is a snapshot of my Kanban board, filtered for the TOSEdit 1.0 release. You can see that there is only one issue left (which is writing this blog entry), which I will move now to the “Done” column. Really satisfying!


The Editor

Wow, no posts for such a long time! Well, it is summer and I have to admit that I did a bit of a break from my PC while the weather was so nice. But did I take a break from the project? No! I spent a bit of time thinking about concepts, I fought a bit with infrastructure (and actually had to change the provider of my development VServer, well you get what you pay for), and I even spent some time with scissors, paper and glue and created some prototypes for a nice packaging. Googling the Internet for creating cardboard boxes is an, um, interesting experience. Apparently only kitsch fanatic house wifes seem to care for this normally. But who says I am normal :-)! The results of all this work are nothing exciting to show off, but were important to work on at some point (at least for me, as I am a bit fanatic about the overall experience of a product). On the code side I did lots of clean-up and integrated the task scheduler and the font swapping module I wrote about further down. I wrote many many unit tests, so I am of good hope that the game will be much less bug-ridden than the compo version.

What I did this weekend was maybe more exciting: I finally started work on the game editor. With all the new features of the game engine, standard tools are just not flexible enough to work with. So here comes a screen shot of the first editor version! A lot is still missing, but I feel I am on a good way:

tosedit It is really heavily inspired by the fantastic CharPad, I hope the creator does not see this as a plagiat, but rather a tribute to his great software! Especially the font swapping needs a lot of intelligence in the software.

Apologies once more for the delayed update! I expect them to happen more frequently now that the weather is getting more favorable for dwelling in my computer cave…

Tasks, schedules and priorities (2)

Apparently I was creating more questions than answering them in my last post. I will try to provide a bit more detail about the implementation of the task scheduler.


The interrupt routine decides, if it wants to schedule a new task. If the new task has a higher priority than the one that is currently running, we want to switch to the new task. So what we need to make sure is, that when we return from the interrupt we do not return to the old task, but to the new one. In order to achieve this, we need to manipulate the stack accordingly, as the return address (and some more information) is stored there. Let’s look at what happens when an interrupt is triggered:

  1. the currently executed instruction is finished
  2. the current program counter is pushed onto the stack, high byte first
  3. the status register is pushed onto the stack
  4. the interrupt flag is set (because you usually would not want your interrupt routine to be interrupted itself)

All these steps are performed by the hardware itself. An interrupt service routine normally saves the registers in software in addition:

  1. A is pushed onto the stack
  2. X is pushed onto the stack
  3. Y is pushed onto the stack

How can we manipulate the stack now to return to a different task? We simply push respective data onto the stack!

    lda #>task
    pha          ; push >task
    lda #<task
    pha          ; push <task
    lda #0
    pha          ; push arbitrary states, IRQ flag must be clear
    pha          ; push arbitrary A
    pha          ; push arbitrary X
    pha          ; push arbitrary Y

When the end of the interrupt service routine is reached, it will restore Y, X and A and the final rti will pop the state register and jump to the new address we just pushed onto the stack! We do not have reasonable values for the registers (as the task was never executed before), so we simply chose 0. There is one caveat regarding the state register: we need to make sure that the interrupt flag is clear, or the interrupt will never be called again. It took me a while to realize this, as I was happily using php to push the current state register instead and as can be seen from step 4 above, that had the interrupt flag set.

Now let’s have a look at the task_done routine that is jumped to by a task when it is finished. In theory, all we need to do is to jump to the task with the next highest priority. Let’s for now assume, that the old task has the next highest priority. Things get slightly messy now: we still have the remains of the interrupt on the stack, but tasks are always executed in the main loop! So instead of jumping to the new task with jmp, we will misuse rti and simply do the same as we would do at the end of an interrupt routine:


Et voilá: we restore all registers and are back in the old task!

There is one additional thing that needs to be organized: when a task is added with a lower priority than the currently executed one, it needs to be stored for later. To do this, a list with upcoming tasks needs to be kept. Every time a task is done, the list needs to be searched for the next highest priority and the task needs to be pushed onto the stack as described above. But I am going to leave this little detail to be worked out by the reader (ha, I always wanted to say this cruel sentence at some point in my life)!

Tasks, schedules and priorities

Here we are with another rather technical update! Last time I was posting some information about how I want to get more diversity into the game graphics by switching parts of the character set on the fly while the player moves through the world. That actually raised an interesting new problem: how to handle tasks with different priorities!

Up to now there were only two possible priorities: tasks that need to be done during the currently displayed frame (like color RAM scrolling or inserting new characters at the border of the screen) and tasks that only need to be finished eventually during the next frames (like scrolling the currently invisible double buffer or checking for trigger areas for events). The character set swapping adds another priority: in theory it can take quite long and there is no need to have it finished in the next 2-3 frames, it is ok even if it takes e.g. 10 frames or so. Instead of patching this into the existing engine I decided to do it right from the start and be prepared for any upcoming additional tasks.  Which means: implement a task scheduler!


What does a task scheduler do? It makes sure, that most of the processor time is spent on the tasks with the highest priority. I simplified the concept a bit, so that higher priority tasks are always finished before lower priority tasks are continued. The implementation is actually simple, although there is some mean stack fiddling involved and it needed a bit of head scratching until I got it right. The interface of the scheduler module has only two routines: task_add, which adds a new task with a given priority and might switch to it immediately (if it is the highest priority), and task_done, which is called by a task when it is done and selects the next highest priority task to be executed.  All that is needed is to modify the stack such, that the RTI instruction at the end of the interrupt routine jumps to the right task. Apologies for the clumsy block diagram trying to depict this with an example. Did I mention that I am not a graphician?

In any case: implementation is done and unit tests are passing (yay!), so I am looking forward to put this new module into good use!

Gold plated tests and breaking the 256 character limit

Last time I was talking a bit about the test framework I am setting up. I actually became a bit obsessive with this in the last weeks and spent a lot of effort to get it to a point where it fulfils everything I need. Meanwhile I am at a point where tests are run every time something is submitted to the version control system. The result can be nicely seen on a html page and a history of the last 20 tests is visible. Log files for passed and failed tests are stored and can be accessed from the html page. All in an eye pleasant view, slightly  heavily inspired by a professional commercial CI system (is anyone able to guess which one? Write it into the comments and you will get personalized kudos in the next update :-)).


From the screen shot above you can guess that I have some unit tests running meanwhile. I decided to run the Vice emulator using a virtual display, because that allows me to run it on my headless server and still store screenshots in case something unexpected happens. This proves to be pretty important, as my tests rely on a Vice break point to be triggered. If this does not happen for some reason (e.g. if I messed up the code and it does totally different things from what I actually intended), the test system will grab a screen shot and kill Vice after a time out. That makes it easier for me to guess what went wrong.

But I did more than gold plating my CI system: I finally started implementing the much more advanced idea I had for increasing the char mode flexibility. The idea is to assemble the current charset arbitrarily out of different char set parts. For any rectangular area on the maps, the combination of charset parts can be changed. This is done in the background, so the player does not notice it. Obviously that does not help against the limitation of 256 character visible on the screen at the same time. But as soon as a couple of characters are not visible anymore, other characters can be swapped in. This is hard to handle during level design, but I think it should be doable to optimize the charset parts automatically in the level editor. Ideally, I could simply design the levels mostly arbitrarily, and the editor will take care for the rest. Let’s see how it works out…

Still alive!

Wow, that is a long time since my last post! Well, I had a lot to do in my daily job and all my other hobbies demanded some well-deserved time. But here I am back in front of my PC and typing away. The project is surely not dead, it just needed a bit of a break to go back to full speed again. There was a bit of a setback with the anticipated help for the graphics, so currently the team consists more or less of one person again. Nothing to worry about though, as there is no time pressure! That’s the nice thing about hobby projects…

I picked it up where I left it and finally finished the mechanism for continuous integration testing. “Continuous wha…?” I hear you say. Well, it is a buzz word for automated testing triggered by any kind of change. There are really really advanced tools to do all this kind of stuff, but I felt that they are waaaay too much overkill for my use. So I quickly setup my own system, that is really really simple. There is a daemon running on the server that watches for submits in the versioning system. As soon as something has changed, it will start a script on the server, that will get the latest version of the software and search for subfolders in a dedicated test folder.  Each of these folders contains a script, that can generally do any kind of test and will output the result on a web page. Passing and failing is signaled by a colored flag, so it is easy to see if something got broken. Additionally the test script itself and all kinds of error outputs are accessible from the web page, to easily identify a problem, when the test fails.

Up to now, the only test builds the disk image for the game, so a build buster can be easily identified. Not so impressive, as this can easily be checked locally on the machine where I do my development. But this is the basis to do much more advanced things. I am planning to run a C64 emulator and perform all kinds of tests in that, e.g. do a speed-run through the game to see, if it is still possible to finish it! I did this manually countless times for the original “Time of Silence” and it really, really sucked big time. You begin to hate a game, when you play it through for the hundred-fiftieth time, and this is definitely not a good prerequisite for coming up with ideas for improvements. So this is something that I will definitely avoid this time.


The infrastructure is now practically complete, so there is no excuse for not ramping up on the game itself again! Well, there is also the level editor, let’s see where I am going to concentrate on next…

All C64 games suck!!!

Ha, got your attention with this vocal headline? It is obviously not true that C64 games suck, well at least not all of them. But when I started thinking about making a C64 game, the first thing I did was reflecting about what I like and what I hate in existing C64 games. And to be honest, the things I hated were an overwhelming majority. I realized that I am actually hardly ever motivated enough to play through an adventure or RPG game on the C64, although I do this pretty frequently for PC games. So I made a list of things that I really hate and that I want to avoid in a game I am writing myself.

  1. Cumbersome user interface. The C64 has so many keys, so let’s use every single one of them! Many old school adventures really made use of this and made me pull my hair off on the desparate search for this darn key for the Fireball spell. Sure, this was documented in the manual, but who plays still with a manual on his knees nowadays? A user interface should be simple and follow the habits that people are used to today. E.g. run/stop could be a replacement for escape and interrupt a cut scene, or cursor keys and return can be used for selecting options.
  2. Frequent and eternal loading. I have especially fond memories of the game “Knights of Legend”, that would require two disk swaps and several minutes of loading to show the inventory (this was cleverly combined with the cumbersome user interface of point 1, so it happened frequently that I would accidently hit the inventory key and needed to follow this procedure without any chance to cancel it). This is a tough one, as the C64 really has a small memory and loading is slow, even with fast loaders. So what can you do? At least try to load, when there is a break in the story anyway. I read an interesting paper about the psychologic effect of progress bars. Not only do progress bars make the waiting feel shorter, but adding an animation makes them even feel progress faster! And finally, if there is the need for a long load, well then at least it should be worth it and reward the player with as much new stuff as possible. New music, new tilesets, etc. Knights_of_Legend
  3. No in-game explanation of background story. Although I love the presence of additional printed material for games, I think a story should be understandable without reading hundreds of pages in the manual before you start. A cut scene here and there and a bit of in-game text can really make a game much more immersive.
  4. Instant death. We all know it from 80s games and hate it. Full stop.Labyrinth_-_C64_-_Game_Over
  5. Starting over and over again. This surely depends on the genre, but for most games I think a save/restore mechanism is not hard to do and can save the player from a lot of frustration.
This is only my personal view and it is probably very focused on adventure/RPG style games. There are absolutely great games that happily ignore these rules. But I will try to follow this vision and hope to make life at least a tiny bit more comfortable for C64 gamers today.