It’s been a mostly behind the scenes week of development, with very little visible (from a player’s point of view) to show for it – other than not crashing, which is always a good feature. 🙂

  • “Murky pools” (pools that appear and aren’t connected to a river) are temporarily disabled, due to a bad habit of leaking! I think I’ve got it tracked down, but not in time for this post.
  • Finished the great CMake project. CMake can now emit a working build on Mac, Linux and Windows – so I’m no longer tracking a separate Visual Studio solution (which often meant changing files in CMake and having to run over and fix the VS build). It’s quite the CMake file; it downloads (unless you have it cached) Boost and builds it, builds SFML, FastNoise, Lua, RLTK, SQLite and Zlib from source (obtained via Git submodules) and then compiles from that (with optimization including LTO/Whole Program on release builds). The result is that I can take a fresh machine with developer tools, clone/submodule init, make a build folder and start the build process to go from zero to game. Compilation time the first time is pretty brutal, but once it is built incrementals are nice and zoomy.
  • Read some great articles on geology, with a view to making the strata system make more sense in the future. I have an ‘A’ level in Geography with a geology module, but it’s been so many years that I don’t remember very much. It was surprisingly refreshing to renew the knowledge!
  • Working on serialization, again. This has been the biggest trouble-spot since I fixed the regular crash problem. Anyway, I ran into a time-dependent serialization problem – the game would fail to load, but only after you’d been playing for a while. Debugging this was driving me crazy, and I realized that my biggest problem is that the save-game file is a really efficient, binary blob that is super-tough to diagnose in a hex editor. So I started out down the road of writing pretty printers and similar, and had the epiphany “why not just use a human-readable format, so that I – allegedly human – can read it?” See the “Serialization Mark 3” stuff below for details.

RLTK – The Modern C++ Roguelike RToolkit Github**

  • All new serialization and inspection system – see above and below!
  • Fixed a performance bug in several examples. The examples were trying to A* path to a tile that we already knew was inaccessible, leading to a thoroughly pointless slowdown.
  • Fixed a map-generation error in the examples.
  • Fixed an issue when compiling on MingW64 that made the random number generator predictable. For some reason, GCC on Windows didn’t bother implementing std::random_device – it always returns the same number! Since the RNG was relying on it, we had RNG code that was entirely predictable on one compiler, and not on others. It now seeds from the system time (unless you specify your own seed, in which case nothing changed).

Thanks to /u/TheAutsider for reporting several bugs!

Serialization mark 3

So moving to a human-readable serialization format also meant that I should fix-up the serialization system built into RLTK. There were a lot of things that I didn’t like, and now that I’m starting to attract some users I thought I’d make life easier for them.

Step 1 was to write a template-based serialization system that can take structures and save them into the psuedo-xml that I’m using. Since C++ barely has refelection, I couldn’t take my preferred approach from other languages and simply inspect classes for attributes; the components had to have a way to say “serialize this”. So I ended up with the following interface, which along with a member variable xml_identity is all you need to save:

void position_t::to_xml(xml_node * c) {
    component_to_xml(c,
        std::make_pair("x", x), 
        std::make_pair("y", y), 
        std::make_pair("z", z)
    );
}

I went with a pair with strings to allow for proper naming in the serialized format (and am working on a helper to get rid of the boilerplate; it will accept an initialization list soon so you can simply do {"x", x}). component_to_xml is piece of template trickery that took far too long to write. It examines each parameter, and determines:

  • Does it have a to_xml member? If it does, then it is put inside its own XML node and the variable’s to_xml is called in there.
  • Is it a vector, set, map, vector of pairs, vector of bools, bitset or other container I support? If so, then create a node, and then recursively call itself on each entry to serialize out the collection.
  • Is it a value type? If so, turn it into a string and put it in as a value.

This took a fair amount of template writing, which is always slow going for me – but it works pretty well, and it wasn’t long before I could produce a usable text dump:

  <position_t>
   <x:value>129</x:value>
   <y:value>127</y:value>
   <z:value>76</z:value>
   <entity_id:value>9</entity_id:value>
  </position_t>

Then I tackled de-serializing, which is mostly a case of determining what to load (calling a helper method provided by the caller) and letting each component de-serialize itself by reading xml_node structs.

Next, I added more template magic to the ECS in RLTK. It detects if a component defines an xml_identity string. If it does, it uses it for serialization. If it doesn’t, it performs an RTTI call and obtains a name to use that way. Then, it detects if a component defines a to_xml call at all; if it doesn’t, it still works – but you can’t save that component. This allowed me to eliminate all of the required boilerplate from the examples in RLTK, and a library user who doesn’t want the serialization system won’t pay for it.

At this point, it worked decently (barring going over component de-serialization to make sure that it works properly – I needed to do that anyway), but compilation times were approaching “go and have a coffee break every time you hit make -j8“. So a lot of the code in ecs.hpp and xml.hpp were moved into their own compilation units, and things stabilized nicely.

I then stumbled upon a huge benefit of this process that I hadn’t even previously considered. I had a “dump” function (originally for debugging) that will dump the contents of any xml_node (and it’s children) to the console in a pretty format. Every component supports producing xml_nodes whenever needed. Suddenly, without trying, I have an ECS inspector system! Awesome! It’s already proven handy – if a component is confusing me, I can simply serialize-dump it for inspection. Likewise, an in-game (debug mode) component/entity inspector is now not an overly difficult task to implement.