blog.robur.coop

The Robur cooperative blog.
Back to index

Improving how Mollymawk handles data

2026-10-14

Mollymawk from the beginning has been running on it's MVP version. This version mainly focused on getting things to work which was often crude, and not much about how efficient it was. At this point of the journey, we are now focusing on polishing mollymawk up a bit more, and finetuning the underlying implementations we have done.

To put this into context, some issues with the MVP were:

It's worth mentioning that by design, OneFFS is not meant to be fault-safe and this is why we also had to rethink how mollymawk persists data.

In this article, we'll walk through how we migrated mollymawk's data structures from lists to maps, introduced strict type safety, decoupled ephemeral data from disk persistence, and moved our block storage layer to a more fault-safe filesystem (including migrating previous data to the new filesytem).

Changes made

Type safety

All data which was previously "stringly-typed" has been migrated to a more appropriate type safe version:

Other values which were correctly strings, such as names, but that need to be in comformity with albatross - we also changed them to directly use what albatross uses internally, which is Vmm_core.Name.Label.t, and now we can be sure that constructing values of this type in mollymawk will have validation within albatross.

Migrating from Lists to Maps

To improve the performance of mollymawk, we replaced the various lists with maps using OCaml's Map functor.

module LM = Map.Make (Vmm_core.Name.Label) 
module SM = Map.Make (String)            
module UM = Map.Make (Uuidm)

So Map.Make takes a module, such as Uuidm and creates a map where the key is of type Uuidm.t. Same applies for the others.

Then we migrated each sub-lists into a map - which are:

Then we created secondary maps (indexes), which are not persisted to disk but help for quick lookups:

With secondary indexing in place, API token authentication no longer needs to scan every user in the database. In terms of complexity, if we wanted to authenticate that a token indeed belonged to a user, we will have to traverse the users list and for each user, scan all their tokens. Now with maps, the lookup happens once in the secondary index, and then in the primary users map.

With 10,000 users holding 5 tokens each (T = 50,000), the list version can do up to 50,000 comparisons. The map version does roughly log₂(50,000) + log₂(10,000), which is 16 + 14 = 30 comparisons, which is a great performance improvement.

Data persistency

With the upates so far, the next logical step was to stop persisting data which is meant to be short-lived, such as cookies and csrf tokens. As mentioned above, every time a user refreshed a page or submitted a form:

  1. A new CSRF token was generated.
  2. The user's cookies were updated.
  3. The entire storage state was serialized to JSON.
  4. The JSON blob was written to the block device.

To plainly put it, everytime a user accessed a page, the entire json blob was written to disk. Now, cookies and csrf tokens are no longer persisted but are kept in memory as ephemeral data. A result of this is:

On-disk storage fault-safety.

We have moved mollymawk's storage from oneffs to a custom filesystem. This works well and is already used by another project of ours tlstunnel, which is also a unikernel. This custom filesystem has one main advantage, fault safety - by which at any point of program execution, the program may stop, but the persistent data is still in a readable state (either the data before is intact or the new data is live). Some details about how this is achieved is

Migrating data from oneffs to the new file system

To be sure we can safely migrate all our data to the new filesytem, we wrote a binary executable which takes a source and destination disk, connects to the source disk with oneffs, reads the data into memory, connects to the destination disk, formats it with the new filesystem and then writes the data to it.

Conclusion

In this article, we explored how we updated mollymawk's internal data structures, improved type safety, changed the filesystem used for persistency, and a way to migrate data from the old block device to a new one. There's still some things to be done about making mollymawk more efficient - such as moving away from mirage to a new stack (mkernel, mnet and vif) - but we are in the right direction.

This refactoring was carried out across the following pull requests: