blog.robur.coop

The Robur cooperative blog.
Back to index

Testing Mollymawk

2026-09-25

Unikernels compile directly to specialized virtual machine images without a traditional operating system, userland, or shell. Testing them has historically presented unique challenges. How do we test a web application running as a unikernel? Over the past month, we set out to answer this question by building a test suite for Mollymawk.

1. Testing a Unikernel

In standard web applications, testing HTTP endpoints usually involves running the server on localhost and sending curl requests, or using test harness libraries that hook into the application's request pipeline.

In a MirageOS unikernel, however:

To solve this, and for a start, we did the following:

a. Decouple business logic into a native library: We extracted Mollymawk's core logic into a reusable OCaml library (mollymawk_libraries). Previously, the code was compiled strictly as part of the unikernel target. Organizing the codebase into a library allows us to write standard native Unix test executables that link against the exact same business logic modules (storage, user management, policy validation, email, Albatross interaction). This ensures that the code being tested is identical to the code running inside the unikernel.

b. Mock external Mirage devices and run loopback services:

c. Unit and Integration testing with Dune and Alcotest: Using Alcotest and Dune, we executed unit and integration tests locally and in CI.

2. Verifying Disk Storage and Schema Backward Compatibility

Mollymawk persists sensitive state—including user credentials, access control policies, API tokens, scaling policies, and email settings—to disk using oneffs (One File FileSystem) over a Mirage block device.

Data corruption or subtle serialization bugs on disk could lock administrators out of their infrastructure. Mollymawk has also undergone several changes to it's data format, so testing for backwards compability enables us to perform data migrations and code refactorings without much worry.

3. Testing HTTP Endpoints

Mollymawk defines 66 distinct HTTP routes. These range from public landing and login pages to administrative API endpoints that trigger VM deployments, update scaling policies, manage block storage, or stream console logs.

We wrote tests covering all 66 routes.

Testing web endpoints requires more than just testing the "happy path". To prevent security regressions and ensure consistent behavior, every endpoint in Mollymawk is evaluated against a comprehensive testing matrix:

  1. Authentication: Does an unauthenticated request get properly rejected (HTTP 401 Unauthorized for API endpoints, or redirected to /sign-in for browser views)?

  2. CSRF Protection: Are state-changing POST requests protected by valid CSRF tokens? Do requests with missing or expired tokens fail with HTTP 403 Forbidden?

  3. Authorization & RBAC: Can regular users access administrative endpoints (e.g. activating user accounts, creating other users, or overriding global Albatross configurations)?

  4. Input Validation: What happens when required JSON or form fields are missing, malformed, or contain invalid types (e.g. invalid IP addresses or negative CPU quotas)?

  5. HTTP Method Enforcement: Does sending a GET request to a POST-only endpoint return HTTP 405 Method Not Allowed or HTTP 400 Bad Request?

Structuring Endpoint Tests with Alcotest

Using Alcotest, we organized our tests by functional domains:

In total, 282 individual test cases which complete in under 7 seconds.

Testing `Mollymawk data serialization tests for storage'.
Test Successful in 0.003s. 17 tests run.

Testing `Mollymawk API Function & Data Format Tests'.
Test Successful in 6.807s. 265 tests run.

4. Tracking Code Coverage with Bisect_ppx_ng

To ensure our tests were actively exercising critical branches and error handlers, we integrated bisect_ppx_ng into our Dune build workflow.

Bisect provides detailed code coverage statistics:

The remaining uncovered points in unikernel.ml represent code which executes when booting inside a real virtual machine hypervisor, whereas the HTTP routing logic, request dispatching, and error handlers are exercised by our native test harness. While there's no end to how much tests can be written, the test suite as of now covers a lot of areas and is a good starting point for mollymawk.

Mollymawk bisect_ppx_ng coverage report

5. Some issues Discovered While Writing Tests

Writing this test suite was not only about asserting that existing code worked; it also flushed out a variety of subtle bugs and edge cases:

6. Continuous Integration with Forgejo on FreeBSD

Having a test suite is only half the battle; it must run automatically on every pull request to catch regressions before they reach the main branch.

As part of migrating our repositories to git.robur.coop (powered by Forgejo), we set up automated CI running on our own infrastructure: a dedicated FreeBSD 15.1 virtual machine runner running Forgejo Actions (forgejo-act_runner).

This CI runner executes dune runtest and checks code formatting with dune build @fmt on every pull request:

Mollymawk CI

6. Conclusion

Mollymawk now has:

The work to build the test suite and CI was completed across the following pull requests: