Mollymawk supporting other virtual machines
2026-04-15Albatross and Mollymawk were initially focused on deploying MirageOS unikernels with Solo5. As the time goes by, the usefulness of gathering metrics in the host system about the unikernels, management of the boot order, capturing console output, turned out so useful that we extended Albatross and Mollymawk to manage normal BHyve virtual machines.
The advantage is that now we can use Mollymawk to manage all the virtual machines that are running on the server, and not only the MirageOS unikernels. This means we can drop the shell scripts (or vm-bhyve or other utilities) to start and manage virtual machines, and use a single daemon for everything. Since Albatross also supports restarting (depending on the exit code), we no longer have to duplicate the non-trivial restart logic. Note that the functionality is at the moment only implemented for FreeBSD's BHyve - that's what we primarily use. Adding support for qemu (on Linux/KVM) is straightforward, please get in touch if you have this use case.
Differences between Solo5 and BHyve
Solo5 is a minimal "tender" - the application that allocates resources and interacts with the MirageOS unikernel. The variety of resources is limited to tap devices (for network) and block devices (for persistent storage). BHyve embeds much more functionality, and uses the VirtIO protocol to setup the devices for the guest operating system. This includes a console, SCSI, 9P, RNG, AHCI, ...
The boot process of BHyve consists of the loader (if launching FreeBSD virtual machines, bhyveload is used - for Linux, grub-bhyve can be used) which sets up the virtual machine, and then bhyve which executes the virtual machine.
Dependencies between virtual machines startup
We added a simple mechanism to order the startup of virtual machines. This is crucial if some virtual machines are depending on other virtual machines: in our setup, the git server needs to be running and ready before the primary DNS server is able to starts its service (since the data is kept in a git repository). This was developed in August 2025 in this PR and the corresponding Mollymawk PR.
You can now configure a virtual machine to be started early or late. This is just a number, as in the old System V init system. Simple and sufficient for now.
Starting non-unikernel virtual machines
Each MirageOS unikernel is (so far) restricted to a single CPU. We extended the main datatype to cope with multiple CPUs (a new field numcpus : int was introduced), and as well the possibility to pin (taskset / cpuset) to a range of CPUs (the field cpuid : int is now cpuids : IS.t - with IS.t being a set of integers). This ensures that in a multi-tenant setup physical cores can be assigned to a single tenant (this is a viable defense strategy with CPU cache-timing attacks in mind). Another addition is an optional string linux_boot_partition, which indicates whether to use grub-bhyve or bhyvectl. If it is set, this is passed as -r argument to grub-bhyve.
Since we keep our virtual machines in separate virtual block devices (on zfs called zvol), which are (not yet) under control of Albatross/Mollymawk, we also introduced an escape way, and the albatross daemon can allow a single user access to /dev/zvol (via a command-line argument --allow-zvol-for). We expect this to disappear once we have a separate Albatross block device daemon - which may come in two versions: one using files (and truncate), the other using zfs features, and a third one for btrfs if desired.
The changes are developed and merged in this Albatross PR.
For Mollymawk, we added some UI elements to create a BHyve virtual machine, allowing multiple CPUs, and the linux_boot_partition. Also, in Mollymawk we can specify the block device via a path now. All this has been developed in this Mollymawk PR.
Conclusion
We can now create and monitor virtual machines that are not MirageOS unikernels in Albatross and Mollymawk. We already capture the console output of these virtual machines. There are some shortcomings: (1) this is only supported on FreeBSD at the moment, (2) there's no console input, (3) there's no output of the loader, and (4) we don't support (yet) to install other virtual machines from scratch. These are tracked in the Albatross repository, please reach out if you need any of them immediately.