As you know from previous posts, I setup hass.io on a new RPI 4 recently. Well that’s been working great for months, more than 6 months actually. However a few days ago the system stopped booting correctly, and I don’t have a micro HDMI cable to check the console, bummer.
This lead me down a path that I’ve had floating around in my mind for a while. I want to move my Hass.io (now called just Home Assistant) into docker so I can manage it along side all my other docker containers.
I’ve also wanted to look into using HashiCorp’s Nomad so I can cluster my docker containers between servers.
So this post is mainly just my findings on OS options for a Docker setup on Raspberry PIs.
So here are the “requirements”:
- Must be able to run on Raspberry PIs (I have a mix of RPI 4, RPI 3, RPI zeros).
- Must be able to install docker.
- Must be able to install Nomad.
So, I spent some time researching options and here is what I found.
HypriotOS#
HypriotOS is actually an OS I used on a RPI years ago and it worked well. It’s based on Debian, but is optimized for Docker.
Pros:
- Based on Debian, so we can use traditional tools like apt.
- Docker engine already installed be default.
- Ability to use cloud-init to get everything setup with an monitor.
Cons:
- Doesn’t seem to have an LTS version, like Ubuntu.
- Not as well known, or supported like Ubuntu.
- Hypriot blog doesn’t have a lot of recent posts.
- OS doesn’t seem to be 64 bit, so won’t work as well on the RPI 4.
balenaOS#
belenaOS is pretty interesting. It looks like it’s based on Yocto Linux, which is targeted at embedded devices. On top of the Yocto Linux OS is a custom container engine instead of plain old Docker. Here are the pros and cons as I see them:
Pros:
- Very minimal.
DockerContainer engine already installed by default.- Supports many RPIs, as well as other SBCs.
- 64 bit OS, therefore will work better on RPI 4.
Cons:
- Heavily tied in with balena tools such as balenaCloud, balenaCli.
- Seems to be heavily marketed towards enterprise customers.
balenaOS looks really cool, but because it’s tied to closely with stuff like balenaCloud I decided against using it. Perhaps if this wasn’t just a homelab thing and more of a business thing I would consider balenaOS.
Another thing I wasn’t sure about was how to get nomad installed on balenaOS. I found this but it doesn’t have any info in the Readme that would help me install it, so for now I decided to pass on balenaOS.
Ubuntu Server#
Ubuntu Server is tried and true. I usually use Ubuntu LTS on all my VMs.
Pros:
- Backed by Canonical, therefore we know it isn’t going to disappear any time soon.
- Has an LTS version so I wouldn’t have to update as much, and would trust it to be more stable.
- 64 bit OS, therefore will work better on RPI 4.
Cons:
- Backed by Canonical, therefore we have to deal with weird stuff they do (snap, fish, Unity, etc)
- Doesn’t have Docker installed by default.
- Not slimmed down or enhanced for Docker.
Misc Contenders#
The OSes above isn’t a complete list of OSes for the RPI, here are a few I passed over quickly.
Raspberry Pi OS (Formally Rasbian)#
Raspberry Pi OS seems to be more for beginners, even the minimal version has stuff I don’t really need/want. It’s based on Debian, and I’d rather go with Ubuntu over Debian.
Ubuntu Core#
Ubuntu Core is interesting as it’s really slim and I like that. However, it’s one of those weird Canonical things. It doesn’t support apt, and instead you have to use snap to install everything, no thanks.
Decision#
I almost went with HypriotOS until I found this blog that explains how to setup ssh and check what the default username and password is. TLDR is that by default Ubuntu server has ssh setup with a default ubuntu:ubuntu credentials. The official Ubuntu docs don’t explain that well, and instead says to hookup a monitor.
So, since I didn’t need a monitor I went with Ubuntu server because it’s more of an “official” distro that I think will have more support than HypriotOS.