As you know from previous posts, I setup hass.io on a new RPI 4 recently. Well that’s been working great for months, more than 6 months actually. However a few days ago the system stopped booting correctly, and I don’t have a micro HDMI cable to check the console, bummer.
This lead me down a path that I’ve had floating around in my mind for a while. I want to move my Hass.io (now called just Home Assistant) into docker so I can manage it along side all my other docker containers.
I’ve also wanted to look into using HashiCorp’s Nomad so I can cluster my docker containers between servers.
So this post is mainly just my findings on OS options for a Docker setup on Raspberry PIs.
So here are the “requirements”:
- Must be able to run on Raspberry PIs (I have a mix of RPI 4, RPI 3, RPI zeros).
- Must be able to install docker.
- Must be able to install Nomad.
So, I spent some time researching options and here is what I found.
HypriotOS
HypriotOS is actually an OS I used on a RPI years ago and it worked well. It’s based on Debian, but is optimized for Docker.
Pros:
- Based on Debian, so we can use traditional tools like apt.
- Docker engine already installed be default.
- Ability to use cloud-init to get everything setup with an monitor.
Cons:
- Doesn’t seem to have an LTS version, like Ubuntu.
- Not as well known, or supported like Ubuntu.
- Hypriot blog doesn’t have a lot of recent posts.
- OS doesn’t seem to be 64 bit, so won’t work as well on the RPI 4.
balenaOS
belenaOS is pretty interesting. It looks like it’s based on Yocto Linux, which is targeted at embedded devices. On top of the Yocto Linux OS is a custom container engine instead of plain old Docker. Here are the pros and cons as I see them:
Pros:
- Very minimal.
DockerContainer engine already installed by default.- Supports many RPIs, as well as other SBCs.
- 64 bit OS, therefore will work better on RPI 4.
Cons:
- Heavily tied in with balena tools such as balenaCloud, balenaCli.
- Seems to be heavily marketed towards enterprise customers.
balenaOS looks really cool, but because it’s tied to closely with stuff like balenaCloud I decided against using it. Perhaps if this wasn’t just a homelab thing and more of a business thing I would consider balenaOS.
Another thing I wasn’t sure about was how to get nomad installed on balenaOS. I found this but it doesn’t have any info in the Readme that would help me install it, so for now I decided to pass on balenaOS.
Ubuntu Server
Ubuntu Server is tried and true. I usually use Ubuntu LTS on all my VMs.
Pros:
- Backed by Canonical, therefore we know it isn’t going to disappear any time soon.
- Has an LTS version so I wouldn’t have to update as much, and would trust it to be more stable.
- 64 bit OS, therefore will work better on RPI 4.
Cons:
- Backed by Canonical, therefore we have to deal with weird stuff they do (snap, fish, Unity, etc)
- Doesn’t have Docker installed by default.
- Not slimmed down or enhanced for Docker.
Misc Contenders
The OSes above isn’t a complete list of OSes for the RPI, here are a few I passed over quickly.
Raspberry Pi OS (Formally Rasbian)
Raspberry Pi OS seems to be more for beginners, even the minimal version has stuff I don’t really need/want. It’s based on Debian, and I’d rather go with Ubuntu over Debian.
Ubuntu Core
Ubuntu Core is interesting as it’s really slim and I like that. However, it’s one of those weird Canonical things. It doesn’t support apt, and instead you have to use snap to install everything, no thanks.
Decision
I almost went with HypriotOS until I found this blog that explains how to setup ssh and check what the default username and password is. TLDR is that by default Ubuntu server has ssh setup with a default ubuntu:ubuntu credentials. The official Ubuntu docs don’t explain that well, and instead says to hookup a monitor.
So, since I didn’t need a monitor I went with Ubuntu server because it’s more of an “official” distro that I think will have more support than HypriotOS.