The Immutable Backup Fortress: Proxmox Backup Server in a Container on OpenSUSE MicroOS

Estimated reading time: 7 minutes

Recently, I hit a significant performance wall. My primary Proxmox node was gasping for air, primarily due to a severe RAM squeeze. One solution I considered was running Proxmox Backup Server in a Container instead of a VM. I realized that running the Proxmox Backup Server (PBS) as a VM on the very same host it was protecting was a bad idea. Not only did it consume precious resources, but it also created an annoying circular dependency.

While my actual backup data resided safely on a separate NFS target, the Backup Server itself was running as a VM on the host it protected. If that node went down, I still had the data, but I lost the tool to access it. Recovery would mean provisioning a new node and restoring the PBS LXC from a cold backup. Only then could I finally restore my production VMs. That is a “chicken-and-egg” scenario I simply didn’t want anymore (although I did test this worst case scenario and it was a doable one). Consequently, I decided to offload my backup infrastructure to my dedicated Lenovo ThinkCentre M92p Tiny running OpenSUSE MicroOS.

I wanted an immutable foundation for my backups. By using Podman Quadlets (instead of Docker Compose), I could treat the PBS container as a native systemd service. This migration wasn’t just about freeing up memory. It was about building a reproducible, automated, and “bulletproof” backup fortress. This is the story of how I transformed a complex legacy project into MicroOS-PBS: a streamlined toolkit with a package-based deployment, a full CI/CD pipeline, and a “Stubborn Tinkerer” build strategy.

Why Proxmox Backup Server in a Container on MicroOS?

To address the usual elephant: “Rámon, why not just keep it in a VM?”

• Resource Efficiency: Moving PBS off my main node immediately freed up several gigabytes of RAM for my other services.
• Risk Mitigation: Separating the backup server from the source host eliminates the circular dependency.
• Immutability: MicroOS ensures my backup host’s root filesystem stays clean and read-only.
• Container-First: Podman Quadlets integrate containers directly into systemd, treating them as first-class citizens.
• Automation-Ready: I built this project as a toolkit to handle everything from local testing to remote deployment.
• Broad Compatibility: While my primary goal was a native MicroOS setup with Quadlets, I’ve added instructions to the README to make this work seamlessly with standard Docker or Podman Compose environments.

My Checklist for a Bulletproof Container

As is my custom, I defined my requirements for this Proxmox Backup Server in a Container before touching a single configuration file.

• Immutable Root: No configuration drift allowed on the host.
• Systemd Integration: Container managed via a Quadlet.
• NFS Datastore: Mounted via systemd with explicit unit dependencies.
• UID Consistency: Ensuring the backup user (UID 34) matches between host and container.
• Multi-Arch Support: The build system must support both amd64 and aarch64 (for futureproofing)
• Validated Recovery: Automated integration tests that perform a “real” backup.

The Journey: From Rust Pain to Package Pleasure

I started by forking an existing project (originally by Kamil Trzciński) that built Proxmox Backup Server in a Container from source. However, I quickly hit a wall.

The Symptom: Circular dependencies and patch failures in proxmox-perl-rs.

The Cause: The upstream source changed faster than the manual patches could keep up.

The Solution: I ripped out the entire source-compilation infrastructure (over 2,000 lines of code in deleted scripts!) and pivoted to a package-based build using the official Proxmox Trixie repositories. Consequently, my build time dropped from one hour to just five minutes on my Lenovo ThinkCentre M92p Tiny (Intel i5-3470T (4) @ 3.60 GHz) and on my Lenovo Yoga Pro 9 16IMH9 (Intel Core Ultra 9 185H (22) @ 5.10 GHz) it just took a bit over one and a halve minute!

The Refactored Dockerfile

The Dockerfile needed some extensive refactoring to use a slim Debian Trixie base. I kept the original runit service supervision but streamlined the installation process to pull pre-compiled packages. This dramatically simplified maintenance while preserving the robust multi-process architecture (API, proxy, cron) that PBS requires.

# Excerpt from Dockerfile.build
RUN apt-get update && apt-get install -y \
    proxmox-backup-server \
    proxmox-backup-client \
    runit \
    ssmtp \
    cron \
    && rm -rf /var/lib/apt/lists/*

ADD dockerfiles/pbs/ /etc/proxmox-backup-default/
ADD dockerfiles/runit/ /runit/
CMD ["runsvdir", "/runit"]

Internal Service Hardening (Runit)

Inside the container, I don’t just “start” the API. The existing runit scripts act as safety rails.

1. Mountpoint Verification: The API startup script checks if /etc/proxmox-backup, /var/lib/proxmox-backup, and /var/log/proxmox-backup are actual mountpoints. If not, it warns you. This prevents you from accidentally writing data to the container’s ephemeral layer.
2. Config Initialization: If the config directory is empty (first run), it copies the default configuration from /etc/proxmox-backup-default/.
3. Lock Cleaning: It actively recycles stale lock files (rm /etc/proxmox-backup/.*.lck) that might persist after an unclean shutdown.

The Refactored Build Engine: release.bash

The original repository had a complex build engine. So I also heavily refactored release.bash to focus purely on Docker/Podman buildx operations. Then I added automatic architecture detection to support aarch64. In addition, I implemented a new deploy mode that pipes the built image directly to a remote host via SSH. This bypasses the need for a registry during development.

# The 'deploy' mode piping logic
$DOCKER_CMD save "$RELEASE_IMAGE_TAG" | ssh "$REMOTE" "$DOCKER_CMD load"

Configuring the Immutable Fortress

MicroOS requires a different mindset for persistence. All configuration and logs must live on writable subvolumes like /var.

The Verbatim Quadlet

I created this Quadlet file (proxmox-backup-server.container) to map the container to MicroOS standards. Note the :Z for private SELinux relabeling and the mandatory Tmpfs=/run mount.

[Unit]
Description=Proxmox Backup Server
After=network-online.target local-fs.target

[Container]
Image=ghcr.io/ramonvanraaij/proxmox-backup-server:latest
PodmanArgs=--hostname=MicroOS-PBS
Network=host
PublishPort=8007:8007

# Persistence (UID 34 ownership handled by setup script)
Volume=/var/lib/config/pbs:/etc/proxmox-backup:Z
Volume=/var/lib/data/pbs:/var/lib/proxmox-backup:Z
Volume=/var/log/pbs:/var/log/proxmox-backup:Z

# Capabilities
AddCapability=SYS_RAWIO

# Mount /run as tmpfs (Required by PBS for shared memory/locking)
Tmpfs=/run

[Service]
Restart=always

[Install]
WantedBy=multi-user.target

The “Stubborn Tinkerer” Toolkit

To make this a true “MicroOS Toolkit”, I wrote a new suite of Bash scripts from scratch to handle the entire lifecycle.

1. Local Build (local_build.bash)

Before I even think about a remote host, I build locally (well, to be honest, I went with the build on MicroOS path first…). This script detects the container engine (Podman or Docker) and builds the image using the same parameters as the CI pipeline. This ensures that the environment I test on my workstation matches what will eventually run in production.

2. Remote Build Resilience (build_on_microos.bash)

Sometimes, local storage drivers (like overlayfs on ZFS) cause build issues. I wrote this script to pack the repository, upload it to the MicroOS host, and trigger a build. Crucially, it runs the build inside a detached screen session. If my VPN drops or SSH times out, the build continues. Consequently, I can simply reconnect and tail the log.

3. Integration Testing (local_test.bash)

I created this script to be the heart of the project’s reliability. It orchestrates a full multi-container lifecycle.

• Dynamic Fingerprinting: It starts the server and executes proxmox-backup-manager cert info to grab the SHA256 fingerprint dynamically.
• Chunk Store Optimization: To speed up tests, I manually initialize the chunk store structure (0000-ffff). This is significantly faster than the PBS manager process for initial creation.$DOCKER_CMD exec -u root "$CONTAINER_NAME" bash -c 'printf "%04x\n" {0..65535} | xargs mkdir -p'

4. Artifact Cleanup (local_cleanup.bash)

A good tinkerer always cleans up. This script handles the teardown of the test environment. It removes the dedicated network, stops the containers, and purges the data directories. In addition, it prompts to remove the locally built images to keep your workstation tidy.

Deployment: The Interactive Setup Wizard

To make deployment painless, I wrote setup_microos.bash that will setup the Proxmox Backup Server in a Container on the MicroOS host. It is an interactive wizard that handles SSH (including jumphosts), firewall rules, and automated NFS mounting.

NFS Datastore Magic

The script generates a native systemd .mount unit for your NAS. It then injects a Requires= and After= dependency into the Quadlet. If the NAS is down, the PBS container won’t start. This prevents potential data corruption or filling up the local root disk.

# Excerpt from setup_microos.bash
LOCAL_MOUNT_UNIT_NAME=$(systemd-escape --path --suffix=mount "$LOCAL_MOUNT_POINT")
sed -i "s/^After=.*/& $LOCAL_MOUNT_UNIT_NAME/" "$QUADLET_TMP"
sed -i "/^\[Unit\]/ a Requires=$LOCAL_MOUNT_UNIT_NAME" "$QUADLET_TMP"

Automation: The RC to Stable Strategy

I completely replaced the original CI/CD with a new GitHub Actions workflow that follows a strict “Promotion” logic.

1. Upstream Monitoring: Every 24 hours, a script curls the Proxmox git summary to find the latest version string.
2. Release Candidate: If a new version is found, it’s tagged as an RC (e.g., v4.1.2-1-RC) and pushed to GHCR.
3. The Gatekeeper: The CI pipeline runs local_test.bash. If the backup fails, the image never publishes.
4. Promotion: Only after manual verification do I trigger the “Promote” workflow, which re-tags the image as latest and creates a stable GitHub Release.

Final Thoughts

The migration was a success 🙂 My Proxmox node is lighter, and my backups are now handled by a Proxmox Backup Server in a Container on a dedicated, immutable MicroOS host. By using OpenSUSE MicroOS and Podman Quadlets, I have achieved a level of stability that standard VMs simply can’not ‘t match.

The MicroOS-PBS project is open-source, give it a try. It sips power, protects itself with immutable snapshots, and ensures backups are always valid.

Stay pragmatic, and keep your backups offshore!

Buy me a coffee 🙂
If you found this post helpful, informative, or if it saved or made you some money, consider buying me a coffee. Your support means a lot and motivates me to keep writing.
You can do so via bunq.me (bunq, iDeal, Bankcontact and Credit- or Debit cards) or PayPal (PayPal and Credit- or Debit cards). Thank you!

Disclaimer
The blog posts, guides, and scripts provided on this website are for informational and educational purposes only. They are provided “as is” and without any warranty of any kind, either express or implied, including, but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.
By using the information or scripts from this blog, you agree that I am not liable for any direct, indirect, incidental, consequential, or any other damages or losses arising from the use of or inability to use the information, scripts, or instructions contained herein. You assume full responsibility for any and all risks associated with the use of this content.
The blog posts, guides, and pages may contain referral/affiliate links. If you make a purchase through these links, I may receive a commission at no additional cost to you.