Replicate your system with self-hosted Arch Linux metapackages

I have spent a lot of time spent over the years to tweak my system to my taste. Things like Openbox, Vim, Zsh, Tmux have a myriad of options and plugins that I put effort into configuring, and also I have my collection of installed programs whose configuration files I tuned.

When it comes to replicating my setup in a new system, such as my work computer, tablet PC, or a VM for instance it is time consuming and off-putting to go through all that process again and again. On top of that, personalization is an endless process so we have the problem of synchronizing future changes to all our devices.

Just as many other Arch Linux users, I ended up writing my installation script. I self host my configuration scripts and files in my home git instance, and I used to keep an automatically up to date list of installed packages as explained in this other post.

While this is a step forward, it is not ideal when you have a bunch of devices to synchronize. I don’t want all my packages in all my systems and some parts of the configuration may vary.

To address the second issue, I normally copy blindly the main configuration file, which then sources a local configuration file that is synchronized separately.

For instance, a .vimrc file can end with a source .vimrc.local file, or sometimes I set up a conf.d kind of directory where I can add or remove files from.

I was inspired by this post to try the meta package approach to solve both problems: replicating my packages and configuration files, and synchronizing my changes. All my credit goes to the author, only that in my case I prefer to self host my packages.

Arch metapackages

A metapackage is just a package that doesn’t actually package files. Instead we use the metadata to do things like grouping dependencies together, specifying package conflicts and so on.

Whenever we install the metapackage, all the dependencies will be pulled in as well. Pretty easy. This is not much better than the previous approach: we merely changed from a script installing a package list to embedding the package list in a metapackage that the package manager is able to understand.

The interesting part is that we can also add our configuration files to the package. When we update the package those files will be updated as well without any additional effort. Also, pacman is aware of what files belong to what package, so it will warn us if we have modified them and it will save a copy named pacnew file so we can compare them. We can configure certain configuration files to be backed up automatically as well, instead of simply overridden.

I also package my many custom scripts, and wrappers that live in my ~/.bin folder. We can package any files we want.

It is also useful to create relationships between metapackages when this makes sense. For instance, I always want to install vim alongside with ctags and cscope, or zsh with zsh-suggestions, powerline and all the cool stuff, together with their respective dotfiles. So I have them grouped in the nacho-vim and nacho-zsh metapackages, and instead of adding vim and zsh as dependencies of nacho-base I do add nacho-vim and nacho-zsh in their place.

Finally we can add custom post ugrade and post install hooks to run operations such as enabling a service we just installed or adjusting permissions of the files that pacman copied over.

The great thing about metapackages is that we get the functionality of a dotfile manager without requiring additional tools. Just running pacman -Syu as we normally do will also update our configuration files and an new packages we might have added to our favorite setup.

Some of this can be done with package groups, but the important feature I am after is that all my systems upgrade automatically. From the Arch Wiki

The advantage of a meta package, compared to a group, is that any new member packages will be installed when the meta package itself is updated with a new set of dependencies. This is in contrast to a group where new group members will not be automatically installed.

Creating our metapackages

No need to explain again what is already well explained. Packages are built from a PKGBUILD file with the makepkg command. You can check out the wiki about how to create packages.

There are many ways to skin a cat. I will only mention a couple things that I like to do.

I like to use this structure

The files to be included need to be in the same directory as the PKGBUILD file. zsh.install is where I define my post-upgrade and post-update hooks

zsh.txt is only the list of dependencies

I follow the same pattern for all my metapackages. First, I source this simple file to avoid repetition

This way I only need to set the git tag in the repository and the package versions will be derived from it. Also I can add the .txt and the .install files if I want to.

Finally, a simplified PKGBUILD can look something like this

Once we have this, 95% of the maintenance is simply updating the package list with the techniques from the last post on this topic.

Self hosting our Arch repository

So we already have our own packages and metapackages in place, now we need to synchronize them between our machines.

A repository is no more than a collection of packages and a database file that contains the package list and metadata.

Once we have our packages we generate the database file with the repo-add helper

Then, we can place packages and database in different places that pacman understands. This can be in our own filesystem (useful to install packages from USB or CD), HTTP or FTP for example.

If we want to use something like a Raspberry Pi all we need is to statically serve a directory with these files. This would suffice

However, since I am already using Apache for NextCloudPi, I added a new Virtual Host on port 8888

Now we have all the pieces so let’s just put them together. Whenever I change my package list in one of the .txt files, I run the following script

Generate packages, then create the database file and send it all to our statically hosted directory.

Finally we add to our installation script a line where we point pacman to our personal repository. We need the following in all our machines

Next time we upgrade our system our personal repository will also be synchronized

There we go, that was pretty easy. Whenever we do our routine pacman -Syu the database will be synchronized and if there’s a new version of the package it will be downloaded and installed, synchronizing all our configuration and new dependencies in the process.

No doubt this will keep evolving over time. Hopefully it gave you some ideas.

Author: nachoparker

Humbly sharing things that I find useful [ github dockerhub ]

Leave a Reply

Your email address will not be published. Required fields are marked *