Setting Up Fast Debian Package Builds Using Sbuild, Mmdebstrap and Apt-Cacher-Ng

In this post I will give a quick tutorial on how to set up fast Debian package builds using sbuild with mmdebstrap and apt-cacher-ng.

The usual tool for building Debian packages is dpkg-buildpackage, or a user-friendly wrapper like debuild, and while these are geat tools, if you want to upload something to the Debian archive they lack the required separation from the system they are run on to ensure that your packaging also works on a different system. The usual candidate here is sbuild. But setting up a schroot is tedious and performance tuning can be annoying. There is an alternative backend for sbuild that promises to make everything simpler: unshare. In this tutorial I will show you how to set up sbuild with this backend.

Additionally to the normal performance tweaking, caching downloaded packages can be a huge performance increase when rebuilding packages. I do rebuilds quite often, mostly when a new dependency got introduced I didn’t specify in debian/control yet or lintian notices a something I can easily fix. So let’s begin with setting up this caching.

Setting up apt-cacher-ng

Install apt-cacher-ng:

sudo apt install apt-cacher-ng

A pop-up will appear, if you are unsure how to answer it select no, we don’t need it for this use-case.

To enable apt-cacher-ng on your system, create /etc/apt/apt.conf.d/02proxy and insert:

Acquire::http::proxy "http://127.0.0.1:3142";
Acquire::https::proxy "DIRECT";

In /etc/apt-cacher-ng/acng.conf you can increase the value of ExThreshold to hold packages for a shorter or longer duration. The length depends on your specific use case and resources. A longer threshold takes more disk space, a short threshold like one day effecitvely only reduces the build time for rebuilds.

If you encounter weird issues on apt update at some point the future, you can try to clean the cache from apt-cacher-ng. You can use this script:

Setting up mmdebstrap

Install mmdebstrap:

sudo apt install mmdebstrap

We will create a small helper script to ease creating a chroot. Open ~/.local/bin/mmupdate and insert:

#!/bin/sh
mmdebstrap \
  --variant=buildd \
  --aptopt='Acquire::http::proxy "http://127.0.0.1:3142";' \
  --arch=amd64 \
  --components=main,contrib,non-free \
  unstable \
  ~/.cache/sbuild/unstable-amd64.tar.xz \
  http://deb.debian.org/debian

Notes:

  • aptopt enables apt-cacher-ng inside the chroot.
  • --arch sets the CPU architecture (see Debian Wiki).
  • --components sets the archive components, if you don’t want non-free pacakges you might want to remove some entries here.
  • unstable sets the Debian release, you can also set for example bookworm-backports here.
  • unstable-amd64.tar.xz is the output tarball containing the chroot, change accordingly to your pick of the CPU architecture and Debian release.
  • http://deb.debian.org/debian is the Debian mirror, you should set this to the same one you use in your /etc.apt/sources.list.

Make mmupdate executable and run it once:

chmod +x ~/.local/bin/mmupdate
mkdir -p ~/.cache/sbuild
~/.local/bin/mmupdate

If you execute mmupdate again you can see that the downloading stage is much faster thanks to apt-cacher-ng. For me the difference is from about 115s to about 95s. Your results may vary, this depends on the speed of your internet, Debian mirror and disk.

If you have used the schroot backend and sbuild-update before, you probably notice that creating a new chroot with mmdebstrap is slower. It would be a bit annoying to do this manually before we start a new Debian packaging session, so let’s create a systemd service that does this for us.

First create a folder for user services:

mkdir -p ~/.config/systemd/user

Create ~/.config/systemd/user/mmupdate.service and add:

[Unit]
Description=Run mmupdate
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=%h/.local/bin/mmupdate

Start the service and test that it works:

systemctl --user daemon-reload
systemctl --user start mmupdate
systemctl --user status mmupdate

Create ~/.config/systemd/user/mmupdate.timer:

[Unit]
Description=Run mmupdate daily

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target

Enable the timer:

systemctl --user enable mmupdate.timer

Now every day mmupdate will be run automatically. You can adjust the period if you think daily rebuilds are a bit excessive.

A neat advantage of period rebuilds is that they the base files in your apt-cacher-ng cache warm every time they run.

Setting up sbuild

Install sbuild and (optionally) autopkgtest:

sudo apt install --no-install-recommends sbuild autopkgtest

Create ~/.sbuildrc and insert:

# backend for using mmdebstrap chroots
$chroot_mode = 'unshare';

# build in tmpfs
$unshare_tmpdir_template = '/dev/shm/tmp.sbuild.XXXXXXXX';

# upgrade before starting build
$apt_update = 1;
$apt_upgrade = 1;

# build everything including source for source-only uploads
$build_arch_all = 1;
$build_arch_any = 1;
$build_source = 1;
$source_only_changes = 1;

# go to shell on failure instead of exiting
$external_commands = { "build-failed-commands" => [ [ '%SBUILD_SHELL' ] ] };

# always clean build dir, even on failure
$purge_build_directory = "always";

# run lintian
$run_lintian = 1;
$lintian_opts = [ '-i', '-I', '-E', '--pedantic' ];

# do not run piuparts
$run_piuparts = 0;

# run autopkgtest
$run_autopkgtest = 1;
$autopkgtest_root_args = '';
$autopkgtest_opts = [ '--apt-upgrade', '--', 'unshare', '--release', '%r', '--arch', '%a', '--prefix=/dev/shm/tmp.autopkgtest.' ];

# set uploader for correct signing
$uploader_name = 'Stephan Lachnit <stephanlachnit@debian.org>';

You should adjust uploader_name. If you don’t want to run autopkgtest or lintian by default you can also disable it here. Note that for packages that need a lot of space for building, you might want to comment the unshare_tmpdir_template line to prevent a OOM build failure.

You can now build your Debian packages with the sbuild command :)

Finishing touches

You can add these variables to your ~/.bashrc as bonus (with adjusted name / email):

export DEBFULLNAME="<your_name>"
export DEBEMAIL="<your_email>"
export DEB_BUILD_OPTIONS="parallel=<threads>"

In particular adjust the value of parallel to ensure parallel builds.

If you are new to signing / uploading your package, first install the required tools:

sudo apt install devscripts dput-ng

Create ~/.devscripts and insert:

DEBSIGN_KEYID=<your_gpg_fingerprint>
USCAN_SYMLINK=rename

You can now sign the .changes file with:

debsign ../<pkgname_version_arch>.changes

And for source-only uploads with:

debsign -S ../<pkgname_version_arch>_source.changes

If you don’t introduce a new binary package, you always want to go with source-only changes.

You can now upload the package to Debian with

dput ../<filename>.changes

Update Feb 22: Advanced options in mmdebstrap

Jochen Sprickerhof, who originally advised me to use the unshare backend, commented that one can also use --include=auto-apt-proxy instead of the --aptopt option in mmdebstrap to detect apt proxies automatically. He also let me know that it is possible to use autopkgtest on tmpfs (config in the blog post is updated) and added an entry on the sbuild wiki page on how to setup sbuild+unshare with ccache if you often need to build a large package.

Further, using --variant=apt and --include=build-essential will produce smaller build chroots if wished. On the contrary, one can of course also use the --include option to include debhelper and lintian (or any other packages you like) to further decrease the setup time. However, staying with buildd variant is a good choice for official uploads.

Update Aug 23: Building for experimental

mmdebstrap does not allow to create tarballs with packages from experimental. Luckily, we don’t actually need this, we can simply solve this with in sbuild by adding experimental as additional repository to our existing unstable tarball and changing the dependency resolver. Let’s create a new script ~/.local/bin/sbuild-experimental:

#!/bin/sh
sbuild --dist=experimental --extra-repository='deb http://deb.debian.org/debian experimental main contrib non-free' --build-dep-resolver=aspcud $@

Let’s not forget to make the file executable:

chmod +x ~/.local/bin/sbuild-experimental

The last step is to create a symlink for the experimental tarball pointing to our existing tarball:

ln -s -r ~/.cache/sbuild/unstable-amd64.tar.xz ~/.cache/sbuild/experimental-amd64.tar.xz

Thanks to josch and smcv on IRC for helping me out with this!

Resources for further reading

https://wiki.debian.org/sbuild
https://www.unix-ag.uni-kl.de/~bloch/acng/html/index.html
https://wiki.ubuntu.com/SimpleSbuild
https://wiki.archlinux.org/title/Systemd/Timers
https://manpages.debian.org/unstable/autopkgtest/autopkgtest-virt-unshare.1.en.html
Thanks for reading!

howtoDebiandebootstrapsystemd