-
Easy: Build any Debian/Ubuntu package in one single command. If you don't have the package or source already downloaded, it will be done automatically. Also, a build container (both Podman and Docker supported) is automatically built using the target Debian or Ubuntu suite in
debian/changelogand with exact build dependencies fromdebian/control. Users are spared from having to manually prepare ahead or maintain sbuild root file systems or alike. The command output is (relatively) easy to read, and it also teaches users about Debian packaging in context. -
Fast: Container layer caching is utilized to make builds and re-builds blazingly fast. Debian packages that support ccache build even faster.
-
Secure: Builds happen inside hermetic containers with no network access. This ensures all dependencies are properly managed and the built binaries equally trustworthy as the source code it was built from. This also protects the host system from getting polluted with extra development libraries, and adds an extra layer of protection to prevent anything malicious in the source code from accessing secrets on the host system. The additional logs provided by Debcraft also helps audit changes in Debian package sources and build artifacts.
Feedback welcome! Debcraft is still in early development and your feedback is greatly appreciated. Bug reports at https://salsa.debian.org/debian/debcraft/-/issues are welcome on for example:
- Documentation: Is it easy to start using Debcraft? How could the documentation be clarified further?
- Structure: Are you able to productively use Debcraft? Is the tool easy to reason about? Do the features and code architecture make sense?
- Compatibility: Does Debcraft work on your laptop and with your favorite Linux distro / release / package?
debcraft build <package>debcraft build --distribution bullseye <package>debcraft build <path to sources>debcraft shell <path to sources>debcraft build --pull <path to sources>DEBCRAFT_PPA=ppa:otto/ppa debcraft release <path to sources>DEB_BUILD_OPTIONS="parallel=4 nocheck noautodbgsym" debcraft build <package>$ debcraft --help
usage: debcraft [options] <build|validate|test|release|shell|prune> [<path|pkg|srcpkg|dsc|git-url>]
Debcraft is a tool to easily build .deb packages. The 'build' argument accepts
as a subargument any of:
* path to directory with program sources including a debian/ subdirectory
with the Debian packaging instructions
* path to a .dsc file and source tarballs that can be built into a .deb
* Debian package name, or source package name, that apt can download
* git http(s) or ssh URL that can be downloaded and built
The commands 'validate' and 'release' are intended to be used to finalize
a package build. The command 'shell' can be used to pay around in the container
and 'prune' will clean up temporary files by Debcraft.
In addition to parameters below, anything passed in DEB_BUILD_OPTIONS will also
be honored (for example DEB_BUILD_OPTIONS='parallel=4 nocheck noautodbgsym').
Note that Debcraft builds never runs as root, and thus packages with
DEB_RULES_REQUIRES_ROOT are not supported.
optional arguments:
--build-dirs-path Path for writing build files and artifacts (default: parent directory)
--distribution Linux distribution to build in (default: debian:sid)
--container-command container command to use (default: podman)
--pull ensure container base is updated
--copy perform the build on a copy of the package directory
--clean ensure sources are clean
-h, --help display this help and exit
--version display version and exit
To gain more Debian Developer knowledge, please read
https://www.debian.org/doc/manuals/developers-reference/
and https://www.debian.org/doc/debian-policy/
$ debcraft build
Running in path ~/entr/entr that has Debian package sources for 'entr'
Use 'podman' container image 'debcraft-entr-debian-sid' for package 'entr'
Building container 'debcraft-entr-debian-sid' in '~/entr/debcraft-container-entr' for build ID '1705046461.1964390+debian.latest'
STEP 1/12: FROM debian:sid
...
COMMIT debcraft-entr-debian-sid
--> d9975574b37
Successfully tagged localhost/debcraft-entr-debian-sid:latest
d9975574b37ac5ff5fd1874ee935a8d35152126798d339a53c076cd0461c9354
Previous build was in ~/entr/debcraft-build-entr-1705046398.1964390+debian.latest
Building package at ~/entr/debcraft-build-entr-1705046461.1964390+debian.latest
Running 'dpkg-buildpackage --build=any,all' to create .deb packages
gbp:info: Performing the build
dpkg-buildpackage: info: source package entr
dpkg-buildpackage: info: source version 5.5-1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Otto Kekäläinen <[email protected]>
dpkg-source --before-build .
...
make -j4 "INSTALL=install --strip-program=true"
make[1]: Entering directory '/tmp/build/source'
cc -g -O2 -ffile-prefix-map=/tmp/build/source=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -D_GNU_SOURCE -D_LINUX_PORT -isystem /usr/include/bsd -DLIBBSD_OVERLAY -Imissing -Wdate-time -D_FORTIFY_SOURCE=2 -DRELEASE=\"5.5\" -Wl,-z,relro -Wl,-z,now -lpthread -Wl,-z,nodlopen -Wl,-u,libbsd_init_func -lbsd-ctor -lbsd missing/kqueue_inotify.c entr.c -o entr
entr.c: In function ‘print_child_status’:
entr.c:289:9: warning: ignoring return value of ‘write’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
289 | write(STDOUT_FILENO, buf, len);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
entr.c: In function ‘run_utility’:
entr.c:433:17: warning: ignoring return value of ‘realpath’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
433 | realpath(leading_edge->fn, arg_buf);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[1]: Leaving directory '/tmp/build/source'
dh: command-omitted: The call to "dh_auto_test -O--buildsystem=makefile" was omitted due to "DEB_BUILD_OPTIONS=nocheck"
create-stamp debian/debhelper-build-stamp
dh_testroot -O--buildsystem=makefile
dh_prep -O--buildsystem=makefile
debian/rules override_dh_auto_install
make[1]: Entering directory '/tmp/build/source'
dh_auto_install -- PREFIX=/usr
make -j4 install DESTDIR=/tmp/build/source/debian/entr AM_UPDATE_INFO_DIR=no "INSTALL=install --strip-program=true" PREFIX=/usr
make[2]: Entering directory '/tmp/build/source'
install entr /tmp/build/source/debian/entr/usr/bin
install -m 644 entr.1 /tmp/build/source/debian/entr/usr/share/man/man1
make[2]: Leaving directory '/tmp/build/source'
make[1]: Leaving directory '/tmp/build/source'
dh_installdocs -O--buildsystem=makefile
dh_installchangelogs -O--buildsystem=makefile
dh_installman -O--buildsystem=makefile
dh_installsystemduser -O--buildsystem=makefile
dh_perl -O--buildsystem=makefile
dh_link -O--buildsystem=makefile
dh_strip_nondeterminism -O--buildsystem=makefile
dh_compress -O--buildsystem=makefile
dh_fixperms -O--buildsystem=makefile
dh_missing -O--buildsystem=makefile
dh_dwz -a -O--buildsystem=makefile
dh_strip -a -O--buildsystem=makefile
dh_makeshlibs -a -O--buildsystem=makefile
dh_shlibdeps -a -O--buildsystem=makefile
dh_installdeb -O--buildsystem=makefile
dh_gencontrol -O--buildsystem=makefile
dh_md5sums -O--buildsystem=makefile
dh_builddeb -O--buildsystem=makefile
dpkg-deb: building package 'entr' in '../entr_5.5-1_amd64.deb'.
dpkg-genbuildinfo --build=binary -O../entr_5.5-1_amd64.buildinfo
dpkg-genchanges --build=binary -O../entr_5.5-1_amd64.changes
dpkg-genchanges: info: binary-only upload (no source code included)
dpkg-source --after-build .
dpkg-source: info: unapplying fix-spelling.patch
dpkg-source: info: unapplying system-test-fixes.patch
dpkg-source: info: unapplying debug-system-test.patch
dpkg-source: info: unapplying kfreebsd-support.patch
dpkg-source: info: unapplying libbsd-overlay.patch
dpkg-buildpackage: info: binary-only upload (no source included)
Cache directory: /.ccache
Config file: /.ccache/ccache.conf
System config file: /etc/ccache.conf
Stats updated: Fri Jan 12 08:01:04 2024
Local storage:
Cache size (GiB): 0.0 / 5.0 ( 0.00%)
Files: 0
Hits: 0
Misses: 0
Reads: 0
Writes: 0
Create lintian.log
Create filelist.log
Create maintainer-scripts.log
Create diffoscope report comparing to previous build
Build completed in 10 seconds and created:
total 72K
32K diffoscope.html
4.0K entr_5.5-1_amd64.build
8.0K entr_5.5-1_amd64.buildinfo
4.0K entr_5.5-1_amd64.changes
20K entr_5.5-1_amd64.deb
4.0K filelist.log
0 lintian.log
Artifacts at ~/entr/debcraft-build-entr-1705046461.1964390+debian.latest
To compare build artifacts with those of previous similar build you can use for example:
meld ~/entr/debcraft-build-entr-1705046398.1964390+debian.latest ~/entr/debcraft-build-entr-1705046461.1964390+debian.latest &
browse ~/entr/debcraft-build-entr-1705046461.1964390+debian.latest/diffoscope.html
Debcraft in Debian unstable ("sid") and Ubuntu 24.10 "Oracular" since July 2024. If running such a new version one can simply install with:
apt install debcraft
If you do not have Podman nor Docker installed, it will install Podman by default.
To use the latest development version, simply clone the git repository and link the
script from any directory you have in your $PATH, such as $HOME/bin
git clone https://salsa.debian.org/debian/debcraft.git
cd debcraft
make install-local
The core design principles are:
- Be opinionated, make the correct thing automatically without asking user to make too many decisions, and when full automation is not possible, steer users to follow the best practices in software development.
- Use git, git-buildpackage and quilt as Debian is on a path to standardize on them as shown by the Debian Trends website.
- Use Linux containers (not chroot like traditional Debian tools do) for improved isolation, security and reproducibility.
- Create build environment containers on the fly so users don't need to plan ahead what containers or chroots to have.
- Be extremely fast in what users are likely to spend most of their time on: rebuilds.
- Store logs and artifacts from builds and help users review changes between builds and package versions to maximize users' understanding of how their changes affect the outcome.
- Don't expect users to run the latest version of Debian or even Debian or Ubuntu at all. The barrier to run Debcraft should be as low as possible, so that anyone can participate in debugging Debian package builds and improving them.
- Encourage users to collaborate and submit improvements upstream and on Salsa instead of just making their own private Debian packages.
- Teach users about the Debian policy gradually and in context, so that over time users grow towards Debian maintainership.
This project is open source and contributions are welcome! The project maintains a promise that the initial review will happen in 48h for all Merge Requests received. The code review will be conducted professionally and the code base aims to maintain a very high quality bar, so please reserve time to polish your code submission in a couple of review rounds.
The project is hosted at https://salsa.debian.org/debian/debcraft with mirrors at https://gitlab.com/ottok/debcraft and https://github.com/ottok/debcraft.
Debcraft does not intend to replace well working existing tools like
git-buildpackage,
but rather build upon them, making the overall process of as easy as possible.
Current development focus is to make the debcraft build as easy and
efficient as possible and it is already quite capable. The release is also
already fully usable.
The validate command only does static testing for the source directory without
modifying anything. Something like polish to run
lintian-brush
and other tools to automatically improve the package source code might be added
later, or a command to run dynamic tests on the built binaries (create local
repo, run piuparts, autopkgtests, some of the Salsa-CI tests locally etc).
The prune command currently does nothing.
To help Debian Developers with recurring work, a command such as update to
automatically import a new upstream version might also be implemented later.
Search for @TODO comments in the sources to see which parts are incomplete and
pending to be written out.
Bash was specifically chosen as the main language for this tool in order to keep the code contribution barrier as low as possible. Additionally, as Debcraft mainly builds upon invoking other programs via their command-line interface, using Bash scripting helps keep the code base small and lean compared to using a proper programming language to run tens of subshells. If the limitations of Bash (e.g. lack of proper testing framework, limited control of output with merely ANSI codes, overly simplistic error handling etc) starts to feel limiting, parts of this tool might be rewritten in a fast to develop language like Python, Mojo, Nim, Zig or Rust.
Note that Bash is used to its fullest. There is no need to restrict functionality to POSIX compatibility as Debcraft will always run on Linux using Linux containers anyway.
Despite being written with Bash, Debcraft still aims to highest possible code
quality by enforcing that the code base in Shellcheck clean along with other
applicable static testing, such as spellchecking. While running set -e is in
effect to stop execution on any error unless explicitly handled.
The Bash code should avoid spawning subshells if it can be avoided. For example
us in-line Bash parameter
substitution instead
of spawning sed commands in subshells.
There are no fixed release dates or fixed milestone scopes. Maintaining high quality trumps other priorities. This tool is intended to automate Debian packaging work that has existed for decades, and the tools should be robust enough to stand the test of time and serve for decades to come.
It is more important for code to be easy to read and reason about than quick to
write. Therefore, always spend a bit of extra effort to make things clear and
easy to read. For example, write --parameter instead of just -p when
possible. Most commands are also run with --verbose intentionally to expose to
users what is happening.
Automation in a developer tool does not mean that things should be hidden - in this tool automation is transparent, doing as much as possible on behalf of the user but still transparent about what is being done.
To help with ensuring the above about code quality, the project has both GitLab
CI for automatic testing and a simple make test command to run the same
testing locally while developing.
Why the name Debcraft? Because the name debuild was already taken. The 'craft' also helps setting users in the correct mindset, hinting towards that producing high quality Debian packages and maintaining operating system over many years and decades is not just a pure technical task, but involves following industry wisdoms, anticipating unknowns and hand-crafting and tuning things to be as perfect as possible.
Debcraft is free and open source software as published under GPL version 3.