With the addition of source code PGP signature checking to makepkg, I have began noticing just how many projects release their source code without any form of verification available. Or even if some form of verification is provided, it is done in a way that absolutely fails (e.g. llvm which was signed by a key that was not even on any keyservers meaning it could not be verified). If code security fails at this point, actually signing packages and databases at a distribution end-point instills a bit of a false sense of security.
To assess how readily validated upstream source code is, I did a survey of what I would consider the “core” part of any Linux distribution. For me, that basically means the packages required to build a fairly minimal booting system. This is essentially the package list from Linux From Scratch with a few additions that I see as needed…
For each source tarball I asked the following questions: 1) Is a PGP signature available and is the key used for signing readily verified? 2) Are checksum(s) for the source available and if they are only found on the same server as the source tarball, are they PGP signed? The packages are rated as: green – good source verification; yellow – verification available but with concerns; red – no verification. Apologies to any colour-blind readers, but the text should make it clear which category each package is in…
| Package | Verification |
|---|---|
| autoconf-2.68 | PGP signature, key ID in release announcement, key readily verifiable. |
| automake-1.11.3 | PGP signature, key used to sign release announcement. |
| bash-4.2.020 | PGP signature for release and all patches, link to externally hosted key from software website. |
| binutils-2.22 | PGP signature, key used to sign release announcement (containing md5sums). |
| bison-5.2 | PGP signature, key ID in release announcement, externally hosted keyring provided to verify key. |
| bzip2-1.0.6 | MD5 checksum provided on same site as download. |
| cloog-0.17.0 | MD5 and SHA1 checksums in release announcement posted on off-site list. |
| coreutils-8.15 | PGP signature, key used to sign release announcement. |
| diffutils-3.2 | PGP signature, key used to sign release announcement. |
| e2fsprogs-1.42.1 | PGP signature, key readily verifiable. |
| fakeroot-1.18.2 | MD5, SHA1 and SHA256 checksums provided in PGP signed file, key readily verifiable. |
| file-5.11 | No verification available. |
| findutils-4.4.2 | PGP signature, link to externally hosted key in release announcement. |
| flex-2.5.35 | No verification available. |
| gawk-4.0.0 | PGP signature, key difficult to verify. |
| gcc-4.6.3 | MD5 and SHA1 checksums provided in release email. MD5 checksum provided on same site as download. |
| gdbm-1.10 | PGP signature, key ID in release announcement (with MD5 and SHA1 checksums), key readily verifiable. |
| gettext-0.18.1.1 | PGP signature, key readily verifiable. |
| glibc-2.15 | No release tarball, download from git (PGP signature available when release tarball is made). |
| gmp-5.0.4 | PGP signature, key ID and SHA1 and SHA256 checksums on same site as source, key difficult to verify otherwise. |
| grep-2.11 | PGP signature, key used to sign release announcement. |
| groff-1.21 | PGP signature, key difficult to verify. |
| grub-1.99 | PGP signature, key used to sign release announcement. |
| gzip-1.4 | PGP signature, key used to sign release announcement. |
| iana-etc-2.30 | No verification available. |
| inetutils-1.9.1 | PGP signature, key readily verifiable. |
| iproute-3.2.0 | PGP signature, key readily verifiable. |
| isl-0.09 | No verification available. |
| kbd-1.15.3 | File size available in file in same folder as source. |
| kmod-0.05 | PGP signature, key readily verifiable. |
| less-444 | PGP signature, key posted on same site as download, key difficult to verify otherwise. |
| libarchive-3.0.3 | No verification available. |
| libtool-2.4.2 | PGP signature, key readily verifiable, MD5 and SHA1 checksums in release email. |
| linux-3.2.8 | PGP signature, key readily verifiable. |
| m4-1.4.16 | PGP signature, key used to sign release announcement. |
| make-3.82 | PGP signature, key used to sign release announcement. |
| man-db-2.6.1 | PGP signature, key used to sign release announcement. |
| man-pages-3.35 | PGP signature, key readily verifiable. |
| mpc-0.9 (libmpc) | PGP signature, key readily verifiable. |
| mpfr-3.1.0 | PGP signature, key readily verifiable. |
| ncurses-5.9 | PGP signature, key used to sign release announcement. |
| openssl-1.0.0g | PGP signature, key readily verifiable. |
| pacman-4.0.2 | PGP signature, key readily verifiable. |
| patch-2.6.1 | PGP signature, key difficult to verify. |
| pcre-8.30 | PGP signature, key readily verifiable. |
| perl-5.14.2 | MD5, SHA1, SHA256 checksums provided on same site as download. |
| pkg-config-0.26 | No verification available. |
| ppl-0.12 | PGP signature, key readily verifiable. |
| procps-3.2.8 | No verification available. |
| psmisc-22.16 | No verification available. |
| readline-6.2.002 | PGP signature for release and all patches, link to externally hosted key from software website. |
| sed-4.2.1 | PGP signature, key difficult to verify. |
| shadow-4.1.5 | PGP signature, key readily verifiable. |
| sudo-1.8.4p4 | PGP signature, key difficult to verify. |
| sysvinit-2.88 | PGP signature, key difficult to verify. |
| tar-1.26 | PGP signature, key used to sign release announcement. |
| texinfo-4.13a | PGP signature, key difficult to verify. |
| tzdata-2012b | Many checksums provided in release announcement. |
| udev-181 | PGP signature, key readily verifiable. |
| util-linux-2.21 | PGP signature, key readily verifiable. |
| which-2.20 | No verification available. |
| xz-5.0.3 | PGP signature, key difficult to verify. |
| zlib-1.2.6 | MD5 checksum provided on same site as download (although download mirrors available). |
Note that some of these packages have additional methods of verification available (e.g. those that are PGP signed may also provide checksums and file sizes), but I stopped looking once I found suitable verification. When I label a key as “readily verifiable”, that means it is either signed by keys I trust, that it is used to sign emails that I can find or it is posted on the developers personal website (which must be different from where the source code is hosted). I personally found my preferred method of verification was packages whose release announcements were signed by the same key as the source.
While you might look at that table and think there is a lot of green (and yellow) there so everything is in reasonable shape, it is important to note that the majority of these are GNU software and all GNU software is signed. Also, 15% of the packages in that list have no source verification at all. From some limited checking, it appears the situation quickly becomes worse as you move further away from this core subset of packages needed for a fairly standard Linux system, but I have not collected actual numbers to back that up yet.
Thanks Allan, really informative and different type of post.
At the very least checksums should be provided for each package, as they are much easier to handle, both for the developers and the users, than say pgp signatures.
Except checksums provide zero security, so they aren’t really that usefull 😉
Checksums are fine (there are even a few packages given a “green” rating with only checksums available), but there are a few things that need satisfied then. Firstly, MD5 only is not secure and SHA1 is debatable. So providing multiple checksums is the best idea. Secondly, do not provide them only in the same place as the download. That provides no security as they can be changed at the same time as the source. Posting multiple checksums and the filesize in the release email is a good compromise if PGP signing is really objected to for some reason.