[allan@server]$ rm sample-apps/
rm: cannot remove `sample-apps/': Is a directory
[allan@server]$ rmdir sample-apps/
rmdir: failed to remove `sample-apps/': Not a directory
Normally when you write an article and label it with part one, it is followed soon after with a part two. Well, a lot more than two years later, I discovered this draft…. So this is part two of me rambling about what I think it means to keep packages “vanilla”. See here for the first part in which I discussed patching. Looking at configure options and dependencies is probably less clear than patching, but lets see if I come to a conclusion in this wall of text!
As I said in the previous post, in an ideal world we could just do “./configure; make; make install” and all packages would build perfectly and interact with each other the way they are supposed to. I will attempt to categorize the various options that can be added to configure and by how they change the package. This will be mostly done by looking at examples from the packages I maintain (or now, packages that I used to maintain) for Arch Linux.
The first type of configure option that will (almost) always be used is setting paths for where various files are located. Most Linux distributions will build their packages with “--prefix=/usr” and perhaps several other configuration options to set file paths. Arch Linux is not a fan of the /libexec directory so uses --libexecdir=/usr/lib where needed. (It appears that moving away from that directory is becoming widespread these days, right after the draft FHS added it…) There used to be a lot of moving of man and info pages to the “right place”, but that is automatically done in most packages these days. I doubt that anyone would consider these types of configuration options to make a package non-vanilla, unless they were set to very extreme values.
The second set of configure flags are those that enable additional features. For example, GMP can be built using the configure flag --enable-cxx to enable C++ support. That builds an additional library and adds an extra header to the package, but does not alter the primary library. Similarly, using --enable-pcre16 and --enable-pcre32 when configuring PCRE adds 16 and 32 bit character support libraries. Given these options have no effect on the primary part of the software and just add completely separate parts to it, it would be hard to argue that such configuration options are not vanilla.
So lets move onto configuration options that actually alter the software. Lets start with glibc, binutils and gcc which all have
set in the Arch Linux packages. Is that vanilla? It can be argued that I am setting a value not considered by upstream, but clearly upstream thought allowing the package to set such a value was a good idea given there is a configuration option. So I’d say that is still vanilla.
How about Less which is configured with --with-regex=pcre and adds a dependency to the software. There is actually quite a number of possible values for this configuration option:
Looking at that list, I would suppose auto is the most vanilla, as that is what happens if you do not specify the option. But it is also the least deterministic, in that it will pick a different option based on what is installed on your system. In fact, on my system it picks “posix” by default, which was a moderate surprise to me. Given that these options are all provided by the software developer (and so should all be supported), I think you could make the case they all are vanilla. But I would be surprised to see (at least) the last five options used on any Linux system, so calling them vanilla is a stretch.
What is the conclusion? I suppose that configuration options are mostly put there to be used by the upstream developers so any use of them would be considered vanilla. But keep in mind what a software developer would expect. Picking strange configurations compared to what is usual for your operating system is likely to get you (virtual) weird looks from upstream, so that can not be considered a vanilla configuration.
Perhaps this part is more boring than the discussion of patching. But the second part of a trilogy is rarely the greatest. Part 3 should appear soon…
I am teaching my daughter about the world and all the different countries there are. She also likes receiving mail, so we came up with the idea of collecting postcards from all over the world.
If you would like to help me out by sending her a post card, all the details are on her website. Thanks!
This backfired on me today. It turns out that systemd-218 does not like an old i686 kernels (I had 3.13.7) and I could not log in. I am not the only one to notice this, so there may be a genuine bug there. It also seems that DO recovery console is a bit useless if you have a problem other than a network issue. Good thing I have DO system backups set to run every week (in a completely unrelated incident… I think one comment on one of my posts got lost and restoring from my more regular WordPress backup is too much effort).
So how to run a newer kernel – I have no idea what DO do in the background, but your installed kernel needs to match the one you select in the DO control panel. I think the kernel and initrd are stored externally and if this does not match the installed version, your system’s modules fail to load. This is how I know the rescue console is good when networking fails!
Kexec to the rescue! You can do a fancy systemd type thing (see Arch wiki page), but I just provided a script in /usr/bin/init. To do this, install kexec-tools and remove systemd-sysvcompat. Then paste this into /usr/bin/init:
kexec --load /boot/vmlinuz-linux --initrd=/boot/initramfs-linux.img --append="root=LABEL=DOROOT init=/usr/lib/systemd/systemd" &&
mount -o ro,remount / &&
Give that file 755 permissions, and all is done.
I’ll give credit where it is due. I had previously criticized Manjaro for holding back all package updates as this ignored security issues. But it appears that Manjaro has a new security policy, which means that packages that are rated as “Critical” or “High” in the Arch Security Advisories get pushed through their “quality assurance” process more quickly.
Posting so more people can get entertainment value from this email.
From: Daniel Skowroński <firstname.lastname@example.org>
I’d like to say you are moron if you were thinking that commiting 61ba5c961e4a3536c4bbf41edb348987a9993fdb to pacman was good idea!
I am Arch Linux user becouse it allows me to do virtually anything so I manage some of my arch-based servers from root account. I also have netbook with root account only for my own reasons. And you’ve destroyed not only my routine but also several programs that were depending on that. Screw you with reverting package version or using patch from aur – it’s supposed to work out-of-the-box even on Arch.
PS. You could have at least let anybody else sign-off that imbecile commit…
Nobody… Arch Linux has the latest glibc release (2.20) so is not affected by the GHOST bug, which was fixed upstream in glibc-2.18 (without realising the security implications).
For those that want to know more about security issues and updates in Arch Linux, check out the arch-security mailing list.
An alarming number of people have noticed, the pacman-4.2 release removed the --asroot option from makepkg. This means that you can no longer build packages as the root user. There are good reasons for this and the option was only included due to issue we had building under fakeroot (only the package() function gets fun under fakeroot these days, and there has been no issues with fakeroot in a while anyway).
Even if your PKGBUILD file is not malicious, there are good examples of when something goes wrong by accident. Remember the bumblebee bug that deleted /usr due to an extra space? Or just this week a steam bug that deletes a user home directory? Do you still want to run code as root? OK then… I am going to show you how not to!
Firstly, we need a build directory. I suggest /home/build. Putting this directory directly under /root will not work unless you want to relax its 700 permissions to allow the nobody user read/write access1. I suppose you could as you are running as root… but I will use /home/build. Create the directory and set permissions with the following:
chgrp nobody /home/build
chmod g+ws /home/build
setfacl -m u::rwx,g::rwx /home/build
setfacl -d --set u::rwx,g::rwx,o::- /home/build
Not that people running makepkg as root need to know what code is doing to run it… I’ll explain what is happening here. Firstly create a /home/build directory, make it owned by the nobody group and ensure that group has write permissions. Also add the sticky flag to the group permissions so all files created in that directory also are owned by the nobody group. Then we set ACLs to ensure all files and directories created in /home/build have group read/write permissions.
Now to building you package! Get you PKGBUILD in your new build directory and run makepkg as the nobody user. You can do this using su but using sudo has the advantage of being able to alias this command. Installing sudo does not create a security risk as you are running as root! You also do not need to configure anything as root will have full sudo permissions by default2. Build your package using:
sudo -u nobody makepkg
Done… I’d add “alias makepkg='sudo -u nobody makepkg” to your ~/.bashrc so you never have to type this again.
There is still a problem here. If you download and manually extract a package sourceball, or use an AUR helper such as cower to do so, the group write permissions get lost:
[root@arya build]# cower -d pacman-git
:: pacman-git downloaded to /home/build
[root@arya build]# ls -ld pacman-git/
drwxr-xr-x+ 2 root nobody 4096 Mar 21 2013 pacman-git/
Doing “chmod -R g+w pacman-git/” will fix this. There is probably a way to avoid this – at least when manually extracting the tarball, but I have no interest in figuring it out. Otherwise, it is a two line function.
And if this does not satisfy you, revert that patch that removed --asroot. It should still revert cleanly.
1 makepkg checks directory write permissions using the full path so fails if any parent directories are not writable. I guess this could be fixed if someone was interested.
2 Note that to have makepkg install missing dependencies and install your built package without being queried the password for the nobody user (which would be difficult to answer…), you will need to configure nobody to run sudo pacman without a password.
Both the pacman package manager and the makepkg tool for building packages verify files using PGP signatures. However, these two pieces of software do it using different keyrings. There seems to be a lot of confusion about this and misinformation is spreading at a rapid pace, so I’ll attempt to clarify it here!
Pacman Package File Signature Verification
By default, pacman is set-up to verify every package using a PGP signature. It has its own keychain for this purpose, located at /etc/pacman.d/gnupg/. This keychain is initialized during the Arch Linux install – a root key is created and the Arch Linux master keys are locally signed by the root key. The master keys sign all Arch Developer and Trusted User keys, creating an effective web-of-trust from your pacman root key to each of the packager keys allowing verification of package files.
If you want to allow the installation of package files from a non-official repository, you need to either disable signature verification (don’t do that…), or trust the packagers signing key. To do this you first need to verify their key ID, which should be well publicized. Then you import it into the pacman keyring using “pacman-key --recv-key <KEYID>” and signify that you trust the key by locally signing it with your pamcan root key by running “pacman-key --lsign <KEYID>“.
Makepkg Source File Signature Verification
When building a package, the source files are often (and should be!) signed, with a signature file available for download alongside the source file. This typically has the same name as the source file with the extension .sig or .asc.makepkg will automatically verify the signature if it is downloaded in the sources array. e.g.:
However, makepkg needs some information to verify the source signature. It will need the public PGP key of the person who signed the source file, and that key to be trusted. The difference here is that you do not trust whoever provided the source file to provide packages for your system (or at least you should not the vast majority of the time), so your user’s keyring is used. To get the key use “gpg --recv-key <KEYID>” and trust it (once suitably verified) using “gpg --lsign <KEYID>“.
If you provide a package to the AUR, it would be a lot of work for everyone to suitably verify a PGP key and locally sign it. To demonstrate that you have verified the key, you can add the following to the PKGBUILD:
validpgpkeys=('F37CDAB708E65EA183FD1AF625EF0A436C2A4AFF') # Carlos O'Donell
Now makepkg will trust that key, even if it is not trusted in the package builder’s PGP keyring. The builder will still need to download the key, but that can be automated in their gpg.conf file.
Hopefully that clarifies the two separate types of PGP signature verification happening in pacman and makepkg and explains why they should be separate… Now can people stop recommending that the pacman keyring is imported into the user’s keyring and vice versa?
I released pacman-4.2 on the 19th of December – which is only marginally after the end of August as originally planned… We had 52 contributors provide patches to this release. Andrew takes the prize for most commits. Here are the top 10:
$ git shortlog -a -s -n v4.1.0..v4.2.0
164 Andrew Gregory
139 Allan McRae
66 Dave Reisner
26 Jason St. John
20 Florian Pritz
18 Pierre Neidhardt
15 Olivier Brunel
9 Jeremy Heiner
9 Jonathan Frazier
8 Dan McGee
The real prize goes to the person who caused the first reported bug. That could have been Dave but he caught it just in time. And I mean just! I posted to IRC “any ideas for the tag message” and the response I got was “I think I broke updpkgsums“. The shame of being first is inversely proportional to your commit count. (The small typos discovered so far do not count…)
There has been a couple of useful features added to makepkg. The main ones are:
Architecture Specific Fields: The source and depends (and related fields) now can all specify architecture specific values. For example:
The source for a given architecture is used in addition to the global source. The ‘+=‘ when specifying extra sources for an architecture does nothing different than just using ‘=‘, but I use it to serve as a reminder that these are additional values. Thanks to Dave!
Templating PKGBUILDs: Many PKGBUILDs share a similar build system, making them highly redundant. This is an attempt to reduce the redundancy by providing a template system. The easiest way to describe this is using an example, so I will use a potential perl module template. We create a file /usr/share/makepkg-template/perl-module-1.0.template. In this file is the build(), check() and package() functions and any common biolerplate. As this is our current version, it is also symlinked to perl-module.template. In our PKGBUILD, we would add:
# template input; name=perl-module;
and run makepkg-template. Now look in the PKGBUILD and you will see that line is replaced with:
# template start; name=perl-module; version=1.0;
# template end;
If we ever need to update the template, we create perl-module-2.0.template and update the symlink. Now run makepkg-template -n to update the PKGBUILD. Read “man makepkg-template” for more details. Thanks to Florian!
Incremental VCS Builds: Previously makepkg would remove its working copy of the VCS source directory before starting a new build. Now makepkg will just update the source copy (or attempt to in the case of SVN…) and build the package. This brings VCS builds in line with those using non-VCS sources. A new option -C/--clean was added to makepkg to remove the old $srcdir before building for cases where incremental builds fail. Thanks to Lukáš (and sorry it took me so long to deal with your patches)!
Source Package Information: To avoid things like the AUR attempting to parse bash to display information from a source tarball, we now provide a .SRCINFO file in an easily parseable format. Thanks to Dave!
Package Functions are Mandatory : The use of package() functions in PKGBUILD was introduced a long time ago. Now it is mandatory that a PKGBUILD has one (with the exception being metapackages that do not have a build() function either). Now that fakeroot usage is limited to the packaging step, the use of fakeroot is mandatory and building as root is disabled.
Misc. Changes: Other things of interest:
- Static libraries are only removed with options=('!static') if they have a shared counterpart
- Source signatures are required to be from a trusted source or listed in the validpgpkeys array. We also support kernel.org style source signing
- Split packages can no longer override pkgver/pkgrel/epoch as that was a silly idea…
No we don’t have hooks… They are strongly planned for the next release.
Directory Symlink Handling: Example time! Arch Linux has a /lib -> /usr/lib symlink. Previously, if pacman was installing a package and it found files in /lib, it would follow the symlink and install it in /usr/lib. However the filelist for that package still recorded the file in /lib. This caused heaps of difficulty in conflict resolving – primarily the need to resolve every path of all package files to look for conflicts. That was a stupid idea! So now if pacman sees a /lib directory in a package, it will detect a conflict with the symlink on the filesystem. If you were using this feature to install files elsewhere, you probably need to look into what a bind mount is! Note that this change requires us to correct the local package file list for any package installed using this mis-feature, so we bumped the database version. Upgrade using pacman-db-upgrade. Thanks to Andrew!
Added an –assume-installed Option: I believe this options was invented during a perl update. Almost all compiled perl modules have a dependency on a specific perl version. So with a major perl update, all the modules need to be updated at the same time, or you can use -d to ignore dependency versions, but for all packages and not just perl. This is not a problem with the Arch repositories where all packages are updated at the same time, but if you have lots of perl modules from the AUR, you will need to remove those, update, then rebuild them. Instead you can use --assume-installed perl-5.18 and all those packages depending on perl=5.18 will not complain. Thanks to Florian!
Repository Usage Configuration: A new configuration keyword was added for repositories – Usage. It can take values Sync, Search, Install, Upgrade, All. For example, I have the [staging] and [multilb-testing] repositories in my pacman.conf with the Sync usage. That way I can look at what is in these repositories without using them for package updates. Thanks to Dave!
Mics. Changes: Other changes to pacman:
- Improved dependency ordering – the dependency ordering did not go deep enough into the tree to ensure correct installation order.
- A warning is printed if a directory on the filesystem has different permissions to the one being “installed” from the package.
- Lock files should now never be left behind…
- Various speed-ups, memory leak plugs and bug fixes
See here for a more complete list of changes.
And I have just realized that the only major change I contributed was the requiring of package() functions, which I am told means 1/3 of the AUR will not build! It feels good to be back to breaking things…