Pacman Package Signing – 1: Makepkg and Repo-add

With pacman development progressing smoothly towards an upcoming 4.0.0 release, I thought it would be nice to write about everybody’s favourite (and a not at all controversial) topic… package signing. I will separate the discussion into several parts over the coming weeks, writing about a new area when I personally consider the interface in that area to being relatively finalised. That is not to say what is written about here will not change before the final release, just that I find it unlikely. Note also that I will focusing more on the technical details of the package signing implementation in pacman and its tools. So there will be limited discussion on issues a distribution may face using these features and I will not be specifically covering how Arch Linux will make use of these features.

The first thing that you are going to need to sign packages and repo databases is a PGP key. All the details of creating one using GnuPG can be found elsewhere. The only real consideration is the choice of key type. Currently a 2048-bit RSA key seems to be the gold standard. Going to 4096-bit is probably excessive and being a larger key has the side effect of slowing down the verification process (to an extent that is noticeable on older CPUs).

Once you have that sorted, it is time to sign some packages using makepkg. The implementation is quite simple. When a package signature is needed, makepkg simply calls gpg --detach-sign on the package(s) it creates. If you have the GnuPG-Agent is running, you will not even be asked for your passphrase (depending on your set-up). Deciding whether to sign packages or not is primarily controlled through the “sign” BUILDENV option in makepkg.conf, but can be overridden on the command line using --sign or --nosign. By default, the package will be signed with your primary PGP key. If you wish to use another key, you can set the GPGKEY variable (either in makepkg.conf or the environment), or use the --key option with makepkg.

The additions to repo-add are similarly simple. When adding a package to a repo database, repo-add checks for a detached signature and if present adds it to the package description entry, ready for libalpm to process. Finally, signing packages is not enough. We also need the ability to sign the package database (e.g. to prevent the holding back of an update to an individual package containing a security vulnerability). This is done using similar options to makepkg, with -s/--sign to tell repo-add to sign the database and --key (or the environmental variable GPGKEY) to select a non-default GPG key to sign with. In addition, repo-add has a -v/--verify flag that checks the current signature is valid before proceeding (very important as repo-add adjusts the current database rather than regenerating it from scratch).

As an aside, a couple of other useful security features have made their way into makepkg and repo-add during this development cycle. The ability to automatically check PGP signatures for source files has been added to makepkg (thanks to first time contributer Wieland Hoffmann). This is done by detecting files in the source ending in the standard extensions .sig and .asc. A source file and signature can be quickly be specified using bash expansion like:

sources=($pkgname-$pkgver.tar.gz{,.sig})

which makes it quite clear which source files have signatures. If wanted, this check can be skipped using the --skippgpcheck or the --skipinteg options (the latter of which also ships checksum checks). Also, repo-add includes a SHA256 checksum in the repo database in addition the the current MD5 checksum, although currently libalpm (and thus pacman) does nothing with this entry. (Despite some prior assertion, adding that properly took more than a one line change… but I will leave that there.)

Finally, a quick note on the challenges faced by distributions using these tools for package and database signing. The facilities provided by makepkg and repo-add work well for repositories where the packages get built locally, added to the repo database and then mirrored to their server (such as the repo I provide), but may not be ideal to use for a larger distribution repository maintained by multiple people. For example, if building a package on an remote build server, then the packager should not want to put their private PGP key onto that server to sign the package. It currently appears that there is no easy way around this, so the package building and signing steps need to be separated, with the built package downloaded locally and then signed (although this may change in future GnuPG releases as I see patches have been recently submitted to their mailing list providing a proof-of-concept implementation to improve remote signing functionality). Similarly, how is it best to sign a repository database that is added to by multiple packagers? Having some sort of master key sign it requires some sort of reduction in security of the passphrase (with either all people pushing to the repo knowing it or having it somehow accessible to the script adding the packages to the repo database). If set-up with care, this may be acceptably low risk for a distribution to use (and, from what I understand, this is what is done by several distributions), but personally I do not see it as an ideal solution. And that brings us back to the issue of how to best sign a remote file. So, implementing the tools may actually be the simple part in all of this…

Simple Arch Linux Theme for LXDM

LXDM is currently my login manager of choice since I abandoned Slim for becoming annoying in the configuration effort it required to keep my system working right. But with LXDM being fairly new to the scene, there are not particularly many good themes available yet.

So I decided to create a very simple theme based on the archlinux-simplyblack theme provided in the archlinux-themes-slim package. I dislike all the options crap typically displayed in a login manager, so this theme will not show a user selection dialog, session manager, keyboard layout changer, language selection or even a quit dialog, no matter what your configuration is. Also my guess is that this theme only works with the GTK greeter given I have not tested anything else…

Grab a copy of the theme here. Extract it into /usr/share/lxdm/themes/<name>, obviously providing the theme with whatever name you want. Adjust /etc/lxdm/lxdm.conf and you are possibly good to go… It works for me but there are no guarantees and I will likely not fix any issues other people have with it.

Classic Gaming – Part 2: Paganitzu

Welcome back to the ongoing series of posts about my adventures in playing old games from my childhood. I know that in Part 1 I said next up was going to be Commander Keen: Vorticons, but that is proving slightly more difficult than expected… Instead I completed the classic puzzle game series Paganitzu.

Like many other Apogee games at the time, Paganitzu was released in three parts. Part 1: Romancing the Rose was released as shareware, while Part 2: Quest for the Silver Dagger and Part 3: Jewel of the Yucatan were (and in fact still are) available for purchase directly from the developer.

The first thing you are greeted with when launching these games is a series of questions allowing you set-up to game colours, controls and sound. Having the ability to handle more than four colours on my MacBook Pro, I decided it was worth going all out and using the full 16 colours. I can feel the heat radiating from my graphics card already! When you are done selecting, you are informed of the command-line flags to select these options by default, which I remember thinking was really cool. (The question is all those years ago did I just adjust the launcher in the DOS menu system or write a one line batch script and use that… I can not remember but either seems possible.)

Onto the actual game. Our hero is famous archaeologist and part time treasure hunter Alabama “Al” Smith, a character not too subtly modelled after Indiana Jones. Unfortunately of late his fame has been slipping with people like Bart Simpson and Oprah Winfrey taking his spotlight. So Al studies his ancient texts and finds reference to an ancient pyramid called Paganitzu, meaning “Temple of the Gods”. Rumoured to be inside this pyramid are objects of great power including the Crystal Rose, a jeweled flower that will bring peace to the man who holds it, and the Silver Dagger, which gives the strength of the gods to it wielder. So off to remote southern Mexico where Al finds a grassy hill in an otherwise flat area. Sure enough, this is the temple he is looking for and there is the entrance…

What follows is twenty levels of pushing around boulders, dodging enemy attacks and finding of hidden areas, all while collecting the needed keys to progress through the door to the next level. Some of these levels do require some serious thinking to solve, but the unfortunate thing about this style of game is that its replay value is actually quite low (unless you are searching for all the hidden secrets). With having completed the first part of this game previously, it did not take particularly long for me to get to the end, although a couple of levels did test my levels of recall. Once you make it through all 20 levels, you are rewarded with a cut-scene in which Al gets his hands on the Silver Rose. Fame and fortune is all his… except he actually ends up releasing the evil god Omigosh instead.

Releasing evil gods is usually not a good thing and Omigosh is no exception. He travels deeper into the pyramid and plans to raise an army of undead to destroy the world. Understandably, Al feels a bit guilty about causing impending destruction and decides to go find the Silver Dagger in the aptly named “Part 2: Quest for the Silver Dagger”. While the overall style of the game remains the same, as we delve deeper into the pyramid it has now become hot and lava filled. Along with a selection of new monsters, this gives quite a different feel to part two of the game. The strategies required for solving the puzzles also shift to quite an extent with this change to make the puzzle solving a challenge again.

Part 2 rewards you with two cut-scenes. The first happens after you reach the half way point where you first meet the Skull Oracle. He suggests that Al should be sent back in time to bring back the great magician Debasco who originally captured Omigosh in the Crystal Rose. That sounds a great plan, until it is mentioned that only the dead can travel back in time to prevent disturbances being made to the space-time continuum (there is no restrictions on live people travelling forward in time). Somewhat predictably, Al decides to not become dead and take his chances with the Silver Dagger. Onwards for another ten levels, including some quite difficult ones, and our hero makes it to the Silver Dagger. There he meets Omigosh who is now inhabiting the body of some woman whose long dead corpse he found in the pyramid. Al grabs the Silver Dagger in attempt to end the evil Omigosh, but the dagger is too powerful and turns on him. A dead hero and an out of control evil god… who will save us? What a thrilling climax to the second part of the game!

Little things like being a ghost are not going to stop our hero. In fact, now he is already dead, travelling back in time does not seem such a bad idea… So onward to “Part 3: Jewel of the Yucatan”. The first thing Al notices is that the pyramid as seen by the dead is a living entity with walls made out of a quivering biomass. This makes for a third style to the games levels, which, along with the addition of some new deadly creatures (ghosts are not immune to being attacked) and new puzzle elements, changes the style of game play yet again. I am not sure if I was just doing the levels in novel ways, but I found there was a lot more usage of moving enemies to block other enemies and having to time movements precisely, bringing more of an action feel to the final part. Another twenty levels and we make it back to the Skull Oracle. Why were you now twenty levels away from the Skull Oracle when you only travelled ten levels away in Part 2? These are the mysteries of the pyramid that are not meant to be solved by the likes of us…

The final episode finishes with a stunning two hour cinematic! OK… it is apparently only 17 minutes long but it certainly takes a while to sit and watch. The Skull Oracle sends our hero back in time 500 years to get the magician Debasco to come save the world. Debasco is nice enough to restore Al back to life and so Al rewards him by getting it on with his daughter Maria, coincidentally being the woman whose body Omigosh inhabits in the future. Unfortunately, Debasco can only take one person with him to the future and that must be Al (or he could affect the space-time continuum), so his daughter gets left behind to be brutally murdered by the invading Spaniards (seriously, that is almost word for word…). Back in the future, the battle between Debasco and Omigosh looks helpless, until somehow the spirit of Maria banishes Omigosh from her body and Debasco can take him to the firely pits of hell. Al and Maria celebrate the vanquishing of Omigosh (seemingly ignoring the fact her father died…) and walk off into the sunset. You would think that 500 years of decay would be off-putting for a man, but you have to Al credit. As an archaeologist, I guess he likes old things…

What I like about these games is that they are unforgiving. If you can not solve a level, then you are stuck there. The game has judged you and found you lacking. Games these days would have you start in some sort of foyer with entrances to each level, allowing you to skip a few on your way to the end. Sure you can go back and complete the levels you skipped, probably to be rewarded with a different ending, but that just does not give the same sense of achievement in the end. And I say this having abandoned the second game in this series for quite a few days and moved onto the third while I tried to figure out one of the levels that was doing my head in.

Next up: Commander Keen: Vorticons… or maybe Crystal Caves if those stupid rat like things keep killing me. I can feel your anticipation building!

Syncing Files Across SFTP With LFTP

My webhost only provides SFTP access (which is not surprising given what I pay…). But this can become annoying for maintaining things like a package repository where I would like to keep the remote files in sync with my local copy. My first thought was to go with a FUSE based solution in combination with rsync. Looking into the current best options to mount the remote directory (probably sshfs), I was eventually lead to LftpFS and on to its underlying software LFTP.

LFTP is a sophisticate command-line file transfer program with its own shell-like command syntax. This allows syncing from my local repo copy to the remote server in a single command:

lftp -c "open -u <user>,<password> <host url>; mirror -c -e -R -L <path from> <path to>"

The -c flag tell LFTP to run the following commands (separated by a semicolon). I use two commands; the open command (should be obvious what it does…) and a mirror command. The only real “trick” there is to add -L to the mirror command, which makes symlinks be uploaded as the files they point to. This is required as the FTP protocol does not support symlinks and repo-add generates some.

That was exactly what I needed and it makes a nice bash alias being a single command.

Local WordPress Install On Arch Linux

After the WordPress update from 3.1.3 to 3.1.4 unexpectedly broke one of the plugins I use (My Link Order – why this was removed as a native feature in WordPress is beyond me…), I decided it was time to actually test updates locally before I pushed them to my site. That would also allow me to locally test theme changes and new plugins rather than just doing it live and attempting to quickly revert all breakages I made. It is still not the worlds best testing set-up as it does not use the same web server, PHP or MySQL version as my host, but I am fairly happy assuming the basics of WordPress will be compatibly with what my host provides and so only really need to test functionality that should not be affected by such differences.

Note I decided to go with Nginx as the web server as it seemed an easy way to go. I also did not use the WordPress package provided in the Arch Linux repos as it kind of defeats the whole purpose of testing the upgrade, requires slightly more set-up in nginx.conf and I think files in /srv/http should not be managed by the package manager (but that is another rant…).

So here is a super-quick ten-step guide to getting a local WordPress install up and running.

pacman -S nginx php-fpm mysql
Adjust /etc/nginx/conf/nginx.conf to enable PHP as described here
Enable the mysql.so and mysqli.so extensions in /etc/php/php.ini
sudo rc.d start mysqld php-fpm nginx
If this is your first MySQL install, run sudo mysql_secure_installation
Give yourself permission to write to /srv/http/nginx
Download and extract the WordPress tarball into /srv/http/nginx
Create the MySQL database and user as described here
Adjust the wp-config.php file as needed (see here)
Point your browser at http://127.0.0.1/wp-admin/install.php

And it is done! I have not attempted to set-up the auto-update features in WordPress as that involves either setting up and FTP or SSH server and I have no need to do either on my laptop.

As a bonus, I can now draft blog posts while offline and preview them with all their formatting. So you can all look forward to more rambling posts here from me…

Error Log

Who would have thought that looking in a file called php_errors.log would help identify the PHP issues I was having on my website…

Where Did My Bandwidth Go?

Here is what happens when you make a post with around 2MB of images in it…

Bandwidth Usage

That was a spike from my usual 100MB bandwidth use a day to over 2GB! I usually only use about 2 or 3GB for the whole month, so that was a bit of a surprise. Also, I only pay for 25GB a month so if it sustained at over 800MB a day I was going to be in trouble… (well, it would only be $2 more for an additional 50GB, so not too much trouble!)

So where did all that bandwidth go? Looking at my blog access stats, only about 20% of it is from people actually visiting my site. So the rest seems to come from people looking at my RSS feed, either directly or through sites like Planet Arch Linux that syndicate the feed.

That means I could drastically reduce my bandwidth usage by posting only a summary to my feed. But given I really dislike seeing only article summaries in my feed reader, it is not something I would really want to do. It is not as if my site has any advertising, so there is little point driving people here. Also, I would probably need to spend a few hours getting WordPress to actually provide summaries in the feeds the way I would like them (because WordPress never does anything quite “right”…).

Anime Guide 2010

Hmm… double checks the date… July 2011… It seems this is a bit late! Well, one of the advantages of being stuck in hospital for a while is that you get to catch up on things that otherwise get deferred. It is amazing how much time becomes available when you have no internet access or television.

This will not be a typical review of the year’s anime as I am not sadistic enough to watch (or even preview) the vast amount of series that I know I am going to find unappealing. It will also not include summaries of each anime as these can be found elsewhere. So, here are some brief opinions on anime I expected to be good enough to invest my time in watching that finished airing in 2010.

Anime of the Year

Fullmetal Alchemist: Brotherhood

Fullmetal Alchemist: Brotherhood
(TV, 64 episodes)

This series was always going to find it particularly tough to impress me despite (and because of) the original Fullmetal Alchemist being one of my all time favourites. And the early episodes, which had overlapping storyline with the original series, had me confirming my opinion of the remake not being worth it. Everything that made the original great was still there, but there was nothing to make it stand out. Then came the divergence of storylines. Many months after I have finished watching this, I am still not sure which ending I like better, although I may lean slightly towards Brotherhood. I think the fact that I am still internally debating which is is the better ending after all this time demonstrates just how good this series is.

I came to this not really knowing what to expect as I had not watched its predecessor (A Certain Magical Index). But it had “Railgun” in the title, so it must be good… Turns out that this is not really a superpowers show. Sure, there are people with superpowers everywhere, but that is considered ordinary in this series. That leaves you with a comedic slice-of-life series with plenty of superpower action spliced on top, which turned out to be an interesting and surprisingly appealing combination.

Angel Beats!

Angel Beats!
(TV, 13 episodes; DVD, 1 OVA)

I initially was not intending to watch this series, but people were raving about it… The series does have a good mystery/action feel to it, although some of the comedy is really over the top. Also, the pacing is all over the place with some episodes dragging to story to a near standstill while others progress blindingly fast. In the end I found the overtones of Haibane Renmei and The Melancholy of Haruhi Suzumiya too strong to have the series stand out on its own.

Durarara!!

Durarara!!
(TV, 24 episodes)

Big things were expected of this series, but it was always going to find it tough to escape the shadow of its elder brother Baccanno! This was a show of two halves… During the first arc, I really thought that it was going to live up to all expectations but the second half of the series became only good. That still makes this a very good anime to watch overall (and a serious contender for Anime of the Year), but I was left feeling unsatisfied due to it not living up to the potential I saw early on.

Highschool of the Dead

Highschool of the Dead
(TV, 12 episodes)

The zombies take over premise has been done repetitively with very little actual variation between implementations to make them stand out. But the key thing to enjoying a good zombie series or movie is to just not take it too seriously. They are not supposed to be masterpieces of originality, but rather mindless entertainment. And the creators of this series seem to know that and have embraced it fully.

Katanagatari

Katanagatari
(TV, 12 episodes)

This series was unusual in that it was broadcast with an hour long episode every month across the year. It also had a very “video game” feel to it initially, with each episode bacially being a fetch quest for a new sword (and I would buy the game immediately). The increased time taken for each episode seems to have paid off, with the action scenes being extremely well animated. Despite this, looking back on the series you realize that there is not sword fights everywhere and in fact it spends more time on the dramatic that action, which is somewhat surprising.

Shiki

Shiki
(TV, 22 episdoes)

A good horror series, although perhaps with perhaps a bit of a slow buildup at times. The atmosphere of the small town setting is well set-up during the beginnings for the eventual horror later in the series. However, the character design just seems too “happy” to fit in with the storyline, with the pointy hair and weird facial features just seeming out of place, and this somewhat spoils the build up of intensity.

Average

Dance in the Vampire Bund

Dance in the Vampire Bund
(TV, 12 episodes)

The series begs the question “What the hell is a Bund?”. Turns out to be a piece of land. Once that is answered, most of the mystery goes out of the series leaving a fairly bland vampire and werewolf series. At least they are traditional vampires and werewolves that kill stuff and not the romantic crap you get these days. The series does focus more on the political than action, but it is really just not intriguing enough to be greatly entertaining.

Eden of The East the Movie II: Paradise Lost

Eden of The East the Movie II: Paradise Lost
(Movie, 93 minutes)

I was hoping for a lot from this movie. The TV series was one of the highlights (if not the best) of 2009. Then it was followed with the first movie (The King of Eden), which really did not stand up on its own. But I knew that the second movie was following to wrap up the plot so the somewhat poor initial follow-up to a great series could potentially be forgiven. All has not been forgiven… That is not to say the movie was not good. It just can not stand beside the TV series it was following.

Psychic Detective Yakumo

Psychic Detective Yakumo
(TV, 13 episodes)

The most average show of the year. I found nothing particularly good about it but also nothing particularly bad. It was just OK. Nothing more to say really…

Rainbow

Rainbow
(TV, 26 episodes)

This is not your usual anime. It is a serious drama without anything to lighten the mood. The first half of this series is dark and depressing. It is perhaps because the first half was was so well done that I found the second half far too cheesy in what seems to be its attempt at being inspirational. In fact, I found it so cheesy that I really just did not care at all what happened to the main characters by the end.

The Disappearance of Haruhi Suzumiya

The Disappearance of Haruhi Suzumiya
(Movie, 163 minutes)

After the disappointment of the second season of The Melancholy of Haruhi Suzumiya (not just due to Endless Eight…), I was hoping that this movie would recapture some of the essence of the original. And there were elements of this movie that really did show the greatness of the original series, but they were not frequent enough to overcome what is for me fast becoming a jadedness towards this series.

Subpar

Occult Academy

Occult Academy
(TV, 13 episodes)

I generally favour watching short (~13 episode) series given there is very little room for episodes that do not forward the main storyline. Occult Academy is definitely an exception to that rule… The series seemed to have so much potential after the first two episodes, only to be followed by eight episodes of near nothingness. Seriously, if the world is about to end and only you can save it, do not waste your time holding a Christmas party to appease the spirit of some ghost girl. That can wait until later. Then it finishes with what I would usually consider to be a very satisfying three episode arc full of action and drama. However, given the ending is so out of place given the rest of the series, it is just not good enough to make this worth watching.

And there we go… a bit over 100 hours of anime for the year. Lesson learned – never calculate how much time you spend doing stuff.

Secondary Package Management With Pacman

Want to try out new software but also keep your system clean of packages that you do not use? Unless you keep good track of everything you install for a trial, you are likely to leave some unwanted packages on your system at some stage. Not that they generally do anything apart from take disk space (at least on Arch Linux), until one day when you are doing an update and you think “What is that package doing on my system?”.

One way I have found to keep track of packages you want to temporarily install is to have a sort of secondary package management system within the main pacman database. This is achieved through abuse of the dependency tracking features of pacman. Any package that is to be installed for a temporary period get installed with the --asdep flag. This tells pacman that the package is a dependency. Given no other package depends on it, it is what is commonly referred to as an “orphan” package and can be listed using pacman -Qtd. Currently on my system I have:

$ pacman -Qtd gimp 2.6.11-6 vlc 1.1.10-6

When I no longer want these packages on my system, they will be uninstalled in the standard way (pacman -Rs pkg). If I decide to keep the package, I can change the pacman database entry using the little known -D/--database flag. E.g. pacman -D --asexplicit vlc will change the install reason for the vlc package from being “Installed as a dependency for another package” to “Explicitly installed”. It will no longer be listed as an orphan, effectively taking it out of this secondary pacman management system inserting it into the main one.

I Am On The Google+

I got around to signing up for a Google+ account today. That is a big step for someone who never had a Facebook (or Diaspora) account. But then I did not know what to do with it. There was just this circle staring at me saying “Friends: 0”. Great… Google even knows I have no friends! And if Google tells you something, then it must be true. So I deleted that circle out of spite.

One thing I did like was the ability to link my other email addresses to my account. I signed up using my Gmail account because I was already logged into Google when I clicked on the invite link. But I do not actually use that account for anything (it forwards to my Hotmail account – yes, seriously… so does my archlinux.org address). So I was slightly concerned that I would be stuck using an e-mail address that no-one knew about, which would make it difficult for my zero friends to find me. Turns out Facebook has this feature too, so no real win for Google+ there, but I am always pleasantly surprised when things have the features I want. This is the joy of constantly low expectations about the world.

Here goes my profile page. Feel free to add me. If I do not like you then you can go into my “People I have added to a circle be polite but really will ignore from here on out” circle and you will never know… (Actually, I have no circle by that name because there is a limit on circle name length.)

Allan McRae

Time Magazine's "Person of the Year" 2006!

Author Archives: Allan