Pacman-4.1 Released

I have just released pacman-4.1 and packages are now in the [testing] repo. This is the first time I have made a release for any software project, so I was glad to have released a 4.1RC a few weeks back to learn everything that needed to be done.

It has been over a year since the pacman-4.0 release and there have been a large number of contributions made:

$ git shortlog -n -s --no-merges v4.0.0..v4.1.0 239 Allan McRae 185 Dan McGee 158 Dave Reisner 52 Andrew Gregory 23 Simon Gomizelj 20 William Giokas 19 Florian Pritz 15 Daniel Wallace ...

I win this time! Apart from the usual three contributors, it was great to see other people regularly helping out, both in providing and reviewing patches. A particular thanks to Andrew Gregory who helped me figure out how to fix something on several occasions and has been actively commenting on patches sent to the mailing list. His patch count also puts him in the top ten contributors of all time. In total we have 45 people with patches accepted for this release. Also a big thank you to our translators – particularly because I was learning how the system worked and may have required additional strings to be translated on a couple of occasions…

Moving on to what has changed. There have been quite a number of features added to pacman and makepkg and a couple of new helper scripts in this release.

The major feature for the release is tight integration between the package manager and systemd. After much discussion about how best to perform updates on a rolling release system, we realized that it was essential to have updates preformed with minimal other processes running. Also, the security aspects of updates mean that it is essential that these get provided as soon as possible. We felt the best way to achieve this was to perform updates on shutdown. This is achieved through a new daemon, pacmand that monitors and downloads updates in the background. When updates are found, it schedules a reboot of the system (hence the need to integrate systemd). At the moment the timing of the reboots is not configurable, but a timer will pop-up to allow you to delay it for a preset amount of time. Configuration will likely be added in pacman-4.2, when pacmanctl will be ready for general use. Until that release is made, Arch Linux will minimize the impact by performing all updates in its [testing] repository and only push updates on a yet to be decided day and time of the week. A news post will be made when that is decided.

Of course, all this makes systemd a hard dependency of pacman. We felt this was acceptable given Arch Linux has officially switched to using systemd. As this release is not tested (and unlikely to work) on systems without systemd, Arch users or other distributions using pacman will be required to make the switch to systemd if they want to continue using pacman as their package manager. The integration with system will become tighter in pacman-4.2 where we plan to use the upcoming kdbus message passing interface – through libsystemd-bus – to allow other programs to interact with pacman, making the development of alternative front-ends easier.

In terms of output, there has been improvements in a couple of areas. First colour support was added. This had been floating around for a long time, but no-one had ever spent the time to create a patchset and submit it. I think the colours for a simple update look good, although those when searching are a bit… rainbow. This can be only configured on or off at the moment. Extra informational output has been added for optdepends, providing details about whether an optdepend is installed or not and giving a warning when removing a package that is an optdepend for another. This also provides the groundwork for more complete optdepend handling in future releases.

When building packages using makepkg from this release, information about all the files in the package is stored, including permissions, modification times, sizes and checksums (md5 and sha256), etc. These can be checked using “pacman -Qkk“, excluding checksums (which requires additional support to be added to libarchive in order to read them in). Other useful features include never overwriting .pacsave files, but instead giving them a number suffix as needed. We have also polished the package signature checking, improving key importing and allowing configuration on how to validate packages installed with “pacman -U“, both using local files and from remote sources.

There are a few improvements to package building too. I have covered support for VCS packaging in makepkg previously, with bzr, git, hg and svn packages just requiring an appropriate line in the source array. Also a pkgver() function can be added to automatically update the pkgver variable in the PKGBUILD. With these VCS source lines, or any other source that is volatile, the value “SKIP” can be used in the checksum array.

An optional prepare() function can now be used in a PKGBUILD for preparation of the sources, such as patching and sed alterations. This function is run after the extraction of the sources and not run when --noextract is used, allowing operations that should only ever been run once on the sources to be skipped. Finally, a new debug option is available that will result all the debug symbols that are stripped from binary files to be stored in a separate package, which can be installed to allow easier debugging (another feature that has had patches floating around for a while).

Finally, two new helper scripts have been added to the contrib section: checkupdates and updpkgsums. The checkupdates script allows you to safely check for package updates without altering the system pacman remote databases. The updpkgsums script will perform an in place update of the checksums in a PKGBUILD, although more complex PKGBUILDs (such as those with different sources for each architecture) will not likely work…

So a long post, but this is a big release! There are enough of running the git version that it should be completely bug free, but just in case I am wrong report any issues to the bug tracker.

Edit: Yes – some of this was April Fools… (moderated comments are now restored too).

Interesting Links – March 2013

And we come to the end of another month. And not surprisingly, more stuff happened…

Software news first:

There was lots and lots and lots and lots of talk about Mir – Ubuntu’s new anti-Wayland.
And if that was not enough, here are more comments from people on the issue
Speaking of Wayland, here is a summary of its progress in Arch land…
And how its support in GTK and GNOME is progressing
Wayland/Weston also have a new fork – for some reason
gcc-4.8 was released (get it from the Arch [testing] repo) and now builds in C++
ZFS is ready for use
GTK+-3.8 was released, closely followed by GNOME-3.8
The math library performance in glibc is getting continuous improvements
A new startup manager for KDE is in the works and it looks like it will speed up your login
ownCloud 5 was released
A summary of plans for libsystemd-bus and kdbus
The security features of RPM – I thought installed file validation would be in there…
Ever wondered how often assembly is used in the software carried by a distribution?
Finally, if you use PostgreSQL, be prepared for the 4th of April…

Various distro news:

openSUSE released an ARM64 port (although there is no hardware…)
Ubuntu also looked at rolling releases, but decided not to
Instead, they halved the support time of non-LTS releases
It seems Arch Linux is the best multimedia distribution (WTF!?)
How Fedora manages building for multiple architectures

And finally…

Want to go to Mars – on a permanent trip? Probably do it with a company with more than a handful of employees…

Pacman 4.1.0rc1

For those that are mildly adventurous, you can try the pre-release of the upcoming pacman-4.1. There are a handful of us who constantly run pacman from git so it should be fairly safe. All bugs found are to be reported to the bug tracker. (Only one issue found so far – in the rarely used pkgdelta script).

~~Download: i686 x86_64~~

I’ll make a post about all the new features when the final 4.1.0 release is made – hopefully before the end of the month.

My Arch Linux Talk at SINFO XX

I was recently invited to give a talk about Arch Linux at SINFO XX at IST in Lisbon, Portugal. It was a whirlwind tour of Europe, with the time I spent in transit almost exactly equal to the time I spent there.

Check out the video of my talk on their YouTube channel. I discuss what makes Arch different from other Linux distributions, what our strengths are as a distribution and briefly cover what future plans people have. I’m not going to watch it as it is never a good idea to dissect talks too much, so I’ll just assume I was awesome… It was also after midnight in my time zone, so I blame any mistakes on that.

Thanks to the organizers for inviting me over!

Edit: Quite a few people have asked for copies of me slides. Here they are (CC BY-SA): ODP PDF

Interesting Links – February 2013

I’m a bit late with the links this month due to travel (more on that later…), but as is the case every month, it is entirely worth the wait!

How to split this up… Lets start with software related links:

More and more and more and more on secure boot
There was (and maybe still is) a bug to brick some Samsung laptops, although not Linux specific
GNOME has made JavaScript its default language for applications and apparently that is a good thing
Google provided some C++ containers that are faster and more memory efficient than the STL versions
Debian has been recompiled with Clang again…
But why would you bother for a whole distribtion when gcc -O1 is equal to (or even a little bit better than) clang -O2 in performance and compile speed
Libreoffice is now one impressive user of make
Almost every piece of software will need an autoreconf to build for AArch64
It seems you can now use OpenRC on Arch Linux – because systemd is evil
Some magic involving the kernel and dbus
A groundbreaking revelation on how to install LibreOffice-4.0 in Arch Linux
This post on shared library permissions can out the same week I was reading up on it! Arch Linux still has no policy on these permissions…
A new XFCE release is always good, mainly due to its minimal change.
Tom did not read the Arch wiki when installing on his MacBook, so another way to remove the boot-up sound
No choice here – I enjoy a good rant
This is why all bug fixes should be accepted by upstream first
A report on what happened with Xorg in 2012
The Python trademark is having issues in Europe – with update

Various Linux distribution stuff:

The AArch64 (64bit ARM) Debian port can boot
Or is the distro called Debian/Ubuntu?
The GNU Hurd moves along slowly, slowly.
Arch Linux is crap and full of broken packages, but here is how to deal with it…
Bad statistics and wobbly lines
Another month, another Arch spinoff…
Remember how last month it was concluded Ubuntu was not going rolling release? To clear that up, things may or may not have changed.
Debian Wheezy is getting closer. Here is what is new.

And some fun stuff:

I saw this post about needed punctuation marks and was reminded of this comedy sketch.
Type your address and watch this go…
Monopoly can be finished rather quickly
Google has some interesting issues in Australia!
And they should give me one of these… I would use it to increase my awesomeness

Interesting Links – January 2013

End of January already! Not sure where that month went…

Lets start with some distro news:

More Arch spinoffs are in the works (and real ones!) – Arch BSD which is a BSD in the Arch sytle and Starch Linux which focuses on static linking
Fedora 18 finally appeared and plans for inclusions in 19 are being worked out
Ubuntu going rolling release? No…

Software related:

If you are using MoinMoin for a wiki, you better update… It is rather critical!
More software nearing python-3 ready. This time django is close…
Given how much is still using libjpeg-6, I’m not sure I’d be compressing JPEGs using this yet…
Bye bye systemd myths – and here goes the prize for best comment.
Another post in the “autotools mythbuster” series – the main “book” is worth a read too

And some collected commentary:

How to get involved at Gentoo – most applies to Arch too..
A quite good summary of what it means to be a rolling release
What is the best MAKEFLAG for your number of processors?
I’m always impressed by the ability to brick things just by booting!
A fix for a timing issue in the “-ck” kernel, discovered by an Arch user tying to compile glibc
Hrmm… which one is supossed to be the bad guy here?

Keeping Packages Vanilla – 1. Patching

I have recently been thinking about a Linux distribution providing “vanilla packages” and what this really means. The basic idea is simple – provide packages as upstream released them. In practice, this is an impossibility (for reasons I will cover below).

Firstly, lets cover the reason for providing vanilla packages. And I use “reason” singular deliberately, because I think it comes down to a single point. The software developer knows their software better than you do. Your patch could unintentionally introduce a security issue – it has happened before… Or you could introduce a new feature that is then introduced by the developer in a different and incompatible way in the next official release. Also, any bugs that are found in your modified piece of software will need to be triaged in an unpatched version of the software in order to report the issue upstream.

So, in an ideal world, we would just run the equivalent of “./configure; make; make install” and all software would install perfectly. But the world is far from ideal… I will cover two points across two posts: firstly patching, followed by configure options and dependencies.

Patching software is a necessity in any Linux distribution. I am only considering rolling release distributions in the discussion below, so that removes backporting fixes and features from newer versions of a software package. So, what patching is minimally required:

Patches for build issues
Patches for security issues
Patches to fix major software features

I am completely ignoring patches released by upstream as part of their update process. For example the Linux kernel provides a large patch that updates from their x.y.0 release to x.y.z. Bash releases a patch for each minor update, so bash-4.2.042 requires applying 42 patches… These are obviously required patching and are fully sanctioned by the developer so do not deviate from vanilla.

It should be fairly obvious what patches for build issues are… the software will not compile for some reason so you need to do patching to fix that. A piece of software is almost never tested in every single environment before release and, even if it was, updates to other pieces of software can cause build issues. This is particularly common with gcc updates which have become progressively stricter on which headers needed included for a function. Even worse, a software developer might release with the “-Werror” flag enabled by default, meaning any new warning will result in a build failure (I do not have kind words about software developers that do that…). Then there are more complicated issues involving a library update with API changes requiring much more extensive fixes. While adding a single extra include really does not require upstream approval before applying, even that should be forwarded upstream.

Security issues are an important part of patching for non-rolling release distributions. However, new versions of software are usually released whenever a security hole is found, so rolling release distributions only need to update. Again, just grabbing a patch from anywhere on the internet and applying it is not a good idea – I have seen this actually result in a larger security issue than the original.

The final category of patches are those that fix major software features. For example, if an IRC client has a bug preventing it connecting to any channels, firstly shake your fist at the developer and tell them to do some testing before release, and then patch it. If there is a typo in the help output, file a bug or submit a patch upstream, but there is no need to patch it. The guiding principle should be something like “if this is the only issue found in the software, would the developer consider making a new release?”. The answer to that question “yes”, then it will be “yes” to provide the patch.

One guiding factor that can not be stressed enough here is that all patches should be approved by upstream. The best situation is if upstream have committed the patch into the version control repository – preferably on the branch for the version you are using so no mistakes can be made back-porting. Failing that, a post on a mailing list or bug tracker by one of the main developers of the software approving a patch is acceptable.

Of course, much of this is subjective. Is that broken feature big enough to patch? Does this bug constitute a security issue? If upstream is rather unresponsive at the moment, should I apply this fix for a security bug? Is this build fix minor enough that I do not need to wait on an upstream comment before applying? Given it is hard to formulate these ideas into precise rules, I think the answer becomes one of how strict the packager is. I was far more likely to include patches when I started packaging for Arch Linux than I am now. So maybe it is not how strict the packager is, but rather how grumpy…

Manjaro Linux: Ignoring Security For Stability

I feel like having a rant today… so nothing particularly unusual there. But after reading yet another post saying:

I used Arch for two years and it was perfectly fine until one day when I updated and it broke my system. Now I have been using Manjaro for a month and it is completely stable.

I have found what to rant about! Is it just me that notices the issue with that statement?

I have no issue ranting about Manjaro because every time I read their forums I see one of their Core Team being less than congenial regarding Arch – but I suppose they have to be given one of the main selling points of their distribution is they can fix the Arch Linux updating “mess”. Also, the defensiveness of their community to any criticism and the amount of self congratulations for being a Manjaro user astounds me – and I spend a lot of time in the Arch forums and IRC channel where the community are widely considered to be elitist pricks (with me being no exception, as this post will plainly show).

For those who do not know how Manjaro works, I will paraphrase this post. The Arch stable repos are synced into Manjaro Unstable on a roughly daily basis. They sit there for 1-2 weeks before being declared stable and moving to Manjaro Testing. Then their test squad declares that stable enough to move to Manjaro Stable, about 3-4 weeks after the packages arrive in Arch Linux.

And this is the issue. There is four weeks until Manjaro users get package updates. That is still a lot quicker than a non-rolling release distribution I hear you say, but it ignores one of the fundamentals of a rolling release distribution. Security fixes come with a new software release. On a fixed-point release distribution, security fixes are backported into your out-of-date software versions to maintain stability. On a rolling release distribution, you just release the newer version of the software that comes with most security fixes (some backporting from the upstream VCS is required if a release is not made).

That means, Manjaro users are vulnerable to security bugs for around a month after Arch users are safe, unless of course the Manjaro Core Team monitors every package and pushes those with security fixes. How many packages in a distribution? Arch Linux has >6000 in its binary repositories. I suppose it is not impossible to monitor that many packages, unless of course your Core Team consists of three people. And given those three people provide five variants of their installation ISO (net install, XFCE, KDE, Cinnamon, MATE – with OpenBox and E17 on the way…) and provide a series of kernel packages and systemd… Things are looking bleak.

And so, Manjaro users are stuck with packages having security issues for a while. I’d assume the big ones get through quicker. Although their firefox package has not been updated to version 18 yet, which fixes 21 security issues – 12 of which are marked critical. In fact, firefox version 18 has not even made their Unstable repo as I am writing this…

Lets say Manjaro had the man-power to monitor all updates in Arch Linux for security issues. Could they be brought to the Stable repositories more quickly? Maybe… But remember Arch Linux rebuilds against new versions of libraries with soname bumps all the time and our toolchain gets updated very quickly after any upstream release. So any security update built against new libraries or with a new toolchain version require those components moved too. And they are the types of updates that could introduce stability issues.

In the end, I think the idea behind Manjaro – rolling release at a more relaxed pace – can be achieved. I am not entirely familiar with these distributions, but I guess that is exactly what apotsid and LMDE achieve. And they start form Debian Unstable, which is reportedly far more of a minefield than Arch Linux.

Anime Guide 2012

It is entirely possible that I am getting old(er) and grumpy(er), but I found this years anime selection very lacking. Lowlights included a series about anthropomorphized guns and one about licking drool left by another student resulting in a drool addiction.

As in previous years, I only consider anime that have finished their run during 2012. So any series that started this year and has not finished yet is not included. Also, I do not do real reviews, but just make my opinions known on a selection of anime that I thought would be interesting enough to watch.

Anime of the Year

Another

Another
(TV Series, 12 episodes)

Admittedly, all horror anime series are fairly predictable and much of this is no exception. So how did this get my “Anime of the Year” stamp? Because when it killed people, it did so very gruesomely! That is enough when there is a lack of better choice… The beginning of this series was so slow that I was considering moving on – I am really talking up the winner this year – then something happened and I had to immediately see the next episode. Still, it is a sad year when an anime with a beach episode (in a horror series, really?!) makes it to this not-very-coveted-at-all spot. And I made it through this without using “another” in a pun, which was a struggle…

There was not a chance of me skipping this anime given the director and who provided the music. This is despite it really not being a usual choice for me as, unlike Watanabe’s usual series, this is appeared relatively slow moving drama. However, I get the impression of there being quite a lot of content when I look back on the 12 episodes, so it can not have been that slow. I think the slower points were a needed tool to give room for the characters to develop. I am no expert on jazz or 1960s Japan, but I hear only good things about the attention to detail in both aspects. A surprisingly good series for something I went into with a mildly negative outlook.

Average

Fate/Zero

Fate/Zero
(TV Series, 25 episodes)

At one stage I was lacking much to watch and I remembered Fate/Stay Night being quite good. But watching this reminded me why as a rule any series based on a video game is bad. Lines like “Sabers parameters are particularly high. Most of them appear ranked A.” are just awful. There was enough dumb action to get beyond that.

Humanity Has Declined

Humanity Has Declined
(TV Series, 12 episodes)

I try watching at least one thing that seems a bit weird each year and here it is… My main issue with this series is that there was not enough of it. Most story-lines were two episodes long, which makes a 12 episode series too short to really achieve anything. Saying that, the short stories really do not link together very well, so extending it further could have made the series too disjointed.

Kokoro Connect

Kokoro Connect
(TV Series, 13 episodes)

Everything I saw here made me thing this was a Melancholy of Haruhi Suzumiya wannabe that tried hard to be different enough. The body swapping was entertaining enough and I was amused at one of the girls saying she was getting good a peeing standing up. But then the ideas for where to take the series ran out. Well, they obviously did not run out, they were just bad.

Last Exile: Fam, The Silver Wing

Last Exile: Fam, The Silver Wing
(TV Series, 23 episodes)

Another sequel that yet again did not live up to the original. Part way through this I got the intense feeling that I did not understand the end of the original, so I went back an watched the final episode. That did not help… It turns out there is a manga that covers the gap between the two stories, but I guess most people (at least outside Japan) have not read it. But we do get a history lesson at the end of this series. That is not good for a series that really relies on the greatness of the original.

Sword Art Online

Sword Art Online
(TV Series, 25 episodes)

This was my pick of the “this video game just got real” genre for the year – there were quite a few series in it. How it approached the long-term effects of being stuck in a virtual reality world was quite interesting. And the over-the-top battle scenes were absolutely fine, because this is a game. And then there is the second half. Not only was the ball dropped, it was… (I got stuck thinking of a good enough hyperbole to use here). In fact, just don’t watch the second half; the ending of the first arc is just fine.

Sub-par

Bodacious Space Pirates

Bodacious Space Pirates
(TV Series, 26 episodes)

So… I watched this entirely because it had “Bodacious” in the title… Had I done a bit of research and seen that it is based on a light novel called “Miniskirt Space Pirates”, I probably would have reconsidered. Because this is the future, pirating is all about cyberwarfare and so we get to watch people type fast. To be fair, there is also lasers to counteract that boredom. And because the series is so seriously slow in places, there is plenty of boredom. One thing that did surprise me – well, because this is a bad anime and usually such an “opportunity” would be taken… – is that miniskirts do not lift up in zero gravity. Probably for the best given the ages of the characters.

K
(TV Series, 13 episodes)

I read this was a bit like Durarara!! so I took a look. I need to find that review, identify its author, buy a plane ticket, knock on their front door and tell them NO! Although at one stage I saw artwork that was similar and there was a large number of characters, any similarity ended there. The final episode is good and potentially could have done some redeeming if the second series was not announced in the middle and essentially gave away part of the ending.

The Future Diary

The Future Diary
(TV Series, 26 episodes)

Only one can survive and there is no doubt which one it will be even though he is absolutely useless. His one advantage is the writers have no qualms using many, many contrived leaps of logic to help him. The one redeeming feature of this series is that it does teach a valuable life lesson: “Don’t stick your dick in crazy”.

And that is it for the year as far as I am concerned… Technically Hellsing Ultimate fits my criterion of finishing in 2012, but I doubt I will get ahold of the final OVA for a while yet – not that waiting is an issue with that series (the first OVA was released seven years ago). At this stage next year’s list looks to be improved, with the four series I am watching now (Space Brothers, From the New World, Psycho-Pass, Blast of Tempest) all worth a look at this stage.

Interesting Links – December 2012

Final collection of links for the year…