Pacman Translations

I was listening to Frostcast in the background today when I heard my name. That always makes me pay some attention. Then I heard wrong information. I don’t know why I care, but I do… so here goes the clarification.

The quote from Philip Müller at 14:35 into the podcast:

The lastest news was Allan McRae – he is a developer of pacman himself – he sent me an email to send over all the translations of Manjaro distribution does. So I forked pacman, and pacman itself has 20 translations and our branch has 44 translations of the same software so Arch Linux is asking us to be upstream and give them our translations…

OK… This is interesting. Time for some background here. When pacman-4.1 was released, we removed the broken SyncFirst option. This is needed by Manjaro Linux to run their update helper script that “fixes” the update process to remove any manual interventions. So Manjaro reverted our patch and brought back SyncFirst to pacman. That required three additional strings to be translated for their version of pacman so they also forked our translation project on Transifex.

As the Arch and Manjaro versions of these projects had started to diverge, I wrote to Phil noting that people were doing more than just translating those three additional strings, and it would be good if the translators were pointed at the Arch project so we all benefited, given the Arch project is the one the pacman developers set up.

Lets compare the status of the Arch and Manjaro translations as of 2013-09-24. There are 24 languages with complete translations in the Arch projects, and being nice and ignoring the additional three strings in the Manjaro project, they have 23. (Of those 23, only 6 actually have the additional three Manjaro strings translated). What are the differences? Manjaro has a complete Hungarian translation while Arch has complete Korean and Romanian translations. The Arch Hungarian translation is at 99%, while the Manjaro Korean and Romanian are at 21% and 62% respectively. So it is clear these languages have diverged since the split, with most of the work done in Arch.

Of the remaining languages with incomplete translations, Manjaro has 19 languages, while Arch has 15. Clearly not a total difference of 20 to 44 languages as claimed. Looking at these in more detail, 9 languages have not deviated between the two projects. The Arabic, Chinese (Taiwan), Dutch, Galician, Polish, Serbian (Latin) translations have all got additional translations in the Arch project since the split with the Manjaro project. So apart from languages that have been have had translations started in Manjaro but not in Arch, the Arch project is behind in 3 strings for the Hungarian language.

Maybe where the Arch translation project for pacman could gain is from the new languages in the Manjaro translation: Czech (Czech Republic) [99%], Bulgarian (Bulgaria) [62%], Uzbek [14%] and Danish (Denmark) [3%]. Also note that 3/4 of those languages have a sub-name there. Taking “Danish (Denmark)” as an example, there is already a “Danish” translation (language code: da) and this is adding a Denmark specialization (language code: da_DK). I might be entirely wrong here, but are there other variants of Czech, Bulgarian and Danish apart from their primary usage, or are these exactly the same and the work is just being repeated?

In summary, the translation project set up by the pacman developers is, and will remain, the upstream translation. I just approached Manjaro to send their translations our way so we would both benefit. Arch from (potentially) more translations, and it would be easier for Manjaro to merge their string translations without ending up removing several hundred perfectly good translations.

Arch Linux Community on Gittip

A few months ago Dusty posted about creating an Arch Linux community on Gittip. For those that do not know what Gittip is, it is a way to make small donations weekly to “to people you love and are inspired by”. Due to his pushing for people to join up, this community is now “official” in Gittip land, which means it has 150 members.

I have joined up because I always need money to buy things that I otherwise would not. And I need to give a big thank you to those that have tipped me so far.

I’m not one to get carried away, but lets take a look at the last three weeks. I received $1, $2.25 and $4.25. If we fit a line through those, we get:

where, y is the amount tipped for week x (x = 1, 2, 3, …). [Aside: I would figure out how to do pretty formulas, but no-one wants much math in a blog post, or anywhere...]

Anyway, if we take that formula and look at my projected tips for the next while:

Now I can start extrapolating that at some stage in the next two years, I will be getting enough tips to quit my job and work on Arch full time. Also, before five years have past I will get $20,000 a week, which converts to ~$1,000,000 a year.

So this has been much more successful than I could have ever imagined! When I am bringing in the millions I will have a party for Arch Linux users in my luxury beach-side property.

Routing Traffic With OpenVPN

Don’t you love messages like “This video is not available in your country”. That generally means it is not available outside the USA (unless you are in Germany in which case it means it is available anywhere but your country). They have finally annoyed me enough to do something about it and my web server is based in the USA so it should be easy… Right?

It turns out that while it should be easy, it took me far longer than expected, mainly because I found most tutorials assume you know things. Also, I found the Arch wiki not to be great in this area (and as you may notice below, I am not the person to fix it). So here is yet another “guide” on how to set up OpenVPN on your server and getting both Linux and Windows clients to access it (mainly posted as notes in case I need to set this up again in the future).

Step #1: OpenVPN

The first thing that needs done is the creation of a personal Certificate Authority and generating the needed keys for your server. Fortunately, OpenVPN comes with a set of scripts called easy-rsa. I am not going to cover this as these scripts are well documented and do all the work for you.

I am also not going into all the configuration of OpenVPN. But here is an /etc/openvpn/server.conf file I found to work:

dev tun
proto udp
port 1194
 
ca ca.crt
cert server.crt
key server.key
dh dh1024.pem
 
user nobody
group nobody
 
server 10.8.0.0 255.255.255.0
 
persist-key
persist-tun
 
client-to-client
 
push "redirect-gateway def1"
push "dhcp-option DNS 8.8.8.8"
 
log-append /var/log/openvpn

Not that the configuration file can be called anything you want. OpenVPN is launched using “systemctl start openvpn@server.service“, where “server” in this case is because my configuration file is “server.conf“.

The only bit of configuration I will directly mention is setting up users to be able to access the VPN using a username/password approach rather than generating individual keys for each client. This is purely because I am a lazy admin and everyone I want to use my VPN has an SFTP shell on my server already. Just add this to your server configuration file:

plugin /usr/lib/openvpn/plugins/openvpn-plugin-auth-pam.so login
client-cert-not-required
username-as-common-name

OpenVPN does give warnings about this configuration, so consider the security implications if you use it.

Step #2: Enable forwarding
By default, packet forwarding is disabled. You can see this by running:

$ cat /proc/sys/net/ipv4/ip_forward
0

We can enable this by adding these lines to /etc/sysctl.conf:

# Packet forwarding
net.ipv4.ip_forward = 1
net.inet.ip.fastforwarding = 1

Note that the file /etc/sysctl.conf is not going to be used from systemd-207 onwards, so this will need to be move the appropriate place. Also, the “fastforwarding” line is purely based on anecdotes I found on the internet, and may not do anything at all! In fact, I think it is a BSD thing, so I have no idea why I am mentioning it. Moving on…

Step #3: iptables

If you have iptables running, you will need to open up access to the VPN. You also have to forward the VPN client traffic through to the internet. This is the bit I found least documented anywhere. Also, I am not an iptables expert, so while this works, it might not be the best approach:

# OpenVPN
iptables -A INPUT -i eth0 -m state --state NEW -p udp --dport 1194 -j ACCEPT
 
# Allow TUN interface connections to OpenVPN server
iptables -A INPUT -i tun+ -j ACCEPT
 
# Allow TUN interface connections to be forwarded through other interfaces
iptables -A FORWARD -i tun+ -j ACCEPT
iptables -A FORWARD -i tun+ -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i eth0 -o tun+ -m state --state RELATED,ESTABLISHED -j ACCEPT
 
# NAT the VPN client traffic to the internet
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE

If your default iptables OUTPUT value is not ACCEPT, you will also need a line like:

iptables -A OUTPUT -o tun+ -j ACCEPT

I have not tested that set-up, so you may need more.

Step 4: Clients
I went with using NetworkManager on Linux and the OpenVPN client on Windows. Note the two different Windows clients on that site are exactly the same, so there is no need to figure out the difference.

The Windows client requires a “.ovpn configuration file. Here is an example:

client
dev tun
proto udp
remote foobar.com 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca foobar.crt
verb 3
auth-user-pass

The only thing to note here is the “ca” line is the public certificate you generated in Step #1. Even with password authentication, you still need to provide that certificate. Also, it is important to have the compression setting being the same in both the server and client configuration. Given I mostly want to use the VPN for video, I have not enabled compression.

And there you have it. Configuring that took me much too much time, but now I can pretend to be in the USA and watch all the content that is geolocked to there that I want. Which as it turns out, is not often. I have only watched one video in the month since I set this up. Good use of my time…

Interesting Links – August 2013

Better do this months post on time!

  • Things I said were published in Linux Format magazine. (It cost $25 to buy in my local newsagent – better get an online copy!)
  • Android 4.3 can be run on x86
  • Linux kernel 3.10 is going to be longterm
  • So we will keep the unreviewed code in linux-3.11
  • Elementary OS “Luna” was released after a long road
  • Wayland is progressing nicely to replace X
  • Groklaw has closed its doors
  • Some interesting facts about Debian for its 20th birthday
  • And looks at some of the DebConf13 talks
  • An article about supporting both python 2 and 3 in one codebase
  • The Xorg foundation needs to keep up with its books
  • GNOME switched to using DuckDuckGo as their default search
  • Some redeeming for the New Zealand government. After passing the GSCB spying law, they ban software patents.

And for your amusement…

  • A brilliant demonstration of how to break wood

Interesting Links – July 2013

Oops… never posted this at the end of July. And there was so many links. You will get over it!

Distro and software news:

  • Fedora 19 is out and it appears they have the release process more under control these days.
  • Android 4.3 is out, and it saved my Nexus 7… maybe.
  • “Yet, it means Archlinux developers are in trouble.”
  • Videos from the GNU Tools Cauldron are available
  • These kinds of posts make the LKML worth reading!
  • Firefox seems to be moving to the front in terms of everything again.
  • Qt 5.1 was released with technology preview for Andriod and iOS platforms
  • Some details on what will happen with the KDE5 transition
  • Even though I read all the emails on this, the LWN article really helped me understand this lock elision thing in glibc
  • More responses to Debian’s systemd survey.
  • A new secure messaging client is on its way.
  • Emacs got its own package manager
  • LXDE-Qt and Razor are joining forces
  • Mir seems to be progressing nicely
  • There is now an unpacked Debian sources mirror
  • Talk about compilers in OpenBSD and the need for an LTS compiler.
  • And a story of a Fedorians trip into Archland. A bit of distro hopping is good for everyone.

And some other stuff…:

  • xkcd’s Time finished – here it is in youtube format
  • Also, if you do not read xkcd’s What If, I recommend you do. This one was good.
  • Making NES games run natively with LLVM
  • This is what Sonic the Hedgehog would look like in first person
  • And it seems like I offend everyone! (Requires being logged in)

Char on ARM

Tweet

Who knew? A char on ARM is unsigned. Turns out, lots of people knew… (I was hitting the EOF test issue.)

Posted in Tweet on by Allan Comments Off

Things I Learned About Crocodiles

While I was on holiday I learned something about crocodiles. They are very considerate and intelligent. Take this crocodile for example:

crocodile

It does not want to endanger humans, so it hangs out under a sign telling people to keep away. Also, note the crocodile is looking slightly confused (it takes quite a bit of experience to read crocodile emotions, so you may miss it). A closer inspection of the sign tells us why. A warning written in German in Australia? A good query for a crocodile. But he obviously had not read about German tourists and their “success” at going near the water in the region.

While we are talking about warning signs in Tropical North Queensland, I have seen enough anime to know where this is heading…

tentacle

Interesting Links – June 2013

Not a lot of links this month – people must be getting progressively more boring! Or it could be I went on holiday for a week with NO INTERNET. Now you have recovered from that horror, time for some links:

  • Ubuntu is making progress with Mir
  • But Mir will not be used by either Kubuntu or Lubuntu
  • Speaking of X replacements, some more info about Wayland
  • The LAS did a one week Arch Challenge, as did someone else.
  • Reports vary about what the default browser will be in the next Ubuntu. Possibly Chromium?
  • Some cgroups changes will be happening in systemd
  • And some more answers to systemd concerns
  • LLVM 3.3 has full C++11 support, in contrast to gcc-4.8.1 that sort of did…
  • Some interesting decisions in the RHEL7 roadmap
  • A useful table detailing toolchain component compatibility for building cross compilers
  • And should I be concerned my Nexus 7 will slow down?

Leaving Identi.ca

When I saw the message that Identi.ca was switching to the pumpio platform, I started thinking when the last time I actually used it was. Turns out, it was the end of 2012. At that time I also discovered post formats for WordPress and saw the status post seems to be a replacement for tweeting.

So I have decided that all my short posts can happen here. This was a driving force to find a new theme, because my old one did not support status posts. Import my old tweets (or whatever they were called on identi.ca) and I am up and running!

The reason I am posting this here is that if people are following my main RSS, they will get the occasional “tweet” from me. If you want to avoid that, the “tweet” category can be excluded using this link.

C++11 – Part 8: Rvalue References and Move Semantics

While moving my hosting I noticed some posts that I never bothered tidying up to post. And although there has been a bit of a break, I am still writing these introduction to C++11 posts. Chances are low that I will have the C++11 posts finished before C++14 arrives! Moving on…

Consider the following code snippet:

template swap(T& a, T& b) {
  T t(a);   // #1
  a = b;    // #2
  b = t;    // #3
}

On line #1, we end up with two copies of a, line #2 gives two copies of b and line three is back to two copies of (the original) a. If T is a type that requires extensive copying this requires quite a bit of overhead. The standard library has specialized cases for std::vector and std::string to remove that overhead, but it would be good to get rid of it in general.

Lets consider another example:

T foo() {
  T t;
  // do "foo" on t...
  return t;
}
T u = foo();

Here foo() returns a temporary copy of t constructed on the stack and then that is passed to the T copy constructor to initialize u. Again this can have a substantial overhead. (I am ignoring return-type optimization here…)

From these examples, it should be clear that such overhead is reasonably common in C++ (and seems to be one of the biggest criticisms of the language). To fix this, a new non-const reference type is added in C++11, called an r-value reference and denoted T&&.

Firstly, what is an r-value? They are temporary objects that tend to fall on the right hand side of an equation. Because they are temporary objects, you can not assign to them. Also, you can not assign a r-value to a regular reference. E.g.

int& foo = bar()

will fail because bar() is returning a temporary value, which would be bad if it was able to be modified.

Using the r-value reference, we can add what is termed a “move constructor” and a “move assignment operator” to our class:

Foo::Foo(Foo&& f);
Foo& Foo::operator=(Foo&& f);

Note their arguments are both non-const, the reasons for which should become obvious.

Lets look at the copy constructor some more. For the sake of this example I will assume that Foo only contains an array and a variable storing its size. The copy constructor would look something like:

Foo::Foo(Foo&& f) {
  size=f.size;
  array=f.array;
  f.size=0;
  f.array=NULL;
}

What the move constructor has done is pilfered the resources of the temporary object, then reset the temporary object’s resources to being empty. This saves overhead of copying the array and setting the NULL prevents the memory being freed when the temporary object goes out of scope. The move assignment operator is exactly the same, except it frees any resources it currently owns first (like a normal assignment operator does).

How does this help our situation in the swap function above? We tell the compiler that it is OK to treat the right hand side of all the assignments as r-values using the std::move() function. So, the function becomes:

template swap(T& a, T& b) {
  T t(move(a));
  a = move(b);
  b = move(t);
}

Now at any time there is only one copy of a and b and the copying does not require any storage overhead.

Posted in C++11 on by Allan Comments Off