C++11 – Part 7: Constant Expressions

Constant expressions are an interesting area of the C++11 standard. They really do not really allow you to achieve anything new with the language, but they do allow it to be achieved in a more simple fashion and extend the list of things that can be done at compile time making everything faster! Lets start off with a simple example of the sort of issues that needed solved:

std::size_t three() { return 3; }

A nice simple function… What might we do with such a function? How about:

const std::size_t i = three();

This is perfectly legal in C++03, but the assignment is evaluated at runtime. That means that many optimizations involving i might be missed. Another example:

int i[three()];

This is not legal C++03, although it may work depending on your compiler. Moving on to an example that definitely breaks!

template<size_t T> struct S {};
S<three()> s;

I do not know of a compiler that accepted that. You may think of these as a bit of a contrived example, but consider that things like std::numeric_limits<int>::max() are functions and running into these sorts of issues was not uncommon. There was a work around in the last case in terms of using C-style macros (e.g. INT_MAX) at the cost of losing type safety, or declaring a constant variable as in the first usage example above with all its disadvantages.

C++11 introduces the keyword constexpr to work around these issues. This is a guarantee that result of the function is a constant, provided the function is passed constant parameters. So, the function above can be declared as

constexpr std::size_t three() { return 3; }

Now the compiler knows that this function produces a constant so it is evaluated at compile time.

There are a few restrictions on what functions can be declared as constexpr. Essentially, it needs to a function that only contains single return statement that uses only literals, constexpr variables (defined later) and other constexpr functions. There are some other minor allowances for what the function may have such as (some) typedefs and using statements, but thinking of it as a one line function is probably best. Some recursion is allowed (compiler dependent but must be up to 512 levels), so calculating a factorial at compile time can done using:

constexpr int f(int x) { return x > 1 ? x * f(x - 1) : 1; }

Evaluating a factorial at compile time could previously be done by template metaprogramming, but that has some overhead and is much more difficult to specify. One thing to note is that the parameter of the factorial function does not need to be a constexpr, but the result will then be calculated at runtime.

This brings me to the difference between const variables and constexpr variables. Variables declared as constexpr are const but with some more restrictions. They must be immediately constructed (so nothing like extern constexpr int i;) and can only be constructed using literals and other constexpr variables and functions.

Now we know that every object that is used in a constexpr function must itself be a constexpr, but what about objects that have constructors? They can be used provided that their constructor is a constexpr so that it has only member initializations (using only other constexpr constructors if needed…). For example:

class Foo
{
  constexpr Foo(int foo) : _foo(foo) {}
  constexpr foo() { return _foo; }
private:
  int _foo;
};

With that class, a variable of type Foo can be declared as a constexpr. Also the Foo.foo() function is a constexpr and thus can be used anyway one is needed.

Great Theory

Tweet

My theory is any mailing list post that generates this many replies is probably the right way to go.

Posted in Tweet on by Allan Comments Off on Great Theory

Updating Arch Linux From A Core Install

NOTE: This post applied to the 2011.08 Arch Linux release. The current install ISO does not need the following steps, although they may still be useful for people attempting to upgrade an old install.

The move to /lib being a symlink to usr/lib in Arch Linux has been completely smooth… for those that followed the detailed instructions on the wiki. Not so smooth for those who had brilliant plans like deleting /lib and updating to the new glibc really quickly. What linker were you going to use to do that?

Anyway… it is now appears quite difficult to get a fully updated system from a core install. I say appears, because I did it in a couple of minutes. But I admit to actually knowing what I am doing! The simple solution is not to do a core install, which uses the packages that were available at the time of the installer creation. Instead do a net install and download the latest version of all the packages and be fully up to date straight away. Or, if you must do a core install, grab a newer testing install image.

If neither of these are an option, here are the steps required to update:

pacman -Sy
rm -rf /var/run /var/lock && pacman -Sf filesystem
pacman -S tzdata
pacman -U http://pkgbuild.com/~allan/glibc-2.16.0-1-i686.pkg.tar.xz
rm /etc/profile.d/locale.sh
pacman -Su --ignore glibc
pacman -Su

You might need to reboot after filesystem install. Also, change i686 to x86_64 in the pacman -U line if needed… Always say “N” to updating pacman first, and say “Y” to all the replacements.

All of that is given in the various news items about interventions required during updates since the last installer release. The only trick is getting the packages updated in the right order.

This is the price you pay for a rolling release distribution. Arch Linux does not get the benefit of being able to make changes like this at the time of a major release because we have none. We could probably automate some of these things with package update trickery… but it is our policy to not hide these things from the user. Finally, there is work being done on getting a new install image released more frequently so that upgrade paths in the future will always be simple.

C++11 – Part 6: Lambda Expressions

Lambda expressions provide a way to specify unnamed functions on the fly. While such a thing may seem a bit weird to those only familiar with the C family of languages, this is a fairly common feature of most programming languages.

The basic syntax of a lambda expression is:

[capture](arguments)->return-type{body}

Lets start with the common example, sorting by absolute value. Taking a std::vector<int> v, we can sort by absolute value using something like:

std::sort(v.begin(), v.end(), [](int x, int y) { return abs(x) < abs(y); } );

Walking through the parts of the lambda expression, firstly we have the capture field, []. This is a list of variables in the scope of the lambda function that are able to be accessed in the expression. In this case there is noting captured, so the capture field is empty. If we want to use variables in the scope of the lambda function, they are specified in a comma separated list within the []. For example, to calculate the sum of all items in a std::vector<int>, we could use:

int total = 0;
std::for_each(v.begin(), v.end(), [&total](int x) { total += x; } );

Note that by default variables are captured by value. The & in front of total tells the lambda function to capture it by reference and thus allow it to be changed. To capture all variables by value, use [=] and use [&] to capture all by reference. The later values in the capture field override the earlier ones, so [=, &x] captures all variables by value except for x which is captured by reference.

The arguments passed to the lambda function are just like the arguments of a normal function. Of course, the lambda function can be passed no arguments by using (). An optional return type can be provided using the suffix return syntax introduced earlier in this series. If not specified, the return type is deduced from the return statement (much like using auto), or is void if no value is returned.

That leaves the function body. Technically, any block of code can be put in the function body, but you are probably doing it wrong if the function is going to be more than a couple of lines long...

Posted in C++11 on by Allan Comments Off on C++11 – Part 6: Lambda Expressions

The Arch Linux [testing] Repo Is For Testing!

Given the number of unpleasant emails I have received over the last few days, it appears that people have forgotten what the purpose of the [testing] repo in Arch Linux is for. Wait for it…. testing packages! Who would have thought!

Now to clarify some things. While the [testing] repo is for testing, we do our best not to break things in it. In fact, the [staging] repo was set up to allow rebuilds to occur away from the main repos and allow [testing] to always be in a usable state. Also, many (maybe even most) of the Arch developers use the [testing] repo on their main computer. I know I use it on all of mine, including on my work laptop. So I really do not want things to break either. And of course, if I cause breakage, that limits my ability to yell at other people who break packages.

One of the common criticisms I got in my “fan mail” was that I did not do enough testing of the change that saw the /lib directory change into a symlink to /usr/lib. Perhaps the need for testing was why I put the package in the [testing] repo… But, also I did do a fair amount of testing of this update. I tested that pacman stopped the update when there was a file in /lib (or in a subdirectory of /lib), both when the file was owned or unowned by a package. I checked that pacman -Syu --ignore glibc && pacman -Su worked from and old base install.

What I did not test was using pacman -Sf glibc to resolve a conflict, mainly because even with my low expectations on the general population’s intelligence, I did not expect Arch Linux [testing] users to be quite that incompetent. I also did not test having another package owning the /lib directory or a subdirectory within that folder but with no actual files in it (or having its files manually moved from in it). And I did not test upgrading from a more complete install that could have packages with versioned dependencies on glibc (which does not actually break the update but makes it a bit more difficult).

Now ignoring the usage of -Sf… the only case I did not test that actually causes breakage was an empty subdirectory in /lib or another package owning /lib with no actual files in there. That should never actually happen, but it appears people manually moved module or firmware files from /lib to /usr/lib without fixing thier packages, creating this situation. In both these cases, pacman would get to extracting the /lib symlink, see there was still a directory there and refuse to extract. That left people with a system looking for the linker in /lib, but it not being there. Annoying, but this situation is easily recoverable, even without a rescue CD.

You may ask why pacman did not detect those conflicts before attempting the extraction of the package. That would be a good question… It seems the situation of changing a directory to a file or symlink is not a very common operation and so was not very well tested in the (quite extensive) pacman test-suite. Also, a bug fix in pacman 4.0 (prevent removal of empty directories if they are owned by another package), will have caused this bug to be exposed much more frequently. A couple of patches to pacman and these conflicts are now caught prior to the upgrade taking place. These patches are now applied to our pacman package, so non-testing users will not be exposed to this issue.

So a couple of things to note… Firstly, this was not a bug in the glibc package. Secondly, I fixed the bugs in the pacman conflict checking code. So… just maybe… I am not as “incompetent” as the emails I received claimed and I should not be “forced to stand down” as an Arch Linux developer.

Finally, if you are going to send angry emails to me, at least attempt to make them well enough worded that they can join my “angry email hall of fame”. The latest batch were surprisingly uncreative… In fact, the two emails claiming I would be responsible for the authors failing a course due to them spending time fixing their system instead of doing work were just stupid. Why would you update right before a major assignment is due and why would you send me an email if you are running late? Also, the style of the emails makes me think this was one person sending from two different accounts, so they seem to have plenty of spare time… I did enjoy a comment in the bug tracker that said the glibc maintainer was going postal.

Using New Spam Control

A couple of people have emailed me saying that their comment was not posted to my blog. I have WordPress set to require all comments from a person to be approved, although no further approval is needed for that person to leave subsequent comments. I do not censoring your comments at all. In fact, I let all genuine comments through, troll or not.

So what is happening to the comments that never appear? I have been using two measures to counter spam. Firstly, all posts are closed for commenting after 30 days. Spam does not start arriving on most posts for about a week, and really becomes abundant after about 20 days, so this gets rid of most spam. Secondly, I have been using the Askimet plug-in. This was doing a very good job at sorting spam from non-spam. However, the set-up required at my hosting providers end for enabling SSL for my WordPress administration resulted in all my comments being recorded as coming from the one IP address. That really screws Askimet over, so all comments were being marked as spam. This requires me to manually check my spam comments before I delete them and it seems I was being over-zealous…

So attempt two at preventing spam! Enter the Quiz plugin. Anybody leaving a comment will be required to answer an “extremely tricky” question to prove they are not a spammer. I will still have the moderation turned on for the time being to ensure this catches the spam, but I will turn that off too if successful.

Long Day

Tweet

Feeling tired – it was a long day yesterday!

Posted in Tweet on by Allan Comments Off on Long Day

Bargin

Tweet

Thanks Amazon, I must rush and buy that book now!

“Price dropped by $0.06 (was $26.07 when added)”

Posted in Tweet on by Allan Comments Off on Bargin

C++11 – Part 5: Initialization

C++11 bring with it a great improvement in initializing data. I’ll first cover initializing containers and then move on to more general variable initialization.

Initializer lists
Arrays could always be initialized using C style initialization:

int arr[] = {1,2,3,4,5};

But, that syntax could not be used with containers. That seemed a bit silly given that (e.g.) a std::vector is just a fancy array, so losing features that arrays have is bad. C++11 allows this type of initialization for all containers. You can also add this type of initialization for any of your classes, or even as an argument to a function. This works by the {} initialization works is that it creates an object of a new class std::initializer_list. So for this to be used in your class, you just need to add the relevant constructor. E.g.:

template<class t> class Vec
public:
  Vec(std::initializer_list i)
  {
  size = i.size();  // set the size of the array in Vec
  reserve(size);    // allocate the required space in the array
  uninitialized_copy(i.begin(), i.end(), array);
  }
// ...

Defining a function that takes an initializer list as an argument is similarly easy.

Uniform Initialization
In C++03, initialization of variables was all over the place. Initializer lists improve on that somewhat, but C++11 takes it one step further. Now everything can be initialized in much the same way. Some cases taken straight from the proposal:

  • Variable initialization; e.g., X x {v};
  • Initialization of a temporary; e.g, X{v}
  • Explicit type conversion; e.g. x = X{v};
  • Free store allocation; e.g. p = new X{v}

There are a few situations where I see this as being particularly useful:

1. Initialisation of dynamically allocated arrays:
int *p = new int[3]{1,2,3};

2. Initialization of an array member variable:
class C {
  int a[3];
public:
  C(int x, int y, int z) : a{x, y, z} { /* ... */ };
};

3. Implicitly initialize objects to return:
return {foo, bar};

4. Implicitly initialize a function parameter:
f({foo, bar});

There are some slight traps with the use of this initialization syntax with classes using constructors. For example, take the class C in case #2 above. The following two statements are equivalent:

C c1(1,2,3);
C c2{1,2,3};

However, for std::vector, which is able to be initialized using an initializer list, the following are different:

std::vector v1(10,3); // vector of length 10
std::vector v1{10,3}; // vector of length 2

Exhausting

Tweet

Attempting to repeatedly stab someone with a knife takes more energy than you would expect.

Posted in Tweet on by Allan Comments Off on Exhausting