EAPI=5 adoption has reached 50% of ebuilds in the portage tree

As of today, more than 50% of the 37527 ebuilds in the Gentoo portage tree use the newest ebuild API (EAPI) version, EAPI=5!
The details of the various EAPIs can be found in the package manager specification (PMS); the most notable new feature of EAPI 5, which has sped up acceptance a lot is the introduction of so-called subslots. A package A can specify a subslot, another package B that depends on it can specify that it needs to be rebuilt when the subslot of A changes. This leads to much more elegant solutions for many of the the link or installation path problems that revdep-rebuild, emerge @preserved-rebuild, or e.g. perl-cleaner try to solve... Another useful new feature in EAPI=5 is the masking of use-flags specifically for stable-marked ebuilds.
You can follow the adoption of EAPIs in the portage tree on an automatically updated graph page.

Perl in Gentoo: Upgrading pains, perl-cleaner, and EAPI=5

In a previous post, we've already looked at the structure of Perl ebuilds in Gentoo Linux. Now, let's see what happens in the case of a major Perl update.

Does this look familiar?

UPDATE THE PERL MODULES:
After updating dev-lang/perl you must reinstall
the installed perl modules.
Use: perl-cleaner --all

Then maybe you have updated your major Perl version recently, since this important message is printed by emerge afterwards. So, what is it about? In short, a certain disconnect between the "Perl way" of doing things and the rest of the world. Both have their merits, they just don't play very well with each other... and the result is that major Perl updates in Gentoo have traditionally also been a major pain. (This will become much better in the future, see below.)

Let's see where a perl package stores its files.

caipi ~ # equery files dev-perl/Email-Address
 * Searching for Email-Address in dev-perl ...
 * Contents of dev-perl/Email-Address-1.898.0:
/usr
/usr/lib
/usr/lib/perl5
/usr/lib/perl5/vendor_perl
/usr/lib/perl5/vendor_perl/5.16.3
/usr/lib/perl5/vendor_perl/5.16.3/Email
/usr/lib/perl5/vendor_perl/5.16.3/Email/Address.pm
/usr/share
/usr/share/doc
/usr/share/doc/Email-Address-1.898.0
/usr/share/doc/Email-Address-1.898.0/Changes.bz2
/usr/share/doc/Email-Address-1.898.0/README.bz2
caipi ~ #

Interesting- the installation path contains the Perl version! The reasons for upstream to do this are pretty much obvious, the application binary interface for compiled modules can change and it's necessary to keep the installed modules for different versions apart. Also, in theory you can keep different Perl versions installed in parallel. Nice idea, however if you have only one "system Perl" installation, and you exchange that for a newer version (say, 5.18.1 instead of 5.16.3), the result is that the new version won't find the installed packages anymore.

The results are rather annoying. Imagine you haven't updated your system for a while, one of the many packages to be updated is dev-lang/perl, and later maybe (just picking an example at random) gnome-base/gsettings-desktop-schemas. Perl is updated fine, but when portage arrives at building the gnome package, the build fails with something like

checking for perl >= 5.8.1... 5.18.2
checking for XML::Parser... configure: error: XML::Parser perl module is required for intltool

Right. Perl is updated, dev-perl/XML-Parser is still installed in the old path, and Perl doesn't find it. Bah.

Enter perl-cleaner, the traditional "solution". This small program checks for files in "outdated" Perl installation paths, finds out which packages they belong to, and makes portage rebuild the corresponding packages. During the rebuild, the installation is run by the updated Perl, which makes the files go into the new, now correct path.

This sounds like a good solution, but there are a lot of details and potential problems hidden. For once, most likely you'll run perl-cleaner after a failed emerge command, and some unrelated packages still need updates. Portage will try to figure out how to do this, but blockers and general weirdness may happen. Then, sometimes a package isn't needed with the new Perl version anymore, but perl-cleaner can't know that. Again the result may be a blocker. We've added the following instructions to the perl-cleaner output, which may help cleaning up the most frequent difficulties:

 * perl-cleaner is stopping here:
 * Fix the problem and start perl-cleaner again.
 *
 * If you encounter blockers involving virtuals and perl-core, here are
 * some things to try:
 *   Remove all perl-core packages from your world file
 *     emerge --deselect --ask $(qlist -IC 'perl-core/*')
 *   Update all the installed Perl virtuals
 *     emerge -uD1a $(qlist -IC 'virtual/perl-*')
 *   Afterwards re-run perl-cleaner

In the end, you may have to try several repeated emerge and perl-cleaner commands until you have an updated and consistent system again. So far, it always worked somehow with fiddling, but the situation was definitely not nice.

So what's the future? Well...

EAPI=5 brings the beautiful new feature of subslots and slot operator dependencies. In short, a package A may declare a subslot, and a package B that depends on A may declare "rebuild me if A changes subslot". This mechanism is now used to automate the Perl rebuilds directly from within emerge: dev-lang/perl declares a subslot corresponding to its major version, say "5.18", and every package that installs Perl modules needs to depend on it with the subslot-rebuild requested, e.g.

RDEPEND="dev-lang/perl:="

The good news about this is that portage now knows the dependency tree and can figure out the correct reinstallation order.

The bad news is, it can only work perfectly after all Perl packages have been converted to EAPI=5 and stabilized. perl-core is done, but with about 2100 ebuilds that use perl-module.eclass in the portage tree still quite some work remains. I've plotted the current EAPI distribution of ebuilds using perl-module.eclass in a pie chart for illustration... Maybe we're done when Perl 5.20 goes stable. Who knows. :)