Thursday, December 13, 2012

How app-office/libreoffice-bin is made

While usually Gentoo users compile all their packages on their own computers, LibreOffice tends to be too big a bite for that. This is why we provide for amd64 and x86 app-office/libreoffice-bin and app-office/libreoffice-bin-debug, two packages with a precompiled binary installation and its debug information. In the beginning we just used the binaries from the official LibreOffice distribution. Turns out, however, that these binaries bundle a large number of libraries that we have in Gentoo anyway (bug 361695), and for a lot of reasons bundled libraries are bad. So, we decided to roll our own binaries for stable Gentoo installations. Let me describe a bit how it is done.
Linux pinacolada 3.4.9-gentoo #2 SMP Thu Oct 11 00:05:55 CEST 2012 x86_64 Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz GenuineIntel GNU/Linux
On the machine doing the build, two chroots are dedicated to the package build process, one a plain amd64 chroot, the other an x86 chroot entered via linux32. Both have no ~arch packages installed at all, only stable keywords are accepted; both have a very minimal world file listing only a few packages useful for a maintainer as e.g. gentoolkit or eix. Procedure is identical for both.  In addition, in both chroots the compiler flags are chosen for as wide compatibility as possible. This means
# for x86
CFLAGS="-march=i586 -mtune=generic -O2 -pipe -g"
# for amd64
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -g"
and obviously the same for CXXFLAGS. Both chroots also use the portage features splitdebug and compressdebug to make debug information available in a separate directory tree. Prior to build, the existing packages are updated, unnecessary packages are cleaned, and dynamic linking is checked:
emerge --sync
emerge -uDNav world
emerge --depclean --ask

revdep-rebuild
In case any problems occur, these are checked, solved, and the procedure is repeated until all the operations become a no-op.
Next step is adapting the (rather simplistic) build script to the new libreoffice version. This means mainly checking for new or discarded useflags, and deciding which value these should have in the binary build. Since LibreOffice-3.6 we also have to decide now which bundled extensions to build. The choice of useflags is influenced by several factors. For example, pdfimport is disabled because the resulting dependency on poppler might lead to broken binaries rather too often.
Then, well, then it's running the build. Generating all 12 flavours (base, kde, gnome with and without java for both amd64 and x86) takes roughly a weekend. Time to go out to the christmas market and sip a Glühwein.
In the meantime, we can also adapt the libreoffice-bin ebuilds for the new version. The defined phase functions are mostly boring, since they only have to copy files into the system. Normally, they can be taken over from the previous version. The dependency declarations, however, have to be copied anew each time from the corresponding app-office/libreoffice ebuild, taking into account the chosen use-flag values. DEPEND is set empty since we're not actually building anything during installation.
Finally, COMMON_DEPEND is extended by an additional block named BIN_COMMON_DEPEND, specific for the binary package. Here, we specify any dependencies that need to be stricter now, where a library upgrade would for a normal package require revdep-rebuild - which is not possible for a binary package. Typical candidates where we have to fix the minimum or exact library version are glibc, icu, or libcmis.
Once the build has finished, 8.8G of files have to be uploaded to the Gentoo server, added to the mirror system, and then given some time to propagate. Then, we can commit the new ebuild, and open a stabilization request bug. Finished!
(Oh and in case you're wondering, new packages are coming tomorrow. :)

6 comments:

  1. Hi, don't you use dev-util/ccache for the different USE profiles?

    ReplyDelete
    Replies
    1. No, because ccache can introduce quite a lot of hard-to-debug problems.

      Delete
  2. Thanks for sharing! Very interesting.

    ReplyDelete
  3. Really interesting!! This is something I always wanted to know.

    ReplyDelete
  4. What about add USE to (download and) install only the selected applications? With the installer you can it and also in debian. For instance I would like to download and install only writer and calc.

    ReplyDelete
    Replies
    1. That is technically possible, but requires someone to do (for each version) the analysis and work to split everything correctly. Out of my free-time league right now... but imho 65M is not that bad.

      Delete