Tuesday, September 21, 2021

Experimental binary Gentoo package hosting (amd64)

IMPORTANT: This blog post is outdated! As direct result of this experiment, we now have official Gentoo binary packages available. See the announcement for more information and further links!  The old blog text follows...

As an experiment, I've started assembling a simple binary package hosting mechanism for Gentoo. Right now this comes with some serious limitations and should not be used for security or mission critical applications (more on this below). The main purpose of this experiment is to find out how well it works and where we need improvements in Portage's binary package handling.

So what do we have, and how can you use it?

  • The server builds an assortment of stable amd64 packages, with the use-flags as present in an unmodified 17.1/desktop/plasma/systemd profile (the only necessary change is USE=bindist).
  • The packages can be used on all amd64 profiles that differ from desktop/plasma/systemd only by use-flag settings. This includes 17.1, 17.1/desktop/*, 17.1/no-multilib, 17.1/systemd, but not anything containing selinx, hardened, developer, musl, or a different profile version such as 17.0.
  • Right now, the package set includes kde-plasma/plasma-meta, kde-apps/kde-apps-meta, app-office/libreoffice, media-gfx/gimp, media-gfx/inkscape, and of course all their dependencies. More will possibly be added.
  • CFLAGS are chosen such that the packages will be usable on all amd64 (i.e., x86-64) machines. 

To use the packages, I recommend the following steps: First, create a file /etc/portage/binrepos.conf with the following content:

[binhost]
priority = 9999
sync-uri = https://gentoo.osuosl.org/experimental/amd64/binpkg/default/linux/17.1/x86-64/

You can pick a different mirror according to your preferences (but also see the remarks below). Then, edit /etc/portage/make.conf, and add the following EMERGE_DEFAULT_OPTS (in addition to flags that you might already have there):

EMERGE_DEFAULT_OPTS="--binpkg-respect-use=y --getbinpkg=y"

And that's it. Your next update should download the package index and use binary packages whenever the versions and use-flag settings match. Everything else is compiled as usual.

What is still missing, and what are the limitations and caveats?

  • Obviously, the packages are not optimized for your processor.
  • Right now, the server only carries packages for the use-flag settings in an unmodified 17.1/desktop/plasma/systemd profile. If you use other settings, you will end up compiling part of your packages (which is not really a probem, you just lose the benefit of the binary download). It is technically possible to provide binary packages for different use-flag settings at the same URL, and eventually it will be implemented if this experiment succeeds.
  • At the moment, no cryptographic signing of the binary packages is in place yet. This is the main reason why I'm talking about an experiment. Effectively you trust our mirror admins and the https protocol. Package signing and verification is in preparation, and before the binary package hosting "moves into production", it will be enforced.
That's it. Enjoy! And don't forget to leave feedback in the comments.

Sunday, March 14, 2021

Gentoo AMD64 Handbook "Preparing the disks" section reworked

Since the text was becoming more and more outdated and also more and more convoluted, I have completely reworked the "Preparing the disks" section of the Gentoo AMD64 handbook

  • Since fdisk supports GUID partition tables (GPT) for a long time now, references to parted have been dropped.
  • The text restricts itself now to the combinations 1) UEFI boot and GPT and 2) BIOS / legacy boot and MBR. While mixing and matching is here certainly possible, we should treat it out of the scope of the manual.
  • Hopefully the terminology regarding the boot partition, UEFI system partition, and BIOS boot partition is more clear now (it was horribly mixed up before).

Please proofread and check for mistakes! I'll drop the "work in progress" label in a few days if nothing comes up.

Sunday, January 31, 2021

New Gentoo riscv (and arm) stages

With the help of our infrastructure team, I've finally managed to integrate the riscv stage builds with our signing and mirroring system. So now we have a riscv tab on the installation media download page, and the mirrors carry weekly signed stage3 archives for riscv64-lp64d and riscv64-lp64, in both openrc and systemd variants.

Using the same build infrastructure based on qemu, there are now also slowly updated stages for all arm variants coming to the mirrors. Please test them, and if anything does not work as expected, file bugs! The qemu-based builds are here a temporary measure; Matt Turner (mattst88) is preparing a fast multi-core arm64 machine, where this task will move to soon.