The interpersonal side of the xz-utils compromise

While everyone is busy analyzing the highly complex technical details of the recently discovered xz-utils compromise that is currently rocking the internet, it is worth looking at the underlying non-technical problems that make such a compromise possible. A very good write-up can be found on the blog of Rob Mensching...

"A Microcosm of the interactions in Open Source projects"

Testing the Gentoo LiveGUI with QEmu

If you want to test the Gentoo LiveGUI, but don't feel like cutting short the precious uptime of your machine, here's a handy bash script for testing the iso files with qemu:

#!/bin/bash

qemu-system-x86_64    -m 4G \
           -enable-kvm \
           -cpu host \
           -smp 4 \
           -name "Gentoo amd64 LiveGUI" \
           --netdev user,id=vmnic,hostname=gentoovm \
           -device e1000,netdev=vmnic \
           -bios /usr/share/edk2-ovmf/OVMF_CODE.fd \
           -device intel-hda -device hda-duplex \
           -usbdevice tablet \
           -vga vmware \
           -cdrom $1

You'll need to be in the kvm group and have kvm configured in your kernel. Also you may want to adapt the memory (now 4Gbyte) and the number of CPUs (now 4) for the virtual machine. Then you can simply start the boot process with

~ $ ./qemu-livegui livegui-amd64-20220412T191925Z.iso &

Cheers!

New Gentoo LiveGUI ISO and artwork / branding contest!

After a long break, we now have again a weekly LiveGUI ISO image for amd64 available! The 4.7 GB download, suitable for DVD burning or an USB stick, boots directly into KDE Plasma and comes with a ton of up-to-date software. This ranges from office applicactions such as LibreOffice, Inkscape, and Gimp all the way to many system administrator tools.

Now, we need your help! Let’s make this the coolest and most beautiful Linux live image ever. We’re calling for submissions of artwork, themes, actually anything from a desktop background to a boot manager animation, on the topic of Gentoo! The winning entry will be added as default setting to the official LiveGUI images, and also be available for download and installation.

The artwork contest

What are we looking for?

Gentoo-themed artwork and branding material to make our Gentoo LiveGUI the coolest Linux live medium ever.

• Incorporates the Gentoo logo and maybe other Gentoo design elements (like Larry the Cow)
• Works for a wide range of screen resolutions etc.
• Is packaged more or less ready-to-use for our LiveGUI image
• Provides a coherent experience to the user, i.e., if it consists of different parts, these fit togehter
• Can be distributed in its entirety under the CC BY-SA 4.0 license

We could for example imagine screen backgrounds, Plasma theming, maybe even a GRUB boot menu animation or a LibreOffice splash screen… Feel free to come up with more ideas.

If you base your work on freely available source material created by others, please keep track of the sources and their licenses in an accompanying readme file.

What are we not looking for?

• Do not submit anything that infringes on third-party copyrights or trademarks. While a Star Trek-themed Gentoo desktop would be cool, Paramount might object and we wouldn’t be able to distribute it. Same for My Little Pony or the Simpsons.
• Do not submit artwork falling under the not-safe-for-work (NSFW) category. We will recognize it when we see it, and we won’t be able to distribute it.
• Do not submit artwork with political or religious statements. No matter how universally acceptable you think that these are, someone will be offended by them.

The artwork should be such that kids or colleagues can walk into your office and you don’t have to quickly cover it up. :) Also, please think of your contribution in terms of the Gentoo Code of Conduct.

How to submit an entry

Package it up

• Package all the relevant files into a single tar archive and upload it to a webserver of your choice, or publish the files (e.g. on github) as a single git repository.
• Add a readme file with your name and contact e-mail address, the license of the files, sources and licenses for third-party material, and detailed installation instructions
• File a bug for the release engineering team, component “LiveCD/DVD”, with the summary starting with “Artwork 2022 contest entry”, and add a link to your file.
• If you link to a git repository, please mention a tag or commit which we should use.
• By submitting your entry, you allow Gentoo to download, re-publish, and distribute your files (see also above remark about the license).

Deadline

• The contest ends 31/May/2022 at 23:59 UTC.
• Please keep your files online for at least one more month after that date, so we can review and copy them.

Selection and announcement of the winner

• The jury consists of the Gentoo Council, the Release Engineering team, the Artwork team, and the Public Relations team (as of beginning of April 2022).
• The winner will be chosen by vote; depending on the amount and quality of the submissions, we may also pick a runner-up or more.
• The announcement of the winner or the winners will be made in June.

The LiveGUI image

The LiveGUI image is first and foremost provided to show off Gentoo and give everyone a chance to test a full-fledged Gentoo installation. As such, we have a lot of typical “desktop applications” installed. Additionally, we tried to integrate as many system administration tools as possible, so you can also use it for everything from repartitioning your hard drives to repairing an installation.

Some of the software on the image:

• KDE Plasma as desktop environment
• Office productivity: LibreOffice, LyX, TeXstudio, XournalPP, kile
• Web browsers: Firefox, Chromium
• IRC and similar: irssi, weechat
• Editors: Emacs, vim, kate, nano, joe
• Development and source control: git, subversion, gcc, Python, Perl
• Graphics: Inkscape, Gimp, Povray, Luminance HDR, Digikam
• Video: KDEnlive
• Disk management: hddtemp, testdisk, hdparm, nvme-cli, gparted, partimage, btrfs-progs, ddrescue, dosfstools, e2fsprogs, zfs
• Network tools and daemons: nmap, tcpdump, traceroute, minicom, pptpclient, bind-tools, cifs-utils, nfs-utils, ftp, chrony, ntp, openssh, rdesktop, openfortivpn, openvpn, tor
• Backup: mt-st, fsarchiver
• Benchmarks: bonnie, bonnie++, dbench, iozone, stress, tiobench
• …

The list of targeted packages (corresponding to a world file) can be found in the catalyst specification file; we install the newest stable version in the Gentoo repository.

In addition, since - as in a normal Gentoo installation - compiler and development tools are available, you can temporarily install more software. Just run emerge --sync and then install whatever you need (though it will be kept in memory and be gone after the next reboot).

Feedback and of course bug reports are welcome! Enjoy!

Akonadi for e-mail needs to die

So, I'm officially giving up on kmail2 (i.e., the Akonadi-based version of kmail) on the last one of my PCs now. I have tried hard and put in a lot of effort to get it working. However, it costs me a significant amount of time and effort just to be able to receive and read e-mail - meaning hanging IMAP resources every few minutes, the feared "Multiple merge candidates" bug popping up again and again, and other surprise events. That is plainly not acceptable in the workplace, where I need to rely on e-mail as means of communication. By leaving kmail2 I seem to be following many many other people... Even dedicated KDE enthusiasts that I know have by now migrated to Trojita or Thunderbird.

My conclusion after all these years, based on my personal experience, is that the usage of Akonadi for e-mail is a failed experiment. It was a nice idea in theory, and may work fine for some people. I am certain that a lot of effort has been put into improving it, I applaud the developers of both kmail and Akonadi for their tenaciousness and vision and definitely thank them for their work. Sadly, however, if something doesn't become robust and error-tolerant after over 5 (five) years of continuous development effort, the question pops up whether the initial architectural idea wasn't a bad one in the first place - in particular in terms of unhandleable complexity.

I am not sure why precisely in my case things turn out so badly. One possible candidate is the university mail server that I'm stuck with, running Novell Groupwise. I've seen rather odd behaviour in the IMAP replies in the past there. That said, there's the robustness principle for software to consider, and even if Groupwise were to do silly things, other IMAP clients seem to get along with it fine.

Recently I've heard some rumors about a new framework called Sink (or Akonadi-Next), which seems to be currently under development... I hope it'll be less fragile, and less overcomplexified. The choice of name is not really that convincing though (where did my e-mails go again)?

Now for the question and answer session...

Question: Why do you post such negative stuff? You are only discouraging our volunteers.
Answer: Because the motto of the drowned god doesn't apply to software. What is dead should better remain dead, and not suffer continuous revival efforts while users run away and the brand is damaged. Also, I'm a volunteer myself and invest a lot of time and effort into Linux. I've been seeing the resulting fallout. It likely scared off other prospective help.

Question: Have you tried restarting Akonadi? Have you tried clearing the Akonadi cache? Have you tried starting with a fresh database?
Answer: Yes. Yes. Yes. Many times. And yes to many more things. Did I mention that I spent a lot of time with that? I'll miss the akonadiconsole window. Or maybe not.

Question: Do you think kmail2 (the Akonadi-based kmail) can be saved somehow?
Answer: Maybe. One could suggest an additional agent as replacement to the usual IMAP module. Let's call it IMAP-stupid, and mandate that it uses only a bare minimum of server features and always runs in disconnected mode... Then again, I don't know the code, and don't know if that is feasible. Also, for some people kmail2 seems to work perfectly fine.

Question: So what e-mail program will you use now?
Answer: I will use kmail. I love kmail. Precisely, I will use Pali Rohar's noakonadi fork, which is based on kdepim 4.4. It is neither perfect nor bug-free, but accesses all my e-mail accounts reliably. This is what I've been using on my home desktop all the time (never upgraded) and what I downgraded my laptop to some time ago after losing many mails.

Question: So can you recommend running this ages-old kmail1 variant?
Answer: Yes and no. Yes, because (at least in my case) it seems to get the basic job done much more reliably. Yes, because it feels a lot snappier and produces far less random surprises. No, because it is essentially unmaintained, has some bugs, and is written for KDE 4, which is slowly going away. No, because Qt5-based kmail2 has more features and does look sexier. No, because you lose the useful Akonadi integration of addressbook and calendar.
That said, here are the two bugs of kmail1 that I find most annoying right now: 1) PGP/MIME cleartext signature is broken (at random some signatures are not verified correctly and/or bad signatures are produced), and 2), only in a Qt5 / Plasma environment, attachments don't open on click anymore, but can only be saved. (Which is odd since e.g. Okular as viewer is launched but never appears on screen, and the temporary file is written but immediately disappears... need to investigate.)

Question: I have bugfixes / patches for kmail1. What should I do?
Answer: Send them!!! I'll be happy to test and forward.

Question: What will you do when Qt4 / kdelibs goes away?
Answer: Dunno. Luckily I'm involved in packaging myself. :)

kmail 16.04.1 and Novell Groupwise 2014 IMAP server - anyone?

Here's a brief call for help.

Is there anyone out there who uses a recent kmail (I'm running 16.04.1 since yesterday, before that it was the latest KDE4 release) with a Novell Groupwise IMAP server?

I'm trying hard, I really like kmail and would like to keep using it, but for me right now it's extremely unstable (to the point of being unusable) - and I suspect by now that the server IMAP implementation is at least partially to blame. In the past I've seen definitive broken server behaviour (like negative IMAP uids), the feared "Multiple merge candidates" keeps popping up again and again, and the IMAP resource becomes unresponsive every few minutes...

So any datapoints of other kmail plus Groupwise imap users would be very much appreciated.

For reference, the server here is Novell Groupwise 2014 R2, version 14.2.0 11.3.2016, build number 123013.

Thanks!!!

VMware Workstation 11 and kwin - hangs and hiccups (solved?)

Since updating to VMware Workstation 11 (from the Gentoo vmware overlay), I've experienced a lot of hangs of my KDE environment whenever a virtual machine was running. Basically my system became unusable, which is bad if your workflow depends on accessing both Linux and (gasp!) Windows 7 (as guest). I first suspected a dbus timeout (doing the "stopwatch test" for 25s waits), but it seems according to some reports that this might be caused by buggy behavior in kwin (4.11.21). Sadly I haven't been able to pinpoint a specific bug report.

Now, I'm not sure if the problem is really 100% fixed, but at least now the lags are much smaller- and here's how to do it (kudos to matthewls and vrenn): 

• Add to /etc/xorg.conf in the Device section

  Option "TripleBuffer" "True"

• Create a file in /etc/profile.d with content

  __GL_YIELD="USLEEP"
  

  (yes that starts with a double underscore).
• Log out, stop your display manager, restart it.

I'll leave it as an exercise to the reader to figure out what these settings do. (Feel free to explain it in a comment. :) No guarantees of any kind. If this kills kittens you have been warned. Cheers.

KDEPIM without Akonadi

As you know, Gentoo is all about flexibility. You can run bleeding edge code (portage, our package manager, even provides you with installation from git master KF5 and friends) or you can focus on stability and trusted code. This is why we've been offering our users for the last years KDEPIM 4.4.11.1 (the version where KMail e-mail storage was not integrated with Akonadi yet, also known as KMail1) as a drop-in replacement for the newer versions.
Recently the Nepomuk search framework has been replaced by Baloo, and after some discussion we decided that for the Nepomuk-related packages it's now time to go. Problem is, the old KDEPIM packages still depend on it via their Akonadi version. This is why - for those of our users who prefer to run KDEPIM 4.4 / KMail1 - we've decided to switch to Pali Rohár's kdepim-noakonadi fork (see also his 2013 blog post and the code).The packages are right now in the KDE overlay, but will move to the main tree after a few days of testing and be treated as an update of KDEPIM 4.4.11.1.
The fork is essentially KDEPIM 4.4 including some additional bugfixes from the KDE/4.4 git branch, with KAddressbook patched back to KDEPIM 4.3 state and references to Akonadi removed elsewhere. This is in some ways a functionality regression since the integration of e.g. different calendar types is lost, however in that version it never really worked perfectly anyway.

For now, you will still need the akonadi-server package, since kdepimlibs (outside kdepim and now at version 4.14.9) requires it to build, but you'll never need to start the Akonadi server. As a consequence, Nepomuk support can be disabled everywhere, and the Nepomuk core and client and Akonadi client packages can be removed by the package manager (--depclean, make sure to first globally disable the nepomuk useflag and rebuild accordingly).

You might ask "Why are you still doing this?"... well. I've been told Akonadi and Baloo is working very nicely, and again I've considered upgrading all my installations... but then on my work desktop where I am using newest and greatest KDE4PIM bug 338658 pops up regularly and stops syncing of important folders. I just don't have the time to pointlessly dig deep into the Akonadi database every few days. So KMail1 it is, and I'll rather spend some time occasionally picking and backporting bugfixes.

LibreOffice KDE integration

I was this close to just masking the LibreOffice KDE integration in Gentoo and let it be done with that. Recently some really ugly bugs in the glue code crept up (crashes in the file dialog and in drag-n-drop), impacting productivity a lot... Asking around I heard that the KDE/Qt integration of LibreOffice was basically unmaintained and bitrotting.
Luckily someone stepped up and started bugfixing, and patches have made their way into the LibreOffice git master branch (and partially also into libreoffice-4.2). I have prepared a backport for the two bugs mentioned above; I'm flying blind and this is completely untested, but feedback on the bug reports would be very welcome! If you want to try (note, this affects only LibreOffice with USE=kde), keyword

=app-office/libreoffice-4.1.3.2-r2 **

somewhere in /etc/portage/package.keywords, start the update, and get some fish for your penguins; they will be busy for a while... :)

News from the 2013/05 Gentoo KDE team meeting

Last week we had our monthly Gentoo KDE team meeting; here are few details that are probably worth sharing.

• So far we've provided the useflag "semantic-desktop" which in particular controls the nepomuk functionality. Some components of KDE require this functionality unconditionally, and if you try to build without it, bugs and build failures may occur. In addition, by now it is easily and reliably possible to disable e.g. the file indexer at runtime. So, we've decided that starting with KDE 4.11 we will remove the useflag and hard-enable the functionality and the required dependencies in the ebuilds. The changes are being done already in the KDE overlay in the live ebuilds (which build upstream git master and form the templates for the upcoming 4.11 releases).
• After recent experiences the plan to drop kdepim-4.4 is off the table again. We will keep it in the portage tree as alternative version and try to support it until it finally breaks.
• In the meantime we (well, mainly Chris Reffett) have started in the KDE overlay to package Plasma Active, the tablet Plasma workspace environment. Since Gentoo ARM support is already excellent, this may become a highly valuable addition. Unfortunately, it's not really ready yet for the main tree and general use, but packaging work will continue in the overlay- what we need most is testing and bug reporting!

Independent of the meeting, a stabilization request has already been filed for KDE 4.10.3;  thanks to the work of the kde stable testers, we can keep everyone uptodate. And as a final note, my laptop is back to kmail1... Cheers!

Edit, 13/6/2013: Johu has posted a blog entry on how to disable the semantic desktop functionality at runtime.

personal experience and opinion - kmail2 still not suitable for on-the-road use :(

I was very sceptic for a long time. Then, I slowly started to trust the kmail2/akonadi combination. I've been using it on my office desktop for a long time, and it works well and is very stable and fast there. (Might be related to the fact that the IMAP server is just across the lawn.) Some time ago, when I deemed things solid enough I even upgraded my laptop again, despite earlier problems. In Gentoo, we've been keeping kdepim-4.4 around all the time, and as you may have read, internal discussions led indeed to the decision to finally drop it some time ago.
What happened in the meantime?
1) One of the more annoying bugs mentioned in my last blog post was fixed with some help from Kevin Kofler. Seems like Debian stumbled into the same issue long ago.
2) I was on vacation. Which was fun, but mostly unrelated to the issue at hand. None of my Gentoo colleagues went ahead with the removal in the meantime. A lot of e-mails accumulated in my account.
3) Coming back, I was on the train with my laptop, sorting the mail. The train was full, the onboard WLAN slightly overstressed, the 4G network just about more reliable. Network comes and goes sometime with a tunnel, no problem. Or so I thought.
4) Half an hour before arriving back home I realized that silently a large part of the e-mails that I had (I though) moved (using kmail2-4.10.3 / akonadi-1.9.2) from one folder to another over ~3 hours had disappeared on one side, and not re-appeared on the other. Restarting kmail2 and akonadi did not help. A quick check of the webmail interface of my provider confirmed that also on the IMAP server the mails were gone in both folders. &%(/&%(&/$/&%$§&/
I wasn't happy. Luckily there were daily server backup snapshots, and after a few days delay I had all the documents back. Nevertheless... Now, I am considering what to do next. (Needless to say, in my opinion we should forget dropping kmail1 in Gentoo for now.) Options...
a) migrate the laptop back to kmail1, which is way more resistant to dropped connections and flaky internet connection - doable but takes a bit of time
b) install OfflineIMAP and Dovecot on the laptop, and let kmail2/akonadi access the localhost Dovecot server - probably the most elegant solution but for the fact that OfflineIMAP seems to have trouble mirroring our Novell Groupwise IMAP server
c) other e-mail client? I've heard good things about trojita...
Summarizing... no idea still how to go ahead, no good solution available. And I actually like the kdepim integration idea, so I'll never be the first one to completely migrate away from it! I am sincerely sorry for the sure fact that this post is disheartening to all the people who put a lot of effort into improving kmail2 and akonadi. It has become a huge lot better. However, I am just getting more and more convinced that the complexity of this combined system is too much to handle and that kmail should never have gone the akonadi way.

kdepim-4.4 (kmail1) in Gentoo - unsupported, dying, dead

Bitrot is accumulating, and while we've tried to keep kdpim-4.4 running in Gentoo as long as possible, the time is slowly coming to say goodbye. In effect this is triggered by annoying problems like these:

• kleopatra-4.4 segfaults immediately on start
• kmail-4.4 does not respect "Sent mail" folder of identity 
• session restore always complains that another kontact-4.4 instance is running
• on each login, a dialog box about (unfounded) problems with gnupg pops up

There are probably many more such bugs around, where incompatibilities between kdepim-4.4 and kdepimlibs of more recent releases occur or other software updates have led to problems. Slowly it's getting painful, and definitely more painful than running a recent kdepim-4.10 (which has in my opinion improved quite a lot over the last major releases).
Please be prepared for the following steps:

• end of april 2013, all kdepim-4.4 packages in the Gentoo portage tree will be package.masked 
• end of may 2013, all kdepim-4.4 packages in the Gentoo portage tree will be removed
• afterwards, we will finally be able to simplify the eclasses a lot by removing the special handling

We still have the kdepim-4.7 upgrade guide around, and it also applies to the upgrade from kdepim-4.4 to any later version. Feel free to improve it or suggest improvements.

R.I.P. kmail1.

KDE 4.10.0 plasma-desktop crashes and qt frustrations

While on my machine KDE 4.10.0 runs perfectly fine, unfortunately a lot of Gentoo users see immediate crashes of plasma-desktop - which makes the graphical desktop environment completely unuseable. We know more or less what happened in the meantime, just not how to properly fix it...
The problem:

• plasma-desktop uses a new code path in 4.10, which triggers a Qt bug leading to immediate SIGSEGV. 
• The Qt bug only becomes fatal for some compiler options, and only on 64bit systems (amd64).
• The Qt bug may be a fundamental architectural problem that needs proper thought.

The links:

• Gentoo bug 450818: "x11-libs/qt-core-4.8.4: QDeclarativeVMEMetaObject::metaCall SIGSEGV when called from QMetaObject::metaCall"
• KDE bug 311751: "QML system tray crashes plasma-desktop on startup"
• Qt bug 29082: "QDeclarativeVMEMetaObject::metaCall SIGSEGV when called from QMetaObject::metaCall"
• Qt code review 46228: "Avoid calling metacall directly from the property animation code"

The bugfixing situation:

• Reverting the commit to plasma-workspace that introduced the problem makes the crash go away, but plasma-desktop starts hogging 100% CPU after a while. (This is done in plasma-workspace-4.10.0-r1 as a stopgap measure.) Kinda makes sense since the commit was there to fix a problem - now we hit the original problem.
• The bug seems not to occur if Qt is compiled with CFLAGS="-Os". Cause unknown. 
• David E. Narváez aka dmaggot wrote a patch for Qt that fixes this particular codepath but likely does not solve the global problem.
• So far comments from Qt upstream indicate that this is in their opinion not the right way to fix the problem.
• Our Gentoo Qt team understandably only wants to apply a patch if it has been accepted upstream.

Right now, the only option we (as Gentoo KDE team) have is wait for someone to pick up the phone. Either from KDE (to properly use the old codepath or provide some alternative), or from Qt (to fix the bug or apply a workaround)...

Sorry & stay tuned.

My personal KDEPIM upgrade (again): laptop

One year after my last blog post on this topic I encountered some minor difficulties with combining KDEPIM-4.4 (i.e. kmail1) and the KDE 4.10 betas. These difficulties are fixed now, and the combination seems to work fine again. Anyway, I became curious about the level of stability of Akonadi-based kmail2 once more. After all, I've been running it continuously over the year on my office desktop with a constant-on fast internet connection, and that works quite well. So, I gave it a fresh try on my laptop too. I deleted my Akonadi configuration and cache, switched to Akonadi mysql backend, updated kmail and the rest of KDEPIM without migrating to 4.9.4, and re-added my IMAP account from scratch (with "Enable offline mode"). The overall use case description is "laptop with large amount of cached files from IMAP account, fluctuating internet connectivity". Now, here are my impressions...

• Reaction time is occasionally sluggish, but overall OK.
• The progress indicator behaves a bit odd, it checks the mail folders in seemingly random order and only knows 0% and 100% completion.
• Random warning messages. It seems that kmail2 uses some features that "my" IMAP server does not understand. So, I'm getting frequent warning notifications that don't tell me anything and that I cannot do anything about. SET ANNOTATION, UID, ... Please either handle the errors, inform the user what exactly goes wrong, or ignore them in case they are irrelevant. Filed as a wish, bug 311265.
• Network activity stops working sometime. This sounds worse than it actually is, since in 99% of all cases Akonadi now detects fine that the connection to the server is broken (e.g., after suspend/resume, after switching to a different WLAN, or after enabling a VPN tunnel) and reconnects immediately. In the few remaining cases, re-starting the Akonadi server does the trick. You just have to know what to kick.
• More problematic is, while you're in online mode, any problems with connectivity will make kmail "hang". Clicking on a message leads to an attempt to retrieve it, which requires some response from the network. As it seems to me, all such requests are queued up for Akonadi to handle, and if that does not get a reply, pending requests are stuck in the queue... OK, you might say that this is a typical use case for offline mode, but then I would have to be able to predict when exactly my train enters the tunnel... Compare this to kmail1 disconnected IMAP accounts, where regular syncing would be delayed, but local work remained unaffected.
• Offline mode is a nice concept, and half a solution for the last problem, but unfortunately it does not work as expected. For mysterious reasons, a considerable part of the messages is not cached locally. I switch my account to offline mode, click on a message, and obtain an error message "Cannot fetch this in offline mode". Well, bummer. Bug 285935.
• This may just be my personal taste, but once something goes wrong (e.g., non-kde related crash, battery empty, ...) and the cache becomes corrupted somehow, I'd like to be able to do something from kmail2 without having to fiddle with akonadiconsole. A nice addition would be "Invalidate cache" in the context menu of a mail folder, or some sort of maintenance menu with semi-safe options.
• Finally... something is definitely going wrong with PGP signatures; the signatures do not always verify on other mail clients. Tracking this down, it seems that CRLF is not preserved in messages, see bug 306005.

On the whole, for the laptop use case the "new" KDEPIM is now (4.9.4) more mature than the last time I tried. I'll keep it now and not downgrade again, but there are still some significant rough edges. The good thing is, the KDEPIM developers are aware of the above issues and debugging is going on, as you can see for example from this blog post by Alex Fiestas (whose use case pretty much mirrors my own).

Calligra 2.4.0 is out!

Let's all congratulate the Calligra team for releasing Calligra 2.4.0, the first version of their integrated application suite for KDE. It has really gone a long way from the old KOffice codebase. Of course, Gentoo already has ebuilds as app-office/calligra. Give it a try! Cheers!

A neat trick for testing patches in Gentoo (source-based distros are great!)

Imagine you are running... say... Kubuntu, OpenSuSE, or Fedora, or any other of these famous and fantastic binary Linux distributions. You find a bug, report it to the authors of the package, and they come up with a possible solution. "Hey, here's a small patch to the source, could you please try if applying that helps?" Well... Either now you compile manually, which may or may not give the same packages as your distribution. Or you start setting up the full build system of your distro, which will take a while...
With Gentoo as a source-based distribution, however, this is very easy. Let's assume you want to apply a patch for more debug output to some kde plasma applets. The patch file is called debug.patch, the package name for the plasma applets is kde-base/plasma-apps. All you have to do is

mkdir -p /etc/portage/patches/kde-base/plasma-apps
cp debug.patch /etc/portage/patches/kde-base/plasma-apps/
emerge -a1 kde-base/plasma-apps

Finished. The patch is automatically applied during the emerge command:

 >>> Preparing source in /var/tmp/portage/kde-base/plasma-apps-4.8.2/work/plasma-apps-4.8.2 ...
 * Applying user patches from /etc/portage/patches//kde-base/plasma-apps ...
 *   debug.patch ...                      [ ok ]
 * Done with patching
>>> Source prepared.

Isn't that a nice trick? :)
Just one small warning... this does not work for all ebuilds in the portage tree yet, in case of doubt better check the build log if your patch was really applied. It does work for all ebuilds using base.eclass, which effectively includes all of KDE and all KDE applications.

Gentoo zero-day packaging of new KDE releases explained

Usually, whenever a new KDE release is published, Gentoo users can update already the same day, as suddenly a complete and polished set of ebuilds appears in the portage tree. (Stay tuned on upcoming wednesday for KDE 4.8.0, it's shaping up very nicely!) How is this possible? Well... let me explain.

If you're a stable version user, you may have never heard of so-called live ebuilds. This is a special variant, usually denoted by a version number ending in 9999, that does not rely on a source tarball. Instead, it contains a URL of a revsion control system (say on anongit.kde.org). When you emerge such a version of a package, the sources of the specified branch are checked out or updated to the newest upstream state, and that is used for building the installation package. Obviously this is not for everyone; depending how well upstream structures commits, things may not build for a while, contain fresh bugs, ... Also, reporting bugs from live versions on Gentoo bugzilla is discouraged as most of the times we can't do anything about it (do it only if you are sure it's a problem with the ebuilds, not with the source). If you're running live, you should be willing to hack yourself and work with upstream.

However, many of the Gentoo KDE team members run these live ebuilds, partly the current bugfix branch (i.e. KDE/4.8), partly even git master. They continuously keep the live ebuilds in the Gentoo KDE overlay updated to the newest state of the source. When a release is made, the corresponding live ebuilds of this branch are copied to the version ebuilds. For example, the KDE/4.8 branch live ebuilds have the version number 4.8.49.9999 (i.e.
kde-base/kdelibs-4.8.49.9999), so when the pre-release tarballs for KDE 4.8.0 were released to the packagers a few days ago, we only had to copy all 4.8.49.9999 ebuilds to 4.8.0 and immediately had a working set for testing. Most problems at that point are only caused by changes in tarball packaging. As distribution packagers get the pre-release tarballs (that still may change due to last-minute bugfixes) a week before the official release date, these can easily be fixed in time.

This also means that KDE maintenance in Gentoo is really a team effort. Whoever moves a released version to the main portage tree and/or commits bugfixes there builds on all the work that the team has done in the overlay in the meantime. Cheers!

My personal KDEPIM upgrade status

Some time ago when we were filing the first stable request for KDE 4.7 I decided I'd have to give KDEPIM-4.7 also a try. I used to be a pine (and later alpine) user for ages, some time ago I switched to kmail1 (maybe at version 4.2 or so) and have been using it ever since... About the setup, incoming mail is stored and filtered server-side (Novell Groupwise), and accessed as disconnected IMAP.

1) Office desktop.
This one's usually running the latest and greatest KDE RC or beta, with a static IP and something like a 100MBps-FD link to the mail server. No local folders, so this was the natural candidate to test first. I did not even try the data migration, but wiped my entire local configuration of the KDEPIM programs as thoroughly as possible before updating. On the whole this went rather well. From 4.7.3 to 4.7.95 I've seen my share of kmail2 / kontact / akonadi crashes, but none of them really led to bigger problems. Impressively, I could also see each of these crash bugs I hit get fixed on KDE bugzilla in the meantime! Since upgrading to 4.7.97 I haven't had a crash anymore.
The only thing that is really bugging me a the moment is that I absolutely can't drag an e-mail message into another folder (I always have to right-click on the entry in the message list, select "Move to folder", ...). However, that is likely a problem in one of the underlying libraries. :| I would really like to help debugging this problem; if you can give me any clue where to look, please message...

2) Laptop.
Going home for christmas meant spending some time on a train. The days before I had upgraded this box to QT-4.8.0 and KDE+KDEPIM 4.7.95. Again, clean start with a new configuration. On the train I spent some time ironing out bugs there. The GPRS connection was happily going down and up again with every tunnel or less-populated countryside. That's when the akonadi backend started really acting up. When I arrived at my family's place, my e-mail had become fully non-functional (no fetching e-mails, no sending e-mails, the backend making kmail hang, regular crashes, and all fumbling with akonadiconsole did not help). After a while of trying, I gave up, wiped my entire KDEPIM configuration and data and downgraded to 4.4 again. I was impressed how responsive my e-mail program suddenly became.
Talking to other people, it seems that some have problems with bad internet connections, some don't. Maybe I should have used networkmanager, at least Alexxy reports that it's working fine with it.

3) Home desktop.
Well... Here I store my e-mail archive since 1996- that's maildir folders with roughly 50000 messages. Maybe I'll consider upgrading KDEPIM sometime around KDE 4.9.

I'm sorry if this blog entry is demotivating for the program developers. Some time ago I followed part of an animated discussion on irc between a kdepim user and a kdepim developer. Quoting a small part,

<user> first: fix bugs. second:dont change data formats. third: change gui only if absolutly nessary.
<developer> no
<developer> first: have fun; second: make it work for others

While this is of course oversimplifying, both points have clear validity, the first because people have come to rely on the software, the second because it's volunteer work by people enthusiastic about their creation. In the end we'll have to find a good compromise.

kdepimlibs-4.7.3 imap4 regression fix

Attention Gentoo users: If you're running kmail-4.4.11.1 and kde-4.7.3, and if you have been hit by bug 382411 (problems with local filters, cannot upload unread message on imap; see also here and here), please keyword kde-base/kdepimlibs-4.7.3-r1 and give it a try. (You may want to log out and log back in.) This upgrade seems to fix the problem... If I get some positive feedback, I'll fast-track it for stabilization. Cheers!

Unit tests - a useful thing causing occasional pain

For whoever has never heard that expression, unit tests are a way to test the functionality of a program or library in an automated way, by for example checking the evaluation result of test data. This is a great thing and can help massively in identifying coding mistakes. CMake provides a mechanism, and in many KDE packages test routines are included.
So far, so good, this is the theory, now let's look at reality. In Gentoo, the main KDE distribution is split into 295 (!) packages. Many dont have any tests (oxygen-icons comes to my mind as a nice example). Of those that have, 37 now have all the test routines hard disabled, and I guess a few are missing that I still have to add to the list. Why? Well, first of all, you have to know that compilation usually runs in Gentoo as an unprivileged user detached from any graphical desktop session, and this user also runs the test routines. In addition, tests should also run fine "offline", i.e. without internet connection. One of our developers is running a great thing called a "tinderbox", where build and unit tests in various configurations are automated- in an isolated environment, and possibly even without network access. So, what can go wrong?

• The tests need X11. Well, actually we can get around this, with a nice hack called virtualx. A background framebuffer X server is started for the test phase only, and the tests are redirected to that display. Works even sometimes. :o)
• The tests need a dbus session bus. I think we could get around this too, but nobody bothered to do it so far and I'm not a dbus expert. 
• The tests need an entire KDE desktop to interact with. Yes, that happens, and unfortunately, we have to disable them in this case, see above. 
• The tests try to download a data set from, say, IMDB, CDDB, or similar. Bad, bad, bad, see above.
• Next, the tests pop up dialog boxes asking for user interaction. A window asking for a GnuPG passphrase is my personal favourite... 
• And last, what I particularly like is a unit test that fails... and after digging into the test log, I see a fat comment "This does not work yet and needs to be fixed!"

I guess at least part of the above points is a matter of policy. Test routines requiring user interaction can provide valid information, but of course they are not particularly useful for automated testing... Anyway...

PS. This blog post took slightly longer to write since a libreoffice-3.5 build in the background went berserk... load 65, >180 concurrent compilers... :D

Gentoo KDE 4.7.3 going stable now... including KDEPIM

Another KDE version stabilized in Gentoo... x86 is done with KDE 4.7.3 since this morning, amd64 is following right now. For those of you who don't read Portage news items, here's again my message about KDEPIM:

The stable upgrade from KDEPIM 4.4.11.1 to KDEPIM 4.7.3 is a MAJOR upgrade with potential for major breakage. Therefore we will try to keep and support the old, so-far stable KDEPIM 4.4.11.1 as long as possible.
If you don't want to upgrade your KDEPIM yet but keep the old version, please download this mask file and add it into your /etc/portage/package.mask.
If you decide to upgrade, please have a look at the upgrade guide first.

In addition there have been a few reports of problems with the Plasma desktop after upgrade. Usually these are caused by single plasmoids malfunctioning, so what should help is to remove some of the plasmoids from your desktop and add them again... Otherwise the upgrade from 4.6 to 4.7 should be pretty much unproblematic, and bring you a shiny new desktop with quite some new features.
With the new KDE we have finally also been able to stabilize a newer version of digikam and kipi-plugins (both at 2.3.0 now!), and also rekonq gets a big bump to version 0.8.0, which should fix an overdue security bug.
Finally and not related to the stabilization, since yesterday we have a package in the main tree that provides integration of Google calendars / contacts into Akonadi: kde-misc/akonadi-google ... It's a bit experimental still and also not an official release but a packaged snapshot, but anyway, if you are interested, give it a try!