Lab::Measurement 3.900 released - also dropping support for legacy scripts

Following a lot of smaller and larger improvements and additions, we have finally released Lab::Measurement 3.900 on CPAN. Highlights compared to the last "round version" include support for the Nanonis Tramea quantum transport measurement, the Zurich Instruments HDAWG arbitrary waveform generator, the Synktek MCL1-540 lock-in amplifier, the Bluefors dilution refrigerator temperature control, the American Magnetics AMI430 magnet power supply, as well as many improvements for Lakeshore temperature controllers.

In addition, version 3.900 now finally drops the unmaintained legacy code layers (Lab::Bus, Lab::Connection, Lab::Instrument, Lab::XPRESS) and focuses on the Moose-based Modern Perl interface alone. This should make maintenance easier in the future. Anyone who still needs the old interfaces for measurement scripts should install the distribution Lab::Measurement::Legacy, where the deprecated modules are archived.

Comp. Phys. Comm. published: "Lab::Measurement - a portable and extensible framework for controlling lab equipment and conducting measurements"

We're happy to announce that our article "Lab::Measurement — a portable and extensible framework for controlling lab equipment and conducting measurements", describing our measurement software package Lab::Measurement, has been published in Computer Physics Communications.

Lab::Measurement is a collection of object-oriented Perl 5 modules for controlling lab instruments, performing measurements, and recording and plotting the resultant data. Its operating system independent driver stack makes it possible to use nearly identical measurement scripts both on Linux and Windows. Foreground operation with live plotting and background operation for, e.g., process control are supported. For more details, please read our article, visit the Lab::Measurement homepage, or visit Lab::Measurement on CPAN!

"Lab::Measurement - a portable and extensible framework for controlling lab equipment and conducting measurements"
S. Reinhardt, C. Butschkow, S. Geissler, A. Dirnaichner, F. Olbrich, C. Lane, D. Schröer, and A. K. Hüttel
Comp. Phys. Comm. 234, 216 (2019); arXiv:1804.03321 (PDF)

Manuscript on Lab::Measurement submitted for publication

Today's news is that we have submitted a manuscript for publication, describing Lab::Measurement and with it our approach towards fast, flexible, and platform-independent measuring with Perl! The manuscript mainly focuses on the new, Moose-based class hierarchy. We have uploaded it to arXiv as well; here is the (for now) full bibliographic information of the preprint:

 "Lab::Measurement - a portable and extensible framework for controlling lab equipment and conducting measurements"
S. Reinhardt, C. Butschkow, S. Geissler, A. Dirnaichner, F. Olbrich, C. Lane, D. Schröer, and A. K. Hüttel
submitted for publication; arXiv:1804.03321 (PDF, BibTeX entry)

If you're using Lab::Measurement in your lab, and this results in some nice publication, then we'd be very grateful for a citation of our work - for now the preprint, and later hopefully the accepted version.

DPG 2018, the proof: Yes we've been in Berlin!

The 2018 spring meeting of the DPG condensed matter physics section in Berlin is over, and we've all listened to interesting talks and seen exciting physics. And we've also presented the Lab::Measurement poster! Here's the photo proof of Simon explaining our software... click on the image or the link for a larger version!

FOSDEM 2018 talk: Perl in the Physics Lab

FOSDEM 2018, the "Free and Open Source Developers' European Meeting", takes place 3-4 February at Universite Libre de Bruxelles, Campus Solbosch, Brussels - and our measurement control software Lab::Measurement will be presented there in the Perl devrooom! As all of FOSDEM, the talk will also be streamed live and archived; more details on this follow later. Here's the abstract:

Perl in the Physics Lab
Andreas K. Hüttel
    Track: Perl Programming Languages devroom
    Room: K.4.601
    Day: Sunday
    Start: 11:00
    End: 11:40 

Let's visit our university lab. We work on low-temperature nanophysics and transport spectroscopy, typically measuring current through experimental chip structures. That involves cooling and temperature control, dc voltage sources, multimeters, high-frequency sources, superconducting magnets, and a lot more fun equipment. A desktop computer controls the experiment and records and evaluates data.

Some people (like me) want to use Linux, some want to use Windows. Not everyone knows Perl, not everyone has equal programming skills, not everyone knows equally much about the measurement hardware. I'm going to present our solution for this, Lab::Measurement. We implement a layer structure of Perl modules, all the way from the hardware access and the implementation of device-specific command sets to high level measurement control with live plotting and metadata tracking. Current work focuses on a port of the entire stack to Moose, to simplify and improve the code.

Lab::Measurement 3.550 released - port to Dist::Zilla

With the release of Lab::Measurement 3.550 we've switched to Dist::Zilla as maintenance tool. If you're not involved in hacking Lab::Measurement, you should essentially not notice this change. However, for the authors of the package, Dist::Zilla makes it much easier to keep track of dependencies, prepare new releases, and eventually also improve and unify the documentation... At the side we've also fixed Issue 4 and Lab::Measurement should now work out of the box with recent Gnuplot on Windows again.

Lab::Measurement 3.544 released - support for Zurich Instruments devices

I've just uploaded Lab::Measurement 3.544 to CPAN. This is our first release containing support for Zurich Instruments equipment, in particular the ZI MFLI digital lock-in amplifier, via Simon's Lab::Zhinst package. Enjoy!

Testers wanted for Gentoo Perl 5.24.1 stabilization

We want to stabilize Perl 5.24 on Gentoo pretty soon (meaning in a few weeks), and do actually not expect any big surprises there. If you are running a stable installation, are willing to do some testing, and are familiar with our Gentoo bugzilla and with filing bug reports, then you might just be the right volunteer to give it a try in advance!

Here's what to do:

Step 1: Update app-admin/perl-cleaner to current ~arch.
I'm deliberately not supplying any version number here, since I might do another release, but you should at least have perl-cleaner-2.25.

Step 2: Make sure your system is uptodate (emerge -uDNav world) and do a depclean step (emerge --depclean --ask).

Step 3: Download the current stabilization list from bug 604602 and place it into your /etc/portage/package.keywords or /etc/portage/package.accept_keywords.

Step 4: Update your world (emerge -uDNav world), which triggers the perl update and the module rebuild.

Step 5: Run "perl-cleaner --all"  (you might also want to try "perl-cleaner --all --delete-leftovers").

... and make sure you file bugs for any problems you encounter, during the update and afterwards! Feedback is also appreciated if all goes fine; then you best leave a comment here on the blog post.

Lab::Measurement at the DPG Spring Meeting Dresden 2017

As nearly every year we'll present a poster on Lab::Measurement at the spring meeting of the German Physical Society again. This time the conference is in Dresden - so visit us on upcoming 23 March 2017, 15:00-19:00, poster session TT75, poster TT75.7!

Please test www-apache/mod_perl-2.0.10_pre201601

We're trying to get both Perl 5.22 and Apache 2.4 stable on Gentoo these days. One thing that would be really useful is to have a www-apache/mod_perl that works with all current Perl and Apache versions... and there's a candidate for that: a snapshot of what is hopefully going to be mod_perl-2.0.10 pretty soon. So...

Please keyword (if necessary) and test www-apache/mod_perl-2.0.10_pre201601!!! Feedback for all Perl and Apache versions is very much appreciated. Gentoo developers can directly edit our compatibility table with the results, everyone else please comment on this blog post or file bugs in case of problems!

Please always include exact www-servers/apache, dev-lang/perl, and www-apache/mod_perl versions!

Perl 5.22 testers needed!

Gentoo users rejoice, for a few days already we have Perl 5.22.0 packaged in the main tree. Since we don't know yet how much stuff will break because of the update, it is masked for now. Which means, we need daring testers (preferably running ~arch systems, stable is also fine but may need more work on your part to get things running) who unmask the new Perl, upgrade, and file bugs if needed!!!
Here's what you need in /etc/portage/package.unmask (and possibly package.accept_keywords) to get started (download); please always use the full block, since partial unmasking will lead to chaos. We're looking forward to your feedback!

# Perl 5.22.0 mask / unmask block
=dev-lang/perl-5.22.0
=virtual/perl-Archive-Tar-2.40.0
=virtual/perl-Attribute-Handlers-0.970.0
=virtual/perl-B-Debug-1.230.0
=virtual/perl-CPAN-2.110.0
=virtual/perl-CPAN-Meta-2.150.1
=virtual/perl-CPAN-Meta-Requirements-2.132.0
=virtual/perl-Carp-1.360.0
=virtual/perl-Compress-Raw-Bzip2-2.68.0
=virtual/perl-Compress-Raw-Zlib-2.68.0
=virtual/perl-DB_File-1.835.0
=virtual/perl-Data-Dumper-2.158.0
=virtual/perl-Devel-PPPort-3.310.0
=virtual/perl-Digest-MD5-2.540.0
=virtual/perl-Digest-SHA-5.950.0
=virtual/perl-Exporter-5.720.0
=virtual/perl-ExtUtils-CBuilder-0.280.221
=virtual/perl-ExtUtils-Command-1.200.0
=virtual/perl-ExtUtils-Install-2.40.0
=virtual/perl-ExtUtils-MakeMaker-7.40.100_rc
=virtual/perl-ExtUtils-ParseXS-3.280.0
=virtual/perl-File-Spec-3.560.0
=virtual/perl-Filter-Simple-0.920.0
=virtual/perl-Getopt-Long-2.450.0
=virtual/perl-HTTP-Tiny-0.54.0
=virtual/perl-IO-1.350.0
=virtual/perl-IO-Compress-2.68.0
=virtual/perl-IO-Socket-IP-0.370.0
=virtual/perl-JSON-PP-2.273.0
=virtual/perl-Locale-Maketext-1.260.0
=virtual/perl-MIME-Base64-3.150.0
=virtual/perl-Math-BigInt-1.999.700
=virtual/perl-Math-BigRat-0.260.800
=virtual/perl-Module-Load-Conditional-0.640.0
=virtual/perl-Module-Metadata-1.0.26
=virtual/perl-Perl-OSType-1.8.0
=virtual/perl-Pod-Escapes-1.70.0
=virtual/perl-Pod-Parser-1.630.0
=virtual/perl-Pod-Simple-3.290.0
=virtual/perl-Safe-2.390.0
=virtual/perl-Scalar-List-Utils-1.410.0
=virtual/perl-Socket-2.18.0
=virtual/perl-Storable-2.530.0
=virtual/perl-Term-ANSIColor-4.30.0
=virtual/perl-Term-ReadLine-1.150.0
=virtual/perl-Test-Harness-3.350.0
=virtual/perl-Test-Simple-1.1.14
=virtual/perl-Text-Balanced-2.30.0
=virtual/perl-Text-ParseWords-3.300.0
=virtual/perl-Time-Piece-1.290.0
=virtual/perl-Unicode-Collate-1.120.0
=virtual/perl-XSLoader-0.200.0
=virtual/perl-autodie-2.260.0
=virtual/perl-bignum-0.390.0
=virtual/perl-if-0.60.400
=virtual/perl-libnet-3.50.0
=virtual/perl-parent-0.232.0
=virtual/perl-threads-2.10.0
=virtual/perl-threads-shared-1.480.0
=dev-perl/Test-Tester-0.114.0
=dev-perl/Test-use-ok-0.160.0
# end of the Perl 5.22.0 mask / unmask block

After the update, first run

emerge --depclean --ask

and afterwards

perl-cleaner --all

perl-cleaner should not need to do anything, ideally. If you have depcleaned first and it still wants to rebuild something, that's a bug. Please file a bug report for the package that is getting rebuilt (but check our wiki page on known Perl 5.22 issues first to avoid duplicates).

Testers needed for dev-lang/perl-5.20.1-r4 (stabilization candidate)

Most of us in the Gentoo Perl packaging team are already running ~arch Perl even on otherwise stable machines, and Perl 5.20 is looking very good so far. Our current plan is to wait for another month or similar and file the stabilization request for it in February. This would be a real achievement, since we'd at that time actually have the latest and greatest upstream stable Perl release also stable in Gentoo; this hasn't been the case for a very long time.
Of course, we need testers for that; the architecture teams cannot possibly try out all Perl programs in Gentoo with the new version. So, if you're feeling adventurous, and if you are running a fully updated stable system, please help us!
What do you need to do? First, upgrade perl-cleaner to ~arch by placing the following line in your package.keywords (or package.accept_keywords)

app-admin/perl-cleaner

and updating perl-cleaner (to currently 2.19):

emerge -u1a perl-cleaner

Then, upgrade Perl (and only Perl) to ~arch by placing the following exact three lines in your package.keywords (or package.accept_keywords):

dev-lang/perl
virtual/perl-*
perl-core/*

Then, upgrade your system with

emerge -uDNav world
perl-cleaner --all

This should now already be much easier than with previous Perl versions. In theory, all Perl packages should be rebuilt by emerge via the subslot rebuild mechanism, and perl-cleaner should not find anything to do anymore, but we cannot be 100% sure of that yet so far. (Looking forward to feedback.)
Well, and then use Perl and use your system, and if you encounter any problems, file bugs!!!

A final remark, once Perl 5.20 becomes stable you may want to remove above keywording lines from your portage configuration again.

RDepending on Perl itself

Writing correct dependency specifications is an art in itself. So, here's a small guide for Gentoo developers how to specify runtime dependencies on dev-lang/perl. First, the general rule.
Check the following two things: 1) is your package linking anywhere to libperl.so, 2) is your package installing any Perl modules into Perl's vendor directory (e.g., /usr/lib64/perl5/vendor_perl/5.20.1/)? If at least one of these two questions is answered with yes, you need in your dependency string a slot operator, i.e. "dev-lang/perl:=" Obviously, your ebuild will have to be EAPI=5 for that. If neither 1) nor 2) are the case, "dev-lang/perl" is enough.
Now, with eclasses. If you use perl-module.eclass or perl-app.eclass, two variables control automatic adding of dependencies. GENTOO_DEPEND_ON_PERL sets whether the eclass automatically adds a dependency on Perl, and defaults to yes in both cases. GENTOO_DEPEND_ON_PERL_SUBSLOT controls whether the slot operator ":=" is used. It defaults to yes in perl-module.eclass and to no in perl-app.eclass. (This is actually the only difference between the eclasses.) The idea behind that is that a Perl module package always installs modules into vendor_dir, while an application can have its own separate installation path for its modules or not install any modules at all.
In many cases, if a package installs Perl modules you'll need Perl at build time as well since the module build system is written in Perl. If a package links to Perl, that is obviously needed at build time too.

So, summarizing:

eclass	1) or 2) true	1) false, 2) false
none	"dev-lang/perl:=" needed in RDEPEND and most likely also DEPEND	"dev-lang/perl" needed in RDEPEND, maybe also in DEPEND
perl-module.eclass	no need to do anything	GENTOO_DEPEND_ON_PERL_SUBSLOT=no possible before inherit
perl-app.eclass	GENTOO_DEPEND_ON_PERL_SUBSLOT=yes needed before inherit	no need to do anything

How do I run ~arch Perl on a stable Gentoo system?

Here's a small piece of advice for all who want to upgrade their Perl to the very newest available, but still keep running an otherwise stable Gentoo installation: These three lines are exactly what needs to go into /etc/portage/package.keywords:

dev-lang/perl
virtual/perl-*
perl-core/*

Of course, as always, bugs may be present; what you get as Perl installation is called unstable or testing for a reason. We're looking forward to your reports on our bugzilla.

Perl in Gentoo: Upgrading pains, perl-cleaner, and EAPI=5

In a previous post, we've already looked at the structure of Perl ebuilds in Gentoo Linux. Now, let's see what happens in the case of a major Perl update.

Does this look familiar?

UPDATE THE PERL MODULES:
After updating dev-lang/perl you must reinstall
the installed perl modules.
Use: perl-cleaner --all

Then maybe you have updated your major Perl version recently, since this important message is printed by emerge afterwards. So, what is it about? In short, a certain disconnect between the "Perl way" of doing things and the rest of the world. Both have their merits, they just don't play very well with each other... and the result is that major Perl updates in Gentoo have traditionally also been a major pain. (This will become much better in the future, see below.)

Let's see where a perl package stores its files.

caipi ~ # equery files dev-perl/Email-Address
 * Searching for Email-Address in dev-perl ...
 * Contents of dev-perl/Email-Address-1.898.0:
/usr
/usr/lib
/usr/lib/perl5
/usr/lib/perl5/vendor_perl
/usr/lib/perl5/vendor_perl/5.16.3
/usr/lib/perl5/vendor_perl/5.16.3/Email
/usr/lib/perl5/vendor_perl/5.16.3/Email/Address.pm
/usr/share
/usr/share/doc
/usr/share/doc/Email-Address-1.898.0
/usr/share/doc/Email-Address-1.898.0/Changes.bz2
/usr/share/doc/Email-Address-1.898.0/README.bz2
caipi ~ #

Interesting- the installation path contains the Perl version! The reasons for upstream to do this are pretty much obvious, the application binary interface for compiled modules can change and it's necessary to keep the installed modules for different versions apart. Also, in theory you can keep different Perl versions installed in parallel. Nice idea, however if you have only one "system Perl" installation, and you exchange that for a newer version (say, 5.18.1 instead of 5.16.3), the result is that the new version won't find the installed packages anymore.

The results are rather annoying. Imagine you haven't updated your system for a while, one of the many packages to be updated is dev-lang/perl, and later maybe (just picking an example at random) gnome-base/gsettings-desktop-schemas. Perl is updated fine, but when portage arrives at building the gnome package, the build fails with something like

checking for perl >= 5.8.1... 5.18.2
checking for XML::Parser... configure: error: XML::Parser perl module is required for intltool

Right. Perl is updated, dev-perl/XML-Parser is still installed in the old path, and Perl doesn't find it. Bah.

Enter perl-cleaner, the traditional "solution". This small program checks for files in "outdated" Perl installation paths, finds out which packages they belong to, and makes portage rebuild the corresponding packages. During the rebuild, the installation is run by the updated Perl, which makes the files go into the new, now correct path.

This sounds like a good solution, but there are a lot of details and potential problems hidden. For once, most likely you'll run perl-cleaner after a failed emerge command, and some unrelated packages still need updates. Portage will try to figure out how to do this, but blockers and general weirdness may happen. Then, sometimes a package isn't needed with the new Perl version anymore, but perl-cleaner can't know that. Again the result may be a blocker. We've added the following instructions to the perl-cleaner output, which may help cleaning up the most frequent difficulties:

 * perl-cleaner is stopping here:
 * Fix the problem and start perl-cleaner again.
 *
 * If you encounter blockers involving virtuals and perl-core, here are
 * some things to try:
 *   Remove all perl-core packages from your world file
 *     emerge --deselect --ask $(qlist -IC 'perl-core/*')
 *   Update all the installed Perl virtuals
 *     emerge -uD1a $(qlist -IC 'virtual/perl-*')
 *   Afterwards re-run perl-cleaner

In the end, you may have to try several repeated emerge and perl-cleaner commands until you have an updated and consistent system again. So far, it always worked somehow with fiddling, but the situation was definitely not nice.

So what's the future? Well...

EAPI=5 brings the beautiful new feature of subslots and slot operator dependencies. In short, a package A may declare a subslot, and a package B that depends on A may declare "rebuild me if A changes subslot". This mechanism is now used to automate the Perl rebuilds directly from within emerge: dev-lang/perl declares a subslot corresponding to its major version, say "5.18", and every package that installs Perl modules needs to depend on it with the subslot-rebuild requested, e.g.

RDEPEND="dev-lang/perl:="

The good news about this is that portage now knows the dependency tree and can figure out the correct reinstallation order.

The bad news is, it can only work perfectly after all Perl packages have been converted to EAPI=5 and stabilized. perl-core is done, but with about 2100 ebuilds that use perl-module.eclass in the portage tree still quite some work remains. I've plotted the current EAPI distribution of ebuilds using perl-module.eclass in a pie chart for illustration... Maybe we're done when Perl 5.20 goes stable. Who knows. :)

Perl in Gentoo: dev-lang/perl, virtuals, and perl-core packages

We've got the stabilization of Perl 5.18 upcoming, so what better chance is there to explain a bit how the Perl-related ebuilds in Gentoo work...

First of all, there is dev-lang/perl. This contains the Perl core distribution, installing the binaries and all the Perl modules that are bundled with Perl itself.

Then, there is the perl-core category. It contains independent ebuilds for Perl modules that are also present in the core Perl distribution. Most Perl modules that are bundled with Perl are also in addition released as independent tarballs. If any of these packages is installed from perl-core, its files are placed such that the perl-core download overrides the bundled copy. This means you can also update part of the bundled Perl modules, e.g. in case of a bug, without updating Perl itself.

Next, there are a lot of virtuals "virtual/perl-..." in the virtual category of the portage tree. What are these good for? Well, imagine you want to depend on a specific version of a module that is usually bundled with Perl. For example, you need Module::CoreList at at least version 3.  This can either be provided by a new enough Perl (for example, now hardmasked Perl 5.20 contains Module::CoreList 3.10), or by a separate package from perl-core (where we have Module::CoreList 5.021001 as perl-core/Module-CoreList-5.21.1).
To make sure that everything works, you should never directly depend on a perl-core package, but always on the corresponding virtual (here virtual/perl-Module-CoreList; any perl-core package must have a corresponding virtual). Then both ways to fulfil the dependency are automatically taken into account. (Future repoman versions will warn if you directly depend on perl-core. Also you should never have anything perl-core in your world file!)

Last, we have lots of lots of modules in the dev-perl category. Most of them are from CPAN, and the only thing they have in common is that they have no copy inside core Perl.

I hope this clarifies things a bit. More Perl posts coming...