Building DWIM Perl for Linux

The idea behind the DWIM Perl distribution is to provide an easy way to start using Perl to build applications.

Therefore it should contain every CPAN module you'd need to build any decent Perl application.

There is already a fairly good DWIM Perl for Windows, built on top of Strawberry Perl, but the experimental version of DWIM Perl for Linux could be a lot better.

Let's see how can we build a much better distribution of Perl for Linux?

Dependencies

There are going to be two major dependency issues for this distribution.

  1. What run-time external dependencies will this package have?
  2. What additional dependencies will it have for installing more CPAN modules?

The Windows version of DWIM Perl (which is based on Strawberry Perl), comes with a C-compiler and the C libraries for some of the most central dependencies. For example it comes libssl, libz and libexpat. On the other hand I am quite sure it still assumes certain Windows-specific libraries (DLLs) to be available on the system. This enables the CPAN client to be useful out-of-the box and allows it to work on any "recent" version of Microsoft Windows.

For the long-run this might be a good objective for the Linux version as well, but for now I'd be happy if we could make perl and the CPAN modules we supply to work. At first we don't need to have a working CPAN client.

The Plan

I don't fully understand yet what are the thing we need to do in order to achieve the grand objective, so let's have a rough plan that we can adjust as we go.

I am certainly looking for your ideas on this.

  1. Compile a relocatable Perl.
  2. Add some CPAN modules that don't have external dependencies.
  3. Add a CPAN module with some external dependency.
    (Which one should be the first?)
    Repeat this step several times.
  4. Enable the CPAN client work out of the box assuming certain things are available (e.g. C compiler, make, etc)
  5. Include the C compiler and all the other tools the CPAN client requires. (Is this really necessary?)

Basically after every such step we can have a new release. As some of the steps can be broken down to lots of smaller steps - the addition of each CPAN module or each external dependency can be a separate sub-step - we will probably have multiple releases during the process.

A couple of other issues we will have to deal with:

  • Make the building process reproducible.
  • Maintain a repository of all the dependencies. (perl, CPAN modules, external dependencies)
  • Create an easy path to upgrade various parts of the distribution.
  • Create an easy path to create derivatives of the distribution. (Including more thing and maybe also including less things)
  • Create a list of CPAN dependencies for each module we plan to install.
  • Provide a listing of software licenses that were used in the various parts of the distribution.
  • Send reports to the CPAN Testers, possibly bug reports to the Perl 5 Porters, the authors of CPAN modules and maybe even to external dependencies.

Compile a relocatable Perl

The first step is to compile perl, make it relocatable and create the skeleton of the distribution.

The Directory Layout

Within the root directory of the package we distribute we will have a subdirectory for perl and another subdirectory for C. This is the same layout used on Windows.

In addition we are going to have a t directory for test. At first I am going to include a few small sanity tests. That will let the end-user test our distribution after installation. Later we might include some or all of the tests that come with perl and the included CPAN modules to make it possible to test the whole system even after installation.

The build environment

If I understand correctly perl, as basically everything else on Linux, depends on glibc. In turn glibc is backward compatible and its run-time is available on every Linux system. That means if we compile our perl on a relatively old version of Linux, with a relatively old version of glibc, then it will run on all the newer version of Linux and glibc.

We will use CentOS 5.8. The 5.x series of CentOS was first released in April 2007.

Install Linux

First download and set up Linux on a Virtual Box. When installing CentOS deselect all the checkboxes (no Gnome GUI and not even "server").

It created only a root user so better create another user.

adduser foobar passwd foobar

then log-in as that user

Directory and getting perl

Create a directory called dwimperl as the root directory of the distribution.

mkdir $HOME/dwimperl

Download the source of perl Specifically we'll start with 5.16.1 which is the most recent version at the time of this writing. Newer versions of DWIM Perl will use newer versions of perl.

Fetch the source code of Perl and unzip the tarball:

wget http://www.cpan.org/src/5.0/perl-5.16.1.tar.gz tar xzf perl-5.16.1.tar.gz cd perl-5.16.1

Start building perl:

sh Configure -Duserelocatableinc -Dprefix=$HOME/dwimperl/perl -des

this will immediately tell us, some stuff is missing from the computer. We need to install a C compiler.

Login as root and type:

yum install gcc

Then we can re-run the previous command to configure perl. Once that's successful, we can run make, make test, and make install.

make make test make install

Add a perl script for testing the installation and another shell script to run the tests with the correct perl. (The original copies of these files are in the Github repository of the project.)

t/selftest.sh t/01-perl.t

Running t/selftest.sh from anywhere in the system should work.

Even if we rename the root directory to dwimperl-5.16.1.1 (or anything else for that matter).

Create the distribution by typing

tar czf dwimperl-5.16.1.1-64b.tar.gz dwimperl-5.16.1.1/

Then I set up another Virtual Box instance if CentOS. Created a different user but has not installed any extra files. Copied the tarball, unzipped it and ran the selftest.sh there too.

It worked.

Then I copied the tarball to another machine with Ubuntu 12.4 on it. Repeated the ceremony.

It worked there too!

Distributed Perl scripts

In the perl/bin/ directory there are number of Perl scripts. For example perldoc and cpan. They all start with an sh-bang pointing to the directory where perl was originally installed when it was built. This is not convenient as it means the end user won't be able to run perldoc.

The files should either have #!/usr/bin/env perl as their sh-bang line hoping that the user puts this perl at the beginning of the PATH environment variable, or they should be updated after relocation to point to the new location of perl.

I am not yet sure which is the better solution and if there is already a tool to update all the scripts?

The Results

Basically this covers the first step in The Plan. Let's distribute it so you can download and check if it works on your Linux?

Download

You can download this first version from here: dwimperl-5.16.1.1-64b.tar.gz and you can "install" it by the following commands:

wget https://dwimperl.googlecode.com/files/dwimperl-5.16.1.1-64b.tar.gz tar xzf dwimperl-5.16.1.1-64b.tar.gz

To run the self-test type:

./dwimperl-5.16.1.1/t/selftest.sh

To see the version of perl and the details of this compilation type

./dwimperl-5.16.1.1/perl/bin/perl -V

If you'd like to turn this perl to be your "default" perl then edit .bashrc, include the following:

export PATH=/the/path/to/dwimperl-5.16.1.1/perl/bin/:$PATH

and reload .bashrc.

Later versions of DWIM Perl for Linux will be published on the DWIM Perl web site.

How can you help?

I'd be happy to get your opinion on the questions I asked above and I'd be happy if you downloaded this first package, tried it, and let me know what works and what does not?