CPAN Testers is only made possible with the support of our sponsors.
For more information on sponsoring, please visit the I CPAN Testers website.

Upgrade Notice

The CPAN Testers Wiki site has been upgraded since you last accessed the site. Please press the F5 key or CTRL-R to refresh your browser cache to use the latest javascript and CSS files.

Testing & Reporting with CPAN + CPAN::Reporter

Testing with CPAN and CPAN::Reporter is easy once CPAN::Reporter is installed and configured. Just use CPAN as normal and whenever it runs a test, the results will be processed with CPAN::Reporter and sent to CPAN Testers. It really is this easy:

  perl -MCPAN -e 'install qw(Some::Module Other::Module ...)'

Testing & Reporting in bulk

While CPAN::Reporter is primarily intended to allow people to become CPAN-testers on a small scale as part of their normal perling, it can also be used as part of a dedicated testing setup to test in bulk. To do this most effectively, you will need to have a perl install that you can repeatedly delete and re-install, and to wrap the testing process in scripts.

This section describes howto set up the tools for testing modules in bulk using CPAN::Reporter. I assume that you will want to run everything in a subdirectory of your own home directory and not mess around with any system-wide version of perl, and that that subdirectory is called 'cpantesting' in all the examples below.

Install libraries that perl will need to link against

This step can be skipped, but experienced perlers will know that when you build perl it tries to build some modules and link them against C libraries. You will need to install those libraries and make their header files available. Probably the most important of these is the GDBM library. How you do this is specific to your operating system.

Building perl

Create the directory that you'll want to work in, download the source code for the version of perl that you want to test against, uncompress and build it. Where you would normally run the 'Configure' script, run it like this:

  sh Configure -Dprefix=$HOME/cpantesting/perl-X.X.X

where X.X.X is the perl version, eg 5.8.8. If you're happy for perl to auto-detect everything else about your environment, then add '-de' to that command line. When Configure has finished, continue with 'make', 'make test', and 'make install' as normal, noting that because you're installing in your own home directory you do *not* need to be root to 'make install'.

Configuring CPAN.pm

Once perl is built and installed, configure the CPAN.pm module. You will need to make sure you run the right version of perl:

  $HOME/cpantesting/perl-X.X.X/bin/perl -MCPAN -e shell

When it asks you whether to follow pre-requisites, tell it 'follow'. When it asks you what directory to use for its metadata, caches, and for building etc, tell it cpantesting/perl-X.X.X/.cpan. You will probably also want to point it at the central CPAN node instead of a local mirror. That is 'http://www.cpan.org/', which you can set by editing the 'urllist' parameter in cpantesting/perl-X.X.X/lib/X.X.X/CPAN/Config.pm.

At this point you should also install and LWP and Module::Build, as CPAN.pm will need them.

Install CPAN::Reporter

Install CPAN::Reporter and configure it.

At this point the perl part of the testing environment is complete. So, slim it down by deleting everything from cpantesting/perl-X.X.X/.cpan (but leave the directory itself there), and tar it all up:

  cd cpantesting
  rm -rf perl-X.X.X/.cpan/*
  tar czf perl-X.X.X.tar.gz perl-X.X.X

Now you can test that your testing environment works, by trying it out on one of your own modules as in the example at the top of this page (but make sure you run the right version of perl!)

Some handy shell scripts

I wrap all my testing up in shell scripts so that I can just change to my cpantesting directory and type './run-tests-5.8.8' (for example) to test all the new modules on the CPAN against perl 5.8.8. That shell script is responsible for getting the list of modules to test, and then invoking perl.

First I have a couple of scripts to turn CPAN::Reporter's email reporting on and off:

To turn email on:

  #!/bin/sh
  grep -v ^send_report= ~/.cpanreporter/config.ini>foo
  (
      cat foo;echo send_report=unknown:ask/yes fail:yes pass:yes na:no
  )>~/.cpanreporter/config.ini
  rm foo

and off:

  #!/bin/sh
  grep -v ^send_report= ~/.cpanreporter/config.ini>foo
  (
      cat foo;echo send_report=default:no
  )>~/.cpanreporter/config.ini
  rm foo

To get a list of modules to test (unfortunately this is very specific to my own setup, but maybe you'll find it useful) there is a 'list-modules' script:

  #!/bin/sh
  ssh my.mail.host grepmail -i -d \"before 'date -u +%Y-%m-%d'\" \
      -h ^From.*PAUSE mailboxes/cpan-testers | \
      grep -A 2 "^has entered CPAN as"|grep file:|sed 's/.*id\///'|grep .tar.gz$| \
      grep -v CPAN|grep -v Apache|grep -v POE|grep -v Catalyst|grep -v Tk-| \
      grep -v Bundle-|grep -v X/XX/XXXXXXXX|grep -v MARC-Charset|grep -v Solaris| \
      grep -v Win32|grep -v Mac-|grep -vi parrot|grep -v ALPO/Templates| \
      grep -v mod_perl|grep -v Angerwhale|grep -v kurila

where 'my.mail.host' is the machine on which my mail lives, 'mailboxes/cpan-testers' is the mailbox where notifications about new modules go from the PAUSE, and most of the rest is a list of modules I don't want to test. Some of those exceptions are obvious (I don't want to automatically test CPAN.pm and CPAN::Reporter themselves because if they're buggy horrible things might happen) and I also exclude some of the insanely big modules, or those that are hard to build. The 'X/XX/XXXXXXXX' entry is how you would stop automatically testing modules by whinging gits.

The 'grepmail' command there extracts all the emails from the mailbox with a Date before today. So obviously once it has grabbed that list, I need to clear those emails out of that mailbox. And how you get the relevant emails into that mailbox, but not everyone's CPAN-testers result, is a small matter of procmail best left to a different tutorial :-)

There is a 'X.X.X' script which actually runs the tests, including deleting any old build of perl that's still lieing around and untarring the pristine build of it that we saved earlier. We do this so that we're more likely to catch the common error where a module author has forgotten to declare one of his pre-requisites:

  #!/bin/sh
  cd ~/cpantesting/
  rm -rf perl-5.8.8
  tar xzf perl-5.8.8.tar.gz 
  echo $*>testing-5.8.8
  /home/david/cpantesting/perl-5.8.8/bin/perl -MCPAN -e "install qw($*)"
  rm -rf perl-5.8.8

and finally, a 'run-tests-X.X.X' script which pulls it all together:

  #!/bin/sh
  cd ~/cpantesting
  ./turn-email-on
  ./5.8.8 './list-modules'

This seems like a lot of scripts, but it breaks the functionality down to make it easy to run tests against several different perl builds at once and gives you a bit more flexibility. In particular, having the 'X.X.X' script seperate means that you can also test a single module directly on the command line.

Testing multiple builds of perl

You will notice how careful I have been to keep scattering the perl version number throughout my scripts and in the directory names. That's because it lets me do crazy things like this. I currently test three different versions of perl, on two operating systems. Those run in virtual machines on a machine that just lurks in a hidden corner of my home. To actually run my tests I open a terminal in GNU screen on another machine entirely and run this little script:

  #!/bin/sh
  screen -c c ssh -t vm1 "cd cpantesting;./run-tests-5.8.8;sh"
  screen -c c ssh -t vm1 "cd cpantesting;./run-tests-5.9.x;sh"
  screen -c c ssh -t vm2 "cd cpantesting;./run-tests-5.8.8;sh"
  ssh -t vm1 "cd cpantesting;./run-tests-5.6.2;sh"

which opens three extra virtual consoles, in each of them sshes to a virtual machine and starts the testing scripts, does the same in the window I ran that script in, and leaves me with a shell prompt after each testing session so I can do any necessary manual cleanup.

Bwahahahaha