Upgrade Notice

The CPAN Testers Wiki site has been upgraded since you last accessed the site. Please press the F5 key or CTRL-R to refresh your browser cache to use the latest javascript and CSS files.

CPAN Authors FAQ

Getting more information that the CPAN Testers page reports

How can I display a log of what happened?

Warning! Add File::Slurp::Tiny as one of your pre-reqs.

Create a showlog.t:

#!perl
use strict;
use Test::More tests => 1;
use File::Slurp::Tiny qw(read_file);
my $log = read_file("config.log");
ok( $log, "show config.log" ) and diag($log);

config.log is auto-generated, so don't worry about that. But, you may need to consider the order in which the test scripts are run.

Making distributions CPAN Testers friendly

"How can I indicate that my distribution only works on certain versions of Perl?"

A minimum Perl version should be specified with a "use VERSION" statement at the top of the Makefile.PL and/or Build.PL. Be sure to use the older style of version numbers. E.g.

# In Makefile.PL or Build.PL
use 5.006; # NOT 5.6.0

You can also specify a prerequisite of 'perl' in a Build.PL, but older versions of Module::Build don't check this early enough when running Build.PL so you can't rely on it to avoid fatal (and confusing) errors running Build.PL. You should still do this to ensure that your perl prerequisite is included in META.yml.

# In Build.PL as an argument to Module::Build->new()
requires => {
  'perl' => 5.008001,
}

The MakeMaker equivalent with ExtUtils::MakeMaker versions >= 6.47 is:

# In Makefile.PL as an argument to WriteMakefile()
MIN_PERL_VERSION => '5.008001',

Older MakeMaker versions did not have an equivalent -- adding perl to PREREQ_PM is not supported. MakeMaker 6.48 comes bundled with Perl 5.8.9, more recent versions are available on CPAN.

"How can I indicate that my distribution requires a version of Perl with threads?"

The current paradigm is to allow thread-based modules to be installed on non-threaded Perls. As such, you should NOT put any such checks in your Makefile.PL, but should allow the module to be installed regardless.

The rationale for this is to allow modules/packages/applications to be developed and run on non-threaded Perls. For instance, threads::shared is available, but essential does nothing if used on non-threaded Perls (or in non-threaded applications). Thread::Queue similarly provides object-oriented FIFO queues on both threaded and non-threaded Perls.

In your test suite, adding the following preamble to the top of any test files that 'use threads;' will keep them from generating failures:

BEGIN {
  use Config;
  if (! $Config{'useithreads'}) {
    print("1..0 # Skip: Perl not compiled with 'useithreads'\n");
    exit(0);
  }
}

Additionally, if possible, you should try to include tests that test your module's functionality in non-threaded applications (i.e., when 'use threads;'is not used).

You also don't need to put in special checks for the 'threads' module in your module's code. Just using 'use threads;' will cause the following error message to occur:

This Perl not built to support threads

"How can I indicate that my distribution only works on a particular operating system?"

While it isn't a very elegant solution, the recommend approach is to either die in the Makefile.PL or Build.PL (or BAIL_OUT in a test file) with one of the following messages:

  • No support for OS
  • OS unsupported

CPAN Testers tools will look for one of those phrases and will send an NA (Not Available) report for that platform.

Devel-AssertOS is an experimental tool that you can bundle with your distribution in an inc/ directory and use in your Makefile.PL or Build.PL to die with the appropriate message if not on a supported OS:

# in a Makefile.PL or Build.PL
use lib 'inc';
use Devel::AssertOS qw(Linux FreeBSD Cygwin);

"How can I stop getting FAIL reports for missing libraries or other non-Perl dependencies?"

If you have some special dependencies and don't want to get CPAN Testers reports if a dependency is not available, just exit from the Makefile.PL or Build.PL normally (with an exit code of 0) before the Makefile or Build file is created.

exit 0 unless some_dependency_is_met();

Devel-CheckLib is an experimental tool that you can bundle with your distribution in an inc/ directory and use in your Makefile.PL or Build.PL to issue a warning and exit if a library is not available:

# in a Makefile.PL or Build.PL
use lib 'inc';
use Devel::CheckLib;
check_lib_or_exit( lib => 'jpeg' );

See the documentation for instructions on specifying multiple libraries or specifying additional library paths to search.

Never, ever, set the MakeMaker parameter PREREQ_FATAL to true, it is a flawed invention. Not only has it the effect that no Makefile is being written thus preventing installers to actually try to install prerequisites. It also exits with an error code, so that cpantesters would rate it as a FAIL.

(vkon: I think this is not fair. Tester is not even getting Makefile and already sends FAIL. This feature is largely un-useful, and this is a bug in tester's smoking scenario, IMO.)

(dcantrell: some really old distributions have no Makefile.PL, so CPAN.pm makes one up. This is expected behaviour. Therefore having no Makefile is not a show-stopper. Generating failure reports for failures at Makefile.PL-time is desirable, as it will find bugs such as Makefile.PL requiring modules that aren't available)

"How should I invoke perl from my test scripts?"

If you don't want to add any extra dependencies:

use Config;
my $path_to_perl = $Config{perlpath};
# now exec/system/qx/whatever using that variable

or use the Probe-Perl module. Perl might not be in the path, or your user might have several different version of perl installed, so using either of these methods instead of just using what you hope is the right version of perl will ensure the test is run with the appropriate perl binary.

Probe::Perl is a more correct choice, as it gets things right under some edge-cases. But it does add another dependency to your code.

This page used to recommend use of $^X. That's still better than just system("perl blahblahblah"), but situations where $^X may be wrong include:

  • perl was executed with a relative path and the script has chdir()ed;
  • because $^X originates in C's argv[0] (in the main() function) it is possible for the calling program to exec() in such a way that argv[0] isn't the path to the interpreter;
  • HP/UX can do weird stuff in scripts that use #!
  • VMS

"Why am I being told 'Can't locate Devel/Autoflush.pm in @INC'?"

CPAN::Reporter sets PERL5OPT="-MDevel::Autoflush" to ensure Makefile.PL or Build.PL prompts are always visible. This error will show up when your tests or code execute 'perl' instead of $^X and the default perl in $ENV{PATH} does not have Devel::Autoflush installed. See the prior question on how to invoke perl safely.

"How can I get more detail about a failing test script?"

If the Test::Harness output in the CPAN Testers report is insufficient, you could email the tester and ask for a more detailed output to help diagnose your test failure. (See answers to "How do I contact a tester?" below.)

About CPAN Testers reports

"Why am I getting these reports?"

The short answer is that you uploaded a distribution, via PAUSE, to the CPAN. The longer answer is that the CPAN Testers will test your distribution on their system, once PAUSE announces it has been indexed. They will run your test suite and submit reports on the outcome. If you have received one of these reports, it is likely that the test suite doesn't exist or cannot be found (UNKNOWN) or one or more of the tests in the test suite failed (FAIL).

"Can't CPAN Testers read? I have detailed the format I require for bug reports!"

Please do not get irate with CPAN Testers. They are there to try and help you. The reports they send are often automated, without any human intervention, and are simply the output of your own test suite. If you want the testers to produce something special, it is suggested you include test scripts that include all the necessary diagnostic information you will need to understand why your distribution failed.

If you ask nicely, I'm sure the tester who tested your distribution will try and help you solve the problem. Rejecting reports simply because they don't meet your expectations of what a bug report should look like does you no favours. Users often read the CPAN Testers website to see whether a distribution fails on a particular platform or perl combination. Ignoring that will only mean your distribution gets less usage than it might otherwise gain.

"Why are you testing (and failing) my Tk-ish module without an X server?"

Until very recently, Tk wouldn't build without a display. As a result, the testing software would look at the test failures for your module and think "ah well, one of his pre-requisites failed to build, so it's not his fault" and throw the report away. The most recent versions of Tk, however, *will* build without a display - it just skips all the tests. So the testing software sees that it passed, and thinks there must be something wrong with your module. You should check for a working Tk (and hence X11) environment by checking if MainWindow->new() throws an exception:

  my $mw = eval { MainWindow->new };
  if (!$mw) { ... skip tests ... }

"Why am I getting a FAIL/UNKNOWN/NA report when prerequisites are not met?"

Some early versions of CPAN Testers tools had bugs that did not properly catch when prerequisites were specified but not met. Please just ignore these reports.

"How can I stop getting these reports?"

At the moment, the suggestion is to use a filtering mechanism in your mail client of choice. However, this may mean you miss valid reports from users, so you might want to filter reports into a dedicated mail folder and periodically check it, deleting anything you are not interested in. At the moment there is no way for you to tell testers that you don't wish to receive reports, although a mechanism is currently being considered, that would allow the author to include an appropriate entry in the META.yml file, which the testing mechanism could then respect.

"How can I get more of these reports?"

Marcel Grünauer has written a script to fetch arbitrary reports: App::sync_cpantesters

"Can I tell if my module is being tested by an automated client (a.k.a. 'smoker')?"

Automated smoke testers should set $ENV{AUTOMATED_TESTING} to a true value. This allows authors to skip certain tests (or include certain tests) when the results are not being monitored by a human being.

One could even go so far as to halt building and testing a distribution under automated testing by exiting with zero at the top of the Makefile.PL or Build.PL file:

exit 0 if $ENV{AUTOMATED_TESTING};

"How do I contact a tester?"

Most testers will Cc you on FAIL and UNKNOWN test results, so you can just reply to the email. But if you're looking at test results through, eg, a web page, then this tool will convert a test report number into the tester's email address.

"Should I write OS-independent code?"

Yes, more or less always.

If you think you won't need it, I encourage you to discard that thought and reconsider.

There are cases, e.g. accessing the MS Windows registry, where you'd say the decision is obvious, but in general, I recommend you make the effort to write OS-independent code for 3 reasons:

  • You are training yourself as to how to write high(er) quality code.
  • You're considering more carefully how you're actually using directories and files.
  • You are providing a guideline for users and readers of your code.

"Are there any traps in writing OS-independent code?"

Yes, many. Next I discuss some related to directory and file management.

"What are the problems with temporary directories?"

Firstly, I'll assume you want to write to this directory, or a file within it, even if you only want to write zero bytes in order to use it as a present/absent flag.

You can't assume the OS is Linux and you can't assume the temp directory is '/tmp'.

Also, you need to ask: Do I want /a/ temp directory or /the/ temp directory?

File::Spec will find /the/ temp directory (as long as it's writable), or return the current directory.

my($temp_dir) = File::Spec -> tmpdir;

Is that what you want? If so, problem solved - perhaps. But keep reading.

However, let's look for /a/ temp directory:

my($temp_dir) = File::Temp -> newdir; # Bad code! Do not use!

Note: This is an alternative to File::Temp's function tempdir(). Don't use that one either!

But what's the problem, I head you cry?

You've fallen for the trap of assuming your end user is not running FreeBSD. You don't know that!

And no, it's not true that FreeBSD's nick is The World of Pain :-), but sometimes it feels like it.

Don't get me wrong - I thank CPAN testers for using FreeBSD, since it's taught me to be much more careful about directory and file handling.

The problem is that to deal with all OSes, including FreeBSD, you need this:

my($temp_dir) = File::Temp::newdir('temp.XXXX', CLEANUP => 1, EXLOCK => 0, TMPDIR => 1);

Here, the EXLOCK is the most important flag, but we might as well be thorough and use all those options. Consult the docs for the details. I'm not spoonfeeding you everything.

And yes, you really do need that EXLOCK option.

"OK. So what do I need to know about temp files?"

Not so fast! You haven't checked that your temp directory is writable.

Some CPAN testers run with the temp directory set to read-only. So, check and double-check.

And by that I mean, write the code that checks /after/ deciding what you're going to do if you determine that you can't write to $temp_dir (assuming still that you want to).

Most likely, you'll call BAIL_OUT(), the escape hatch built into Test::More.

Errr, you are using Test::More, right? Phew.

"OK. Now what do I need to know about temp files?"

Fundamentally, make the directory separator OS-independent with:

my($path) = File::Spec -> catfile(@directories, $filename);

Here, @directories is probably just $temp_dir from above, but may not be.

Of course, there are packages build on top of File::Spec, such as Path::Class. However I've found that in testing code I virtually never need an object to represent a directory or file, so I rarely use Path::Class. But it's good to know it's available.

"Are there any other directory/file matters I need to be aware of?"

Yes indeed. Let's consider all the problems to do with a file with a fixed name.

An example of this would be session management with a module such as Data::Session. In particular, if you opt for Data::Session::ID::AutoIncrement to provide a sequence of IDs, it's default file name is:

File::Spec -> catdir(File::Spec -> tmpdir, 'data.session.id').

Look familiar? Good. But here we're worried about the last component, 'data.session.id'.

What could go wrong? Well, it's assuming that:

  • File::Spec -> tmpdir always returns the same directory.
  • The owner of the file is always the same user.

The first point's pretty safe (I hope), so let's concentrate on the second point.

Since the CGI script (assuming that's the environment we're working it) is probably always run as the same user, we assume we should be able to access this file.

But is that true? Hint: The correct answer is: No, it's not always true.

For instance, the main web server may be Plack or Apache, run by a special user called, e.g. nobody or (under Debian) www-data. Or perhaps you're running Engine X (nginx), and the user is nginx.

The problem arises when you're testing and running code as yourself, or in the process of switching web servers, or proxying to a different web server rather than just another copy of the same web server running on a different port.

Of course I expect you'll be on a different machine, but the other possibilities are real.

"So how do I handle such a situation?"

See also the next question.

Provide the session management code with a file name, and here it'd be:

Data::Session::ID::AutoIncrement -> new(id_file => '...');

which is passed in, when you create a new session object, via:

Data::Session -> new(id_file => '...');

This is the best solution - or arrange for the web servers to all run as the same user.

There may be other solutions. But that's not the point.

The point is to be aware of what's happening behind the scenes, and to code defensively. Hence your motto: Check and double-check.

"What else can I try?"

Use File::HomeDir. This may, repeat may, solve your problem.

Since by now we're using a per-user directory, we've circumvented the problem of multiple users competing for a file with a fixed name.

File::HomeDir has a method called my_data() which looks like a candidate to help, and my_dist_data(), which looks like an even better one. So let's settle on my_dist_data().

Now we have these questions to deal with:

  • Does the current user have a home directory?

No, not always. CPAN testers, e.g., sometimes run with users without home directories. So check.

  • If I use the create option to my_dist_data(), does it actually find or create the directory?

Not necessarily. So check.

  • If so, is the directory writable?

Not necessarily. So check and double-check.

Each of these needs to be checked, and if all turns out well, use that directory.

If not, BAIL_OUT().

"What if unexistent domain resolves on testers machine?"

Some providers are buggy and do not want to change it

sub inexistent_domain_do_not_resolve {
  use Socket;
  return 1 unless inet_aton('some.non.existent.host.');
  return 0;
}
#returns 1 if works correctly

if (inexistent_domain_do_not_resolve()) { }