CPAN Testers is only made possible with the support of our sponsors.
For more information on sponsoring, please visit the I CPAN Testers website.

Upgrade Notice

The CPAN Testers Wiki site has been upgraded since you last accessed the site. Please press the F5 key or CTRL-R to refresh your browser cache to use the latest javascript and CSS files.

Why is testing important?

... or not. What if, instead of "All tests successful", you see a series of cryptic messages:

% sudo make test
/usr/bin/perl Makefile.PL --config=-target,skiptest
--installdeps=Term::Size,0.01
*** Installing dependencies...
*** Installing Term::Size...
Can't bless non-reference value at lib/CPANPLUS/Internals/Module.pm
line 239.
Compilation failed in require at Makefile.PL line 20.
make: *** [installdeps] Error 255

An experienced Perl user will probably take the following steps:

  • Look at lib/CPANPLUS/Internals/Module.pm and see if it's a trivial mistake. Unfortunately, it is not.
  • Search README for the project's mailing list address. Discovering that it's hosted in SourceForge, check Geocrawler (or any search engine) for existing bug reports and discussions. Unfortunately, there are none.
  • Copy-and-paste the error message into a bug-reporting email to the mailing list, including your operating system name and version, perl version, and wait for a fix.

Apparently, the above is exactly what Jorrit Waalboer (the bug's original reporter) did when he ran into this trouble.

However, there are a number of problems with the procedure above. It might be thwarted at every step:

  • What if Jorrit had been less experienced, and unaware of this bug-reporting procedure?
  • What if he decided to kluge around it by commenting out line 239, instead of taking the laborious copy-and-paste path to report it?
  • CPANPLUS has a mailing list; other modules might not. Worse, they may not even have a working contact address -- maybe the author is on vacation. (Note that this problem is somewhat alleviated by the CPAN Bug Tracking service at http://rt.cpan.org/)
  • He might not have the sense to provide adequate debug-related data, and simply wrote "CPANPLUS didn't work! It can't install!#!@#@#!" in red-colored blinking HTML email, wasting bandwidth and causing developers to ignore his mail completely.
  • Even after a fix is made available, and posted as a patch on the mailing list, other people about to install CPANPLUS 0.03 still have no way to find out about it beforehand. Thus, when the same problem occurs again, they may forget to search in the archives, resulting in duplicate bug reports, or (worse) more dissatisfied users.
  • Actually, we had not mentioned the very likely scenario: what if Jorrit didn't run make test at all? If so, the developers will have no way to know that this bug has ever happened.
  • Finally, while CPANPLUS (like most CPAN modules) includes a regression test suite, some modules may omit their tests altogether. Users may not feel comfortable to mail the author to request a test script to accompany the distribution; consequently, bugs will surface in harder-to-detect ways, and may never get fixed.

As you can see, both authors and users can certainly benefit a lot from an improved test reporting system -- one that simplifies the process of reporting a failed (or successful) installation, and lets users check for existing reports before downloading a module.

Such a system already exists; it's called CPAN Testers