Two or Three Things I Learned about Maintaining CPAN Modules

James E Keenan

Yet Another Perl Conference::North America::2006

Tuesday, June 27, 2006, 11:15 am


Good morning. Today I'd like to discuss the challenges that I and other Perl hackers have faced when taking over maintenance of CPAN distributions from their authors or previous maintainers. I'm going to start with my experiences since YAPC last year in assuming maintenance of ExtUtils::ModuleMaker.

Who here hasn't used modulemaker?

[Get volunteer from audience and have him/her sit at keyboard.]

Please come up here.

To create a Perl extension that is ready for uploading to CPAN, all you have to do is type this:

    $ modulemaker

... and then answer the questions!

[Prompt audience for a module name, abstract, author info, compact, etc. Have volunteer enter info and then generate the distribution.]

Okay, now type this:

    $ perl Makefile.PL; make; make test; make dist

This distribution is ready to be uploaded to CPAN!

ExtUtils::ModuleMaker, Geoff Avery and Me

What you've just seen was the brainchild of Philadelphia-area Perl hacker Geoff Avery. Geoff created ExtUtils::ModuleMaker and its command-line utility, modulemaker, five years ago. Since July of last year I have been the maintainer of ModuleMaker.

I heard Geoff speak about ModuleMaker at YAPC::NA::2003 in Boca Raton, but I didn't immediately begin to use it. A year later, however, I was planning a trip to New Orleans to visit a musician friend. Before I did so, I joined the New mailing list and checked out their wiki. I noticed they had a page on the wiki called ''Call for Speakers'' ( and I figured: Why not?

It turned out that I was the very first out-of-town guest speaker they had ever had. We decided that I should speak on building a CPAN-ready Perl distribution. I talked about the usual stuff. You know ... writing your documentation first, then writing your tests, finally writing your code. (You guys know all about that, right?)

Then I faced the question: How should I show the New Orleanians how to create the structure for a CPAN-ready distribution? Did I really want them to say:

    $ h2xs -Axn Module::Name

I myself had used h2xs exactly once. Why should I use such an obscurely-named utility and have to type three options just to get that utility's most typical use?

Then I remembered ModuleMaker. I decided to check it out. Once I tried out the interactive mode, I was amazed at how well it reflected one of the three great Perl virtues. I wrote Geoff Avery, ''modulemaker is truly f*$#!@g Lazy!''

Now, I also told Geoff, ''I do have some criticisms.'' As you might guess, those criticisms were the itch that I had to scratch. And scratch. And scratch. By the time of my New Orleans talk I knew I wanted to do a lot of work on ModuleMaker.

But about that time Geoff became preoccupied with mundane matters such as getting married; for many months he didn't respond to my e-mails. By January 2005 I had become involved in Perl Seminar NY's Phalanx project work on Text-Template and HTML-Template, about which Marc Prewitt and I spoke at YAPC last year. I had to set my work on ModuleMaker aside.

Fast forward to last year's YAPC in Toronto. Geoff Avery and I got to talking, and he spoke the words I had secretly been waiting to hear: ''Jim, why don't you take over ModuleMaker's maintenance?'' Deal done. I thought I'd be able to do what I wanted to do in about a month's time. Little did I know that I would be spending every evening and weekend for the rest of the summer working on improvements in ModuleMaker.

The Maintenance Process: First, Do No Harm

Use Version Control

If you heard the talk Marc and I gave on Phalanx at YAPC last year (, you'll recall that I said that I now use Subversion for version control on all my projects. So the first thing I did was to check Geoff's last version of ModuleMaker, 0.32, into my repository. I then pulled in all the revisions I had done the previous November, so I could get an immediate svn diff on those changes. I used Subversion religiously; the evidence is here in the printout of my svn log for this project.

[ Toss out svn log to audience. ]

At this point I knew that before I added new functionality to ExtUtils::ModuleMaker, I would have to clean up the internals without damaging the currently existing functionality. How did I do that? By writing documentation, of course.

Revise Documentation First

More precisely, by editing the existing documentation to make sure that it constituted an accurate specification of the current, non-deprecated functionality.

When you take over maintenance of existing code, it's your job to make sure that all functionality is documented and, I would add, that the documentation covers only functional code. The maintainer should either document previously undocumented code or explicitly deprecate it.

Upon inspection I found that some of ExtUtils::ModuleMaker's then current functionality was not documented, or only sketchily so. The modulemaker command-line utility was the best thing about the module, but its documentation was lacking. I immediately set out to improve it. I also discovered a number of subroutines in lib/ExtUtils/ that were not documented (or tested) and which appeared to constitute code left over from earlier in ModuleMaker's development. That code got scrapped. There were other subroutines that Geoff had documented but explicitly deprecated. I had no intention of maintaining deprecated functionality, so I scrapped both the code and the documentation.

The result: The documentation found in v0.33 -- the first uploaded to CPAN under my maintenance -- reflected the current functionality much more closely.

Boost Test Coverage

What's true of documentation is also true of testing: first, do no harm. If you as the maintainer find code that is untested, you either have to write tests for it, explicitly warn the user of untested features, or scrap it.

So the next thing I did was to write more tests and increase the test suite's coverage of ModuleMaker's code. Those of you who heard Marc and me speak at YAPC will recall how we measured our progress during Phalanx by using CPAN module Devel::Cover to measure our code coverage. Running make test and Devel::Cover on v0.32 gave these results:

    v0.32:              test files:   4         tests:    37 
    File                           stmt   bran   cond    sub
    ---------------------------- ------ ------ ------ ------
    ...b/ExtUtils/   80.7   48.4   44.4   82.5

Running Devel::Cover on v0.33 yielded this:

    v0.33:              test files:   7         tests:   105 
    File                           stmt   bran   cond    sub
    ---------------------------- ------ ------ ------ ------
    ...b/ExtUtils/   95.1   64.1   47.6   97.1

You can see that I improved the coverage substantially. To jump ahead a minute to v0.47 (the current CPAN version):

    v0.47:              test files:  96         tests:  2198 
    File                           stmt   bran   cond    sub
    ---------------------------- ------ ------ ------ ------
    ...b/ExtUtils/   95.8   79.7  100.0  100.0
    ...s/ModuleMaker/  100.0    n/a    n/a  100.0
    ...duleMaker/  100.0   95.0   77.8  100.0
    ...duleMaker/   94.8   78.0  100.0  100.0

We'll see that adding new functionality required splitting the code up into multiple packages instead of the original one. However, most of the improvements in code coverage were made between versions 0.32 and 0.33. That's because between these two versions I was writing tests for previously untested code rather than writing tests for new functionality.

Having corrected the bugs I spotted a year previously, improved the documentation and written tests to improve the coverage, I uploaded v0.33 to CPAN on July 8, 2005.

The Five-Fold Path

From this point forward my work on ExtUtils::ModuleMaker reflected five impulses:

Let's look at each in turn.

Cleaning Up the Author's Code

When I take over maintenance of code originally written by someone else, I never really feel comfortable with it until I've imposed my own version of ''best practices'' on it. This imposition doesn't change the code one iota (at least, it shouldn't), and it won't change the code if your test suite thoroughly covers the code and alerts you to breakage. Imposing your best practices does, however, make it easier for you to manipulate the code. So over the course of several versions I found myself:

In certain cases I adopted Geoff's style of doing things in situations where I would have started out differently. For example, in documenting subroutines Geoff used a tabular style rather than the paragraph style I previously would have used:

    Usage     : $self->default_values()
    Purpose   : Defaults for new().
    Returns   : A hash of defaults as the basis for new().
    Argument  : n/a
    Throws    : n/a
    Comments  :

Adding New Functionality

Personal Defaults File

When I took over maintenance of ModuleMaker last July, I did not immediately have plans to add new functionality. In mid-July, however, I met a Perl hacker who argued that if you were repeatedly using the modulemaker command-line utility, you would not want to constantly re-type your name, e-mail address or web site. You'd want to save this information -- along with any preferences as to the content of files created -- in a personal defaults file stored on disk.

In July I also became aware that in 2004 -- while Geoff Avery's attention had shifted away from ModuleMaker and I was just learning about it -- Andy Lester had uploaded a distribution called Module::Starter which, like ExtUtils::ModuleMaker, creates the directories and files needed for a CPAN-ready Perl extension. Module::Starter also has a command-line utility, module-starter, analogous to modulemaker -- though module-starter lacks the interactive mode which makes modulemaker so incredibly easy to use.

And, as befits the public relations director of the Perl Foundation, Andy did a good job at promoting Module::Starter. It's specifically mentioned in both Perl Best Practices and in Perl Testing: A Developer's Handbook by Ian Langworth and chromatic ( Damian Conway, in fact, wrote a CPAN module called Module::Starter::PBP which creates the framework for a CPAN-ready Perl distribution which follows the practices Damian recommends in his book. When you first use Module::Starter, it prompts you to store a limited amount of personal information in a defaults file.

I figured that if Module::Starter could duplicate ExtUtils::ModuleMaker's overall functionality, I could borrow one of Module::Starter's original concepts and include it in an improved ExtUtils::ModuleMaker.

To implement this, I needed a publicly available method, make_selections_defaults(), that could be called within a Perl program once an ExtUtils::ModuleMaker object had been constructed. In addition, I needed to offer a command-line option for the modulemaker utility that would trigger make_selections_defaults(), and I needed to modify the modulemaker interactive mode for that same purpose.

I manage to accomplish all that -- but testing it, as we shall see in a moment -- proved to be hairy.

New Attributes

While make_selections_defaults() was the principal new functionality added, I also added a number of attributes which permit more fine-grained control over the content of the lib/*.pm files and the names and content of the t/*.t files. These will mainly be of interest to users interested in subclassing ExtUtils::ModuleMaker.

To demonstrate what could be done with these attributes, I subclassed ExtUtils::ModuleMaker to create another distribution now up on CPAN: ExtUtils::ModuleMaker::PBP. This distribution uses ModuleMaker to create a framework in the Conway-approved style identical to Module::Starter::PBP. (Damian gave his blessing to my borrowing his boilerplate copy for my implementation of PBP.)

Restructuring the Distribution

Setting aside two packages which held code for licenses and one package intended to be an example of subclassing, v0.32 of ExtUtils::ModuleMaker consisted only of lib/ExtUtils/ and the modulemaker utility. Over two-and-a-half months of development, I moved much of the code in those files to other packages, largely for the purpose of making subclassing easier. I moved subroutines to particular packages largely on the basis of where they were called in ModuleMaker's public methods.

Default values, for instance, went to a module called Subroutines called within the constructor were moved to a package called Subroutines which supplied boilerplate copy for a distribution's README, Makefile.PL and POD were moved to a package called Most of the code for the modulemaker utility was refactored out of scripts/modulemaker and into a package called The options handling code was then transferred to a package called

As a result of these refactorings, if you want to subclass ModuleMaker, you can zero in easily on the subroutines you need to change. For example, when I was working on ExtUtils::ModuleMaker::PBP, I wanted a command-line utility that would offer the same functionality that the modulemaker utility offered to the parent class. This was now easy:

    use ExtUtils::ModuleMaker::PBP::Interactive;
    use ExtUtils::ModuleMaker::Opts;
    my $opt = ExtUtils::ModuleMaker::Opts->new(
    my $mod = ExtUtils::ModuleMaker::PBP::Interactive->new(
    $mod->run_interactive() if $mod->{INTERACTIVE};

Only six statements, by my count!

Overhauling the Test Suite

Most of the time I spent on ExtUtils::ModuleMaker between July and September of this year was spent not on improving the code but on overhauling the test suite. As noted above, I went from 4 test files holding 37 tests to 96 test files holding 2198 tests. Why such a big increase?

As a veteran of the Phalanx project, I have become much more rigorous about testing. In particular, if I'm testing code that creates files and directories and if I'm running those tests on someone else's machine, I don't want those files to be left around when the testing is done. I want them to be created in temporary directories which are automatically removed once testing is complete.


For that purpose, I use the File::Temp module which has shipped with the Perl core since 5.6. It works quite well, though not as well on Windows as on Unix-like systems. My use of File::Temp, however, was somewhat restricted by the fact that I wanted to keep the working code in ExtUtils::ModuleMaker as pre-5.6-compatible as possible. I don't, for example, use our or use warnings; in the module. If I made File::Temp an absolute prerequisite for running the test suite, the suite would fail horribly on older Perls -- and there are many good Perl programmers who are required to work in pre-5.6 environments.

There are a number of different ways to handle this problem. The one I chose was to put all the tests that require File::Temp inside SKIP blocks. I made locating Perl 5.6 or later the condition for entering those blocks. Programmers using older Perls can get the benefit of ModuleMaker's improved functionality. But they'll have to take the accuracy of that functionality largely on faith, as most tests are skipped on older Perls.

Testing the Personal Defaults File

As mentioned earlier, the most significant improvement I made to ModuleMaker's functionality was to permit you to save a file on disk with your personal selections for attribute values. Whether you initiate those selections in a script, from the modulemaker command-line or from modulemaker's interactive mode, you ultimately save the personal defaults file via the make_selections_defaults() method. Since this method, when used in production, saves a file on the user's disk outside of the normal Perl library directories and usually in the user's home directory, to test the method I had to write tests that asked:

Ultimately, I felt I demonstrated that I could do all that. But I had to test each step along the way to ensure that the testing process itself was sound and did not leave any footprints on the user's disk after testing was complete. This ballooned the size of each test file, but I was able to refactor a lot of this testing code out into subroutines and ultimately into two separate packages included with the distribution. I then decided that having smaller individual test files was a good thing. Once I had broken up big test files into many smaller ones, I took a suggestion from Langworth and chromatic and created several subdirectories under t/, each of which corresponded to an individual t/*.t file in an earlier version of ExtUtils::ModuleMaker.

Responding to Reports from Testers and Perl Hackers

Over the course of the two-and-a-half months I was working most heavily on ModuleMaker I received considerable assistance in the form of feedback from beta testers and testers on the perl.cpan.testers list (a.k.a.

The Simple Stuff

Some of this feedback consisted of reports of easily correctible errors. For example, two of my test files test whether a user who prefers Module::Build over ExtUtils::MakeMaker can use ModuleMaker to create Build.PL. At first I failed to account for the possibility that a user might not have Module::Build installed at all. (As of 5.8, it's not Perl core.) So the tests using Module::Build, like those mentioned earlier using File::Temp, had to go in a SKIP block.

I've Already Been Subclassed

One day last July, purely on a whim, I used Randy Kobes' search engine to look up ExtUtils::ModuleMaker on CPAN ( Much to my surprise, I found that someone whom I had never heard of had written a module which subclassed Geoff Avery's version of ExtUtils::ModuleMaker. That someone was David Golden, who spoke here yesterday on inside-out objects. That subclass was ExtUtils::ModuleMaker::TT, a module to create Template Toolkit-style templates. Suddenly I had to worry about the possibility that the revisions I was going to make would break someone else's code.

Naturally, I sought the counsel of my fellow Perl hackers as to how to proceed. Fortunately, there was a New York Perlmongers meeting scheduled at the Peculier Pub for that very evening. I was the first to arrive. As I was sitting there nursing a Dogfish I.P.A., in walked David Golden! Neither I nor anyone else in had ever met him -- notwithstanding the fact that he lives in Brooklyn and had attended YAPC in Toronto along with us!

David's collaboration was very helpful. I went over to his house one evening -- those of you who know New York City know how rare going over to a fellow geek's house is -- I met his lovely wife and I heard what he had to say about ExtUtils::ModuleMaker and how it should evolve. David had implemented the concept of a personal defaults file in ExtUtils::ModuleMaker::TT well before Module::Starter did so. He strongly advocated that I do so for ModuleMaker as well. He also urged me to rationalize the naming of variables and subroutines, to make subclassing easier, and to document how ModuleMaker offers its users more sophisticated ways to use it as they become more familiar with it.

David's assistance was generous, notwithstanding the fact that my revisions broke the code in his subclass. The problem was that its code was very tightly tied to that of Geoff Avery's version of ModuleMaker. But its purpose -- creation of Template::Toolkit-style templates -- was not as close to that of ModuleMaker as it first appeared. By mid-September, I had to decide what features of ModuleMaker I would be prepared to support for the indefinite future. I came to the conclusion that I wouldn't be able to guarantee support for all the features David wanted. David conceded that ExtUtils::ModuleMaker::TT was one of his early attempts at a CPAN module and subsequently reworked it so as not to break.

CPAN Testers

As challenging as David Golden's feedback was, it was less shocking than the feedback which came from the people who do automated testing of CPAN distributions. These testers send automated test reports which you can view in a web interface, a news interface and a interface. The testers grade your distribution on a PASS/FAIL basis. If you FAIL, you get a printout equivalent of what you yourself would get if you called make test. You then have to figure out why your test suite -- which presumably PASSed with flying colors when you ran it on your own box -- FAILed at some specific point on someone else's box. You might also have to figure out why it passed on one tester's Linux 2.4 box while failing on another tester's Linux 2.4 box. And when you upload a new version, you may get a FAIL from someone who PASSed your earlier version.

If a tester has a setup which installs everything it PASSes -- and installs all the prerequisites needed attain that PASS -- a subtle bias in favor of PASSing is imparted to later versions of the same distribution run on the same system.

But some CPAN testers -- notably imacat from Taiwan -- maintain a very pristine testing environment. It was quite an effort to get ExtUtils::ModuleMaker and ExtUtils::ModuleMaker::PBP to pass on both her Linux and Cygwin boxes. The source of the problem was the placement of the personal defaults file. Before I knew how to make sure that my testing removed the testing version of the personal defaults file from her home directory, I was leaving that version there, which polluted the environment when she went to test a later version.

Take It on the Road

Once I was convinced that ModuleMaker's functionality was improved and that it would pass its tests on anybody's box, I decided to take it on the road for some promotion. In late October of last year I gave the first version of this talk to the Toronto Perlmongers. As we were sitting in the bar after the talk, Michael Graham pointed out that the functionality I had added to guarantee safe testing in a user's home directory was not inherently connected to ModuleMaker. He urged me to split that off into a separate Perl extension. The result was my next CPAN distribution, File::Save::Home ( A couple of weeks later I applied the same process of refactoring out code that wasn't strictly tied to the module's functionality to File::Save::Home. The result: yet another CPAN distribution: String::PerlIdentifier (

But Enough About Me

I'd now like to shift the focus away from my adventures with ModuleMaker and discuss the role that new module maintainers play in the ongoing development of CPAN. The concept of an open source community is at most twenty years old; CPAN is ten. Like most open source projects, CPAN is entirely a volunteer effort. Very few people, if any, are monetarily compensated for maintaining CPAN distributions. So it's not surprising that, as CPAN authors go through different experiences in their lives, they often don't have the time to maintain their modules as well as they would like. Quite a few CPAN distributions have been transferred by their original authors to new maintainers. As CPAN and the Perl community mature, this will necessarily occur more often.

In preparing for this talk, I decided to ask some other Perl hackers who had taken over maintenance of CPAN modules about their experiences in doing so. In May and June I interviewed, either in person or via e-mail, the maintainers of seventeen CPAN distributions, some well known, some not. I wanted to see what they had experienced in taking over other people's modules and what they felt about the process. The questions I asked included the following:

  1. Were you a frequent user of the module before taking over its maintenance from the author or previous maintainer?
  2. In most cases the new maintainer had used the module before assuming maintenance. In a few cases the maintainer was a frequent user of another module for which the module taken over was a prerequisite. But in one case a maintainer took over a module with the explicit purpose of deprecating it (with the original author's consent) and redirecting users to a new and better CPAN module he was writing.

  3. Before taking over the module's maintenance, had you submitted patches to its author or previous maintainer?
  4. In almost all cases, the new maintainer either had submitted patches to the author or had privately developed enhanced functionality.

  5. Did the author initiate the change in maintenance, or did you? (Or was it some combination of the two?)
  6. Sometimes the author's workload had increased and he asked a person who had submitted bug reports to take over. In other cases the maintainer initiated the change. And in still others my interviewees described it as a combination of the two. In one case a module author was unresponsive to bug reports and the moderators of the list designated the new maintainer as co-maintainer.

  7. Once you took over maintenance of the module, how much interaction did you have with its previous maintainer?
  8. In most cases the new maintainer had little or no contact with the author or previous maintainer after assuming maintenance. But in certain cases the current maintainer became a co-maintainer of the module.

  9. Once you took over maintenance of the module, to what extent did you change its coding style from the original?
  10. In most cases, the maintainer either made no stylistic changes or made changes only in new sections of code added.

  11. Were there aspects of the module that you felt were incorrect and subsequently fixed?
  12. Wide range of responses. Some made almost no changes; others did a complete re-write.

  13. To what extent did you modify the module's test suite?
  14. Four of the new maintainers made few changes in the test suite, but most did substantial revisions (e.g., adding more tests, re-implementing the test suite with Test::More, improving the test suite's coverage of the code, etc.).

  15. To what extent did you re-write or modify the module's documentation?
  16. Some maintainers made minimal changes to documentation. But most wrote new documentation where new functionality had been added or where the API had changed.

  17. In retrospect, what were the most positive -- and most negative -- aspects of the process of taking over the module's maintenance.
  18. On the positive side, let me cite one respondent:

    ''The pride I took in publishing the modules (gives you a warm feeling inside when you can help a greater cause) and of course the positive feedback from the users, even if combined with support requests.''

    Another cited the great satisfaction he got in solving a bug; the solution led to an article on Perlmonks.

    Another cited the satisfaction of making the code backwardly compatible.

    And yet another said, ''I took over a module from Schwern!''

    On the negative side: A number of respondents cited how frustrating it was when module authors failed to respond to bug reports or patch submissions.

    And one cited the frustration involved in making the code backwardly compatible. (This was the same maintainer who took joy in making the code backwardly compatible.)

  19. What would you say you learned in the maintenance process -- with respect to either Perl code or interacting with other programmers?
  20. My respondents emphasized what they learned about interacting with other programmers much more than what they learned about Perl code. Several maintainers cited the importance of testing and documentation. One sent me a 3-page email on the subject of ''comments are your friends.'' Another cited the importance of posting bug reports on as a way of establishing your interest in the development of the module and thereby positioning yourself as a possible future maintainer.

    In two cases a maintainer took over maintenace of core modules from the Perl5 Porters. He cited learning about the process of maintaining core modules and interacting with the porters.

Lessons Learned

On the basis of my experience with Phalanx and with ExtUtils::ModuleMaker, and taking into consideration what other module maintainers reported, I would say that the maintenance of CPAN modules works best when the following conditions are met:

In closing, I'd like to quote one British CPAN author and maintainer, who wrote me:

''[T]he small, unglamorous modules that people just use shouldn't be just left to whither in some forgotten corner. I'm sure there are people clamouring to maintain the more high-profile modules for which kudos abounds, but there are probably hundreds more that haven't been touched for ages and may not even work anymore: this lowers the overall impression of quality of CPAN and also causes people to re-invent wheels. People should be actively encouraged to take over the maintenance of old modules ...''