The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.

NAME

ExtUtils::MakeMaker::BigHelper - for helping ExtUtils::MakeMaker with big XS projects.

SYNOPSIS

use ExtUtils::MakeMaker::BigHelper qw(:find);

  This exports find_files and find_directories, which might be useful with
  WriteMakefile from ExtUtils::MakeMaker.

use ExtUtils::MakeMaker::BigHelper qw(:MY);

  Use this after a "package MY;" statement.  It will export methods that
  are used by ExtUtils::MakeMaker and available for customization.  These
  customized methods will alter the behaviour of ExtUtils::MakeMaker as
  documented below.

DESCRIPTION

This package extends or alters the functionality of ExtUtils::MakeMaker, in a way more suitable perhaps for large projects using a lot of perl XS.

This allows multiple .xs files in your project, strewn about the lib directory hierarchy, hopefully side by each with the corresponding .pm file. Multiple t directories are also allowed, hopefully located next to the .pm files they test.

See the man page for perlxs for more information about perl XS.

With ExtUtils::MakeMaker there can only be one .xs file, which limits the size of the project. Well, there is a way to have more, but you're left with one master .xs file responsible for bootstrapping all the other ones. The way to have two .xs files is documented, but not easy, and the way to have more than two is, well, unnatural.

With ExtUtils::MakeMaker you're allowed to provide customizations for various MakeMaker methods in the package MY. This gives you the ability to make a total overhaul of ExtUtils::MakeMaker.

This package uses the customization facility built in to ExtUtils::MakeMaker to allow multiple .xs files, multiple t files, and multiple subproject (Makefile.PL and Build.PL) directories in the lib hierarchy rather than at the top level of the project. This allows you to more conveniently use perlxs in your project, without having to package up multiple Makefile.PL projects embedded in your directories. It also allows you to convert all those .pm files with Inline code into real perlxs without using dynamite.

The methods here are meant to be exported into the package MY, to provide customizations of the ExtUtils::MakeMaker methods.

For example, here's a possible project layout:

Changes
lib/Big/Project.pm
lib/Big/Project/Collections/MyLRU.xs
lib/Big/Project/Collections/MyLRU.pm
lib/Big/Project/Collections/t/00-usage.t
lib/Big/Project/Collections/t/10-testlru.t
lib/Big/Project/Worker.pm
lib/Big/Project/Slave.pm
lib/Big/Project/Dispatcher/Scheduler.xs
lib/Big/Project/Dispatcher/Scheduler.pm
lib/Big/Project/Dispatcher/t/00-usage.t
lib/Big/Project/Dispatcher/t/10-roundrobin.t
lib/Big/Project/clib/Makefile
lib/Big/Project/clib/toolbox.c
lib/Big/Project/clib/toolbox.h
lib/Big/Project/clib/hashmaker.c
lib/Big/Project/clib/hashmaker.h
Makefile.PL
MANIFEST
README

The Makefile.PL would look like this:

use ExtUtils::MakeMaker;

WriteMakefile(
  NAME => "Big::Project",
  VERSION_FROM => "lib/Big/Project.pm",
  ($] >= 5.005 ?     ## Add these new keywords supported since 5.005
    (ABSTRACT       => 'Big Project',
     AUTHOR         => 'John Bigbooty <john.bigbooty@yoyodyne.com>') : ()),
  DEFINE            => '',
  LIBS              => ['-lm' ], # e.g., '-lm'
  INC               => '-I. -Ilib/Big/Project/clib',
  MYEXTLIB          => 'lib/Big/Project/clib/libmyextlib.a',
);

package MY;

use ExtUtils::MakeMaker::BigHelper qw(:MY);

That should do it. It uses File::Find to descend the hierarchy and find all the .xs files and t directories.

$found = find_files($regex, @directories)

This function is exported by the :find tag, and looks up all files matching the regex, under the listed directories. It returns them in a hashref where the keys are the files found and the value is 1.

$found = find_directories($regex, @directories)

This function is exported by the :find tag, and looks up all directories matching the regex, under the listed directories. It returns them in a hashref where the keys are the directories found and the value is 1.

$obj-init_dirscan>

This function is exported by the :MY tag. It extends the MM init_dirscan.

The MM init_dirscan only looks for .xs files in the top level directory. This extension descends into lib to find all .xs files, and sets them into the $self XS setting.

Note that the XS setting is supposed to be a hashref of .xs files, where the value is the corresponding .c file. However, with these overrides, the value need only be 1 if the .c file is as expected.

$obj-test>

This function is exported by the :MY tag. It extends the MM test.

The MM test only looks for t directories at the top level directory. This extension descends into lib to find all t directories, and sets them into the $self TESTS setting.

The $self TESTS setting is supposed to be a blank separated list of t files (with shell wildcards) to run.

Note that the XS setting is supposed to be a hashref of .xs files, where the value is the corresponding .c file. However, with these overrides, the value need only be 1 if the .c file is as expected.

$obj-init_PM>

This function is exported by the :MY tag. It extends/replaces the MM init_PM.

This method only puts .pm files into the $self PM setting.

The default init_PM puts all files into the $self PM setting.

$obj-init_XS>

This function is exported by the :MY tag. It extends the MM init_XS.

This sets up macros INST_STATIC, INST_DYNAMIC, INST_BOOT, and uses INST_ARCHLIB.

note: init_dirscan will set XS and DIR to defaults. However, init_dirscan has limitations, mostly that it cannot handle large projects.

input and output:

XS          as documented in ExtUtils::MakeMaker, this is a hashref.
            the key is the relative path of the xs file.
            the value can be the target c file as documented.
            EXTENSION: the value can be simply 1 if the c file can be
                       simply generated from the xs file.
            EXTENSION: the value can be the package name XXX::YYY::ZZZ for
                       the xs code.
            EXTENSION: the value can be an array of c dependencies
            EXTENSION: the XS hash is reworked by this code.

DIR         as documented, an arrayref of subdirectories containing
            Makefile.PL
            EXTENSION: may be an arrayref of pathnames for Makefile.PLs
                       and Build.PLs init_dirscan only looks one deep for
                       Makefile.PLs, and does not look for Build.PLs.
                       With this extension, you could do a recursive file
                       find for all Makefile.PLs and Build.PLs no matter
                       how deep.
            
MYEXTLIB

output

MY_XS_DEPENDENCIES
MY_SUBDIRS
MY_EXTENSION_LIBS
MY_XS_TARGETS

$obj-clean_subdirs>

This function is exported by the :MY tag. It replaces the MM clean_subdirs.

This handles cleaning up subdirectories with their own Makefile, such as Makefile.PL, Build.PL, and extension libraries.

$obj-clean>

This function is exported by the :MY tag. It extends the MM clean.

This handles cleaning up all the debris left by building .xs files, without having to have the .c files named explicitly in the $self XS setting.

$obj-postamble>

This function is exported by the :MY tag. It extends the MM postamble.

This handles setting up the compilation of the .xs files with a VERSION for each. The bootstrap code checks the VERSION, and it has to match.

$obj-dynamic_bs>

This function is exported by the :MY tag. It extends the MM dynamic_bs.

This handles the .bs files for multiple non-chained .xs files.

$obj-dynamic_lib>

This function is exported by the :MY tag. It extends the MM dynamic_lib.

This handles building multiple .xs files into .so.

EXPORT

None by default. Use the tags :find and :MY to export the useful things.

SEE ALSO

Mention other useful documentation such as the documentation of related modules or operating system documentation (such as man pages in UNIX), or any relevant external documentation such as RFCs or standards.

If you have a mailing list set up for your module, mention it here.

If you have a web site set up for your module, mention it here.

AUTHOR

Rob Janes, <edgewise@cpan.org>

COPYRIGHT AND LICENSE

Copyright (C) 2013 by Rob Janes

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.12.3 or, at your option, any later version of Perl 5 you may have available.