[Bioperl-l] Spankin' new (alpha) build system for Bioperl-Run

Tue Sep 30 06:07:36 UTC 2014

Very cool! Nice work Mark!!

On Mon, Sep 29, 2014 at 11:30 PM, Mark A. Jensen <maj at fortinbras.us> wrote:

>  (minus damn linebreaks)
>
>
> All (esp. George)-
>
> My work on Issue #11 (https://github.com/bioperl/bioperl-run/issues/11) has metastasized.
>
> The proximate problem was tests that fail because of once-local prerequisites. The ultimate problems are
>
> - Why should I have to install every single wrapper when I only want X?
>
> - Why should I care about any test that doesn't deal with X?
>
> - Why doesn't X bring along its own prereq metadata (including Bio prereqs), rather than tag along with the distro and hope for the best?
>
> (And I think these are the ultimate problems across BioPerl in terms of decentralized distribution.)
>
> My solution was
>
> - Add to the distro real, manually prepared metadata on prerequisites for all the tools
>
> - Add an interactive selector that allows a user to pick their desired tools at perl Build.PL-time
>
> - Have Module::Build check only (and ALL) the prereqs of the desired tools, and inform user of missing ones at perl Build.PL-time
>
> - Make use of the persistence of the config information to skip/run .t files as appropriate
>
> - Update ALL the tests to check whether to skip based on user selection
>
> - Make M::B install only the relevant distro modules and documentation, not everything, at ./Build install-time
>
> This is ready for brave alpha-testers at https://github.com/bioperl/bioperl-run/tree/topic/issue11. Just do 'perl Build.PL'.
>
> Pod below has some more details-- comments very welcome
>
> MAJ
>
> NAME
>     Bio::Tools::Run::Build - Instrument the build for features
>
> SYNOPSIS
>
> ...
>
> DESCRIPTION
>
> Bio::Tools::Run::Build is a subclass of Module::Build that allows an author to offer users the ability to select and install pre-configured subsets of modules that are packaged in a single large M::B-based distribution.
>
> Grouping and selection of distro modules is driven by the optional features concept as defined in CPAN::Meta::Spec and used by Module::Build.
>
> The subclass provides the following:
>
> *   Author specification of features and their prereqs
>
> The build author develops metadata files in json that follow "optional_features" in CPAN::Meta::Spec to group distribution modules and dependencies as selectable features.
>
> *   Interactive user selection of features
>
> The user can be presented with an interactive selector during Build.PL runs.
>
> *   Prereq checking of user selected features only
>
> M::B only checks for the presence of selected feature dependencies.
>
> *   Build-persistent recording of user selections
>
> The build object records the selection of features in the $build->feature field. This can be used in test files to determine whether tests should be skipped (and not failed). See Bio::Tools::Run::Build::Test.
>
> *   Installation only of selected feature modules
>
> Bio::Tools::Run::Build adds a build action, "deselect", which runs after the "code" and "docs" actions. "deselect" removes unselected modules from the blib/lib directory and unneeded documentation from the blib/libdoc directory. This keeps the "install" action from installing unwanted files.
>
> MOTIVATION
>
> The BioPerl-Run distribution contains a large variety of wrappers and parsers that handle the execution and output of many different bioinformatics tools. It has been provided as a large distro that installs and attempts to test all of its modules. Many users need only a small fraction of the functionality BioPerl-Run provides, relevant only to the tools they have installed. On the other hand, managing many different packages is unwieldy and uninviting for volunteer maintainers.
>
> The system described here is a compromise that enables a user to select, test and install only those modules that meet the need, yet reduces the maintenance effort to the management of a set of metadata files in a single distribution.
>
> On 2014-09-29 23:41, Mark A. Jensen wrote:
> > All (esp. George)-
> >
>
>  All (esp. George)-
>
> My work on Issue #11 (https://github.com/bioperl/bioperl-run/issues/11) has metastasized.
>
> The proximate problem was tests that fail because of once-local prerequisites. The ultimate
> problems are
>
> - Why should I have to install every single wrapper when I only want X?
> - Why should I care about any test that doesn't deal with X?
> - Why doesn't X bring along its own prereq metadata (including Bio prereqs),
>   rather than tag along with the distro and hope for the best?
>
> (And I think these are the ultimate problems across BioPerl in terms of decentralized
> distribution.)
>
> My solution was
>
> - Add to the distro real, manually prepared metadata on prerequisites for all
>   the tools
> - Add an interactive selector that allows a user to pick their desired tools at
>   perl Build.PL-time
> - Have Module::Build check only (and ALL) the prereqs of the desired tools, and
>   inform user of missing ones at perl Build.PL-time
> - Make use of the persistence of the config information to skip/run .t files as
>   appropriate
> - Update ALL the tests to check whether to skip based on user selection
> - Make M::B install only the relevant distro modules and documentation, not everything,
>   at ./Build install-time
>
> This is ready for brave alpha-testers at https://github.com/bioperl/bioperl-run/tree/topic/issue11.
> Just do 'perl Build.PL'.
>
> Pod below has some more details-- comments very welcome
>
> MAJ
>
> NAME
>     Bio::Tools::Run::Build - Instrument the build for features
>
> SYNOPSIS
>
> ...
>
> DESCRIPTION
>     Bio::Tools::Run::Build is a subclass of Module::Build that allows an
>     author to offer users the ability to select and install pre-configured
>     subsets of modules that are packaged in a single large M::B-based
>     distribution.
>
>     Grouping and selection of distro modules is driven by the optional
>     features concept as defined in CPAN::Meta::Spec and used by
>     Module::Build.
>
>     The subclass provides the following:
>
>     *   Author specification of features and their prereqs
>
>         The build author develops metadata files in json that follow
>         "optional_features" in CPAN::Meta::Spec to group distribution
>         modules and dependencies as selectable features.
>
>     *   Interactive user selection of features
>
>         The user can be presented with an interactive selector during
>         Build.PL runs.
>
>     *   Prereq checking of user selected features only
>
>         M::B only checks for the presence of selected feature dependencies.
>
>     *   Build-persistent recording of user selections
>
>         The build object records the selection of features in the
>         $build->feature field. This can be used in test files to determine
>         whether tests should be skipped (and not failed). See
>         Bio::Tools::Run::Build::Test.
>
>     *   Installation only of selected feature modules
>
>         Bio::Tools::Run::Build adds a build action, "deselect", which runs
>         after the "code" and "docs" actions. "deselect" removes unselected
>         modules from the blib/lib directory and unneeded documentation from
>         the blib/libdoc directory. This keeps the "install" action from
>         installing unwanted files.
>
> MOTIVATION
>     The BioPerl-Run distribution contains a large variety of wrappers and
>     parsers that handle the execution and output of many different
>     bioinformatics tools. It has been provided as a large distro that
>     installs and attempts to test all of its modules. Many users need only a
>     small fraction of the functionality BioPerl-Run provides, relevant only
>     to the tools they have installed. On the other hand, managing many
>     different packages is unwieldy and uninviting for volunteer maintainers.
>
>     The system described here is a compromise that enables a user to select,
>     test and install only those modules that meet the need, yet reduces the
>     maintenance effort to the management of a set of metadata files in a
>     single distribution.
>
> ...
>
>   Message 1 of 152
>
> _______________________________________________
> Bioperl-l mailing list
> Bioperl-l at mailman.open-bio.org
> http://mailman.open-bio.org/mailman/listinfo/bioperl-l
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.open-bio.org/pipermail/bioperl-l/attachments/20140930/d6f9a4ac/attachment.html>