[Bioperl-l] Spankin' new (alpha) build system for Bioperl-Run

[Mark A. Jensen]

(minus damn linebreaks)

All (esp. George)-

My work on Issue #11
(https://github.com/bioperl/bioperl-run/issues/11) has
metastasized.

The proximate problem was tests that fail because of
once-local prerequisites. The ultimate problems are

- Why should I have
to install every single wrapper when I only want X?

- Why should I care
about any test that doesn't deal with X?

- Why doesn't X bring along
its own prereq metadata (including Bio prereqs), rather than tag along
with the distro and hope for the best?

(And I think these are the
ultimate problems across BioPerl in terms of decentralized
distribution.)

My solution was

- Add to the distro real, manually
prepared metadata on prerequisites for all the tools

- Add an
interactive selector that allows a user to pick their desired tools at
perl Build.PL-time

- Have Module::Build check only (and ALL) the
prereqs of the desired tools, and inform user of missing ones at perl
Build.PL-time

- Make use of the persistence of the config information
to skip/run .t files as appropriate

- Update ALL the tests to check
whether to skip based on user selection

- Make M::B install only the
relevant distro modules and documentation, not everything, at ./Build
install-time

This is ready for brave alpha-testers at
https://github.com/bioperl/bioperl-run/tree/topic/issue11. Just do 'perl
Build.PL'.

Pod below has some more details-- comments very
welcome

MAJ

NAME
 Bio::Tools::Run::Build - Instrument the build for
features

SYNOPSIS

...

DESCRIPTION

Bio::Tools::Run::Build is a
subclass of Module::Build that allows an author to offer users the
ability to select and install pre-configured subsets of modules that are
packaged in a single large M::B-based distribution.

Grouping and
selection of distro modules is driven by the optional features concept
as defined in CPAN::Meta::Spec and used by Module::Build.

The subclass
provides the following:

* Author specification of features and their
prereqs

The build author develops metadata files in json that follow
"optional_features" in CPAN::Meta::Spec to group distribution modules
and dependencies as selectable features.

* Interactive user selection
of features

The user can be presented with an interactive selector
during Build.PL runs.

* Prereq checking of user selected features
only

M::B only checks for the presence of selected feature
dependencies.

* Build-persistent recording of user selections

The
build object records the selection of features in the $build->feature
field. This can be used in test files to determine whether tests should
be skipped (and not failed). See Bio::Tools::Run::Build::Test.

*
Installation only of selected feature modules

Bio::Tools::Run::Build
adds a build action, "deselect", which runs after the "code" and "docs"
actions. "deselect" removes unselected modules from the blib/lib
directory and unneeded documentation from the blib/libdoc directory.
This keeps the "install" action from installing unwanted
files.

MOTIVATION

The BioPerl-Run distribution contains a large
variety of wrappers and parsers that handle the execution and output of
many different bioinformatics tools. It has been provided as a large
distro that installs and attempts to test all of its modules. Many users
need only a small fraction of the functionality BioPerl-Run provides,
relevant only to the tools they have installed. On the other hand,
managing many different packages is unwieldy and uninviting for
volunteer maintainers.

The system described here is a compromise that
enables a user to select, test and install only those modules that meet
the need, yet reduces the maintenance effort to the management of a set
of metadata files in a single distribution.

On 2014-09-29 23:41, Mark
A. Jensen wrote:
> All (esp. George)-
> 

All (esp. George)-

My work on
Issue #11 (https://github.com/bioperl/bioperl-run/issues/11) has
metastasized.

The proximate problem was tests that fail because of
once-local prerequisites. The ultimate
problems are

- Why should I have
to install every single wrapper when I only want X?
- Why should I care
about any test that doesn't deal with X?
- Why doesn't X bring along its
own prereq metadata (including Bio prereqs),
 rather than tag along with
the distro and hope for the best?

(And I think these are the ultimate
problems across BioPerl in terms of decentralized
distribution.)

My
solution was

- Add to the distro real, manually prepared metadata on
prerequisites for all
 the tools
- Add an interactive selector that
allows a user to pick their desired tools at
 perl Build.PL-time
- Have
Module::Build check only (and ALL) the prereqs of the desired tools,
and
 inform user of missing ones at perl Build.PL-time
- Make use of the
persistence of the config information to skip/run .t files as

appropriate
- Update ALL the tests to check whether to skip based on
user selection
- Make M::B install only the relevant distro modules and
documentation, not everything,
 at ./Build install-time

This is ready
for brave alpha-testers at
https://github.com/bioperl/bioperl-run/tree/topic/issue11.
Just do 'perl
Build.PL'.

Pod below has some more details-- comments very
welcome

MAJ

NAME
 Bio::Tools::Run::Build - Instrument the build for
features

SYNOPSIS

...

DESCRIPTION
 Bio::Tools::Run::Build is a
subclass of Module::Build that allows an
 author to offer users the
ability to select and install pre-configured
 subsets of modules that
are packaged in a single large M::B-based
 distribution.

 Grouping and
selection of distro modules is driven by the optional
 features concept
as defined in CPAN::Meta::Spec and used by
 Module::Build.

 The
subclass provides the following:

 * Author specification of features
and their prereqs

 The build author develops metadata files in json
that follow
 "optional_features" in CPAN::Meta::Spec to group
distribution
 modules and dependencies as selectable features.

 *
Interactive user selection of features

 The user can be presented with
an interactive selector during
 Build.PL runs.

 * Prereq checking of
user selected features only

 M::B only checks for the presence of
selected feature dependencies.

 * Build-persistent recording of user
selections

 The build object records the selection of features in the

$build->feature field. This can be used in test files to determine

whether tests should be skipped (and not failed). See

Bio::Tools::Run::Build::Test.

 * Installation only of selected feature
modules

 Bio::Tools::Run::Build adds a build action, "deselect", which
runs
 after the "code" and "docs" actions. "deselect" removes
unselected
 modules from the blib/lib directory and unneeded
documentation from
 the blib/libdoc directory. This keeps the "install"
action from
 installing unwanted files.

MOTIVATION
 The BioPerl-Run
distribution contains a large variety of wrappers and
 parsers that
handle the execution and output of many different
 bioinformatics tools.
It has been provided as a large distro that
 installs and attempts to
test all of its modules. Many users need only a
 small fraction of the
functionality BioPerl-Run provides, relevant only
 to the tools they
have installed. On the other hand, managing many
 different packages is
unwieldy and uninviting for volunteer maintainers.

 The system
described here is a compromise that enables a user to select,
 test and
install only those modules that meet the need, yet reduces the

maintenance effort to the management of a set of metadata files in a

single distribution.

...

Message 1 of 152 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.open-bio.org/pipermail/bioperl-l/attachments/20140930/de509ea1/attachment.html>