[Bioperl-l] Bioperl partitioning (was Re: SVN and ...Re: Perltidy)

[Chris Fields]

On Jun 19, 2007, at 1:54 PM, Steve Chervitz wrote:

> Valid points, Sendu. I wonder if there might be a best-of-both-worlds
> approach here. I would not be advocating for a major slice and dice,
> but just identifying a few large, reasonably well established and
> encapsulated blocks of functionality that could be managed more
> independently and segregating them away from the rest. For example:
> DB, Graphics, Search+SearchIO, Tools.

There should also be a consensus between the core devs on this; I  
don't see it going very far w/o Lincoln, Jason, Hilmar, etc. voicing  
their opinions as it will directly impact projects which rely on core  
functionality (GBrowse/GMOD, bioperl-db, etc).  I also agree with  
George that this should be postponed until after svn issues are taken  
care of.

Stating that, I think this is a good idea in general, though we'll  
need to be careful which ones we segregate out as non-core.  I agree  
with your choices; I would add in Bio::Restriction, Bio::Assembly,  
Bio::Structure, and a few more.  As long as the distribution required  
installation of 'core' prior to test runs it shouldn't be too much of  
a problem.

In order for this to work we would need to delineate what defines  
'core' (how broad the definition should be), then identify those  
modules that don't fit and decide what to do with them.  Would we  
want to split the others into separate packages or lump together as a  
bioperl-auxiliary (horrid name, but you get my point)?  Too many  
could be a logistical nightmare, as Sendu has pointed out.

> Once per year, we could have a "whole caboodle" release where the core
> and all sub parts are tested and released as a group, as we currently
> do. Then, updates to the sub parts can occur as-needed but without
> necessarily involving updates to other sub parts or the core.

Sounds fine by me.  Actually, my thought was we could reimplement  
Bundle::BioPerl on CPAN (which Module::Build effectively obsoleted)  
to install all the necessary subpackages in order to emulate an old- 
style 'core' installation, or act as an 'install everything BioPerl- 
related' Bundle.  Regular updates of the subpackages to CPAN should  
just require updating the Bundle (which would update only the  
relevant parts, at least I believe it would).

> The onus would be on the pumpkin for the sub part release to make sure
> it continues to work with the last whole caboodle release. This would
> minimize the number of release clashes, since sub part updates would
> only be sanctioned relative to the last caboodle release, and it would
> ensure that the whole set continues to interoperate.
>
> Perhaps it would be worth experimenting with such an approach so we
> can judge it based on actual experience. We could identify one
> functional sub part and segregate it out, do a release cycle or two,
> along with a sub part release, and decide if this makes things easier
> or harder, for devs as well as users. We could always bring it back
> into the fold if it doesn't work out.
>
> My fear is that as bioperl continues to grow, the monolithic approach
> will become increasingly onerous for a single release pumpkin to
> manage, and harder to find someone who feels up to the task. It could
> also discourage new developers from diving into the codebase if it
> looks too deep. And they are our lifeblood.

Agreed!

> A more functionally segregated bioperl codebase could lower the
> activation energy needed to recruit release pumpkins and new devs,
> leading to more release iterations, fewer bugs, more features, and
> more sustainable growth.

'Activation energy.'  Hmm.  Spoken like a true biologist.

> When I first discovered Bioperl in 1996, it had three modules. At
> ~900, I  probably wouldn't have joined ranks as a developer (well, I
> probably would, but it would have taken a while to digest it and
> become a contributor).
>
> Steve

I pretty much agree, though this will require quite a bit more  
discussion.

chris