[Biojava-dev] Build system changes

[Thomas Down]

Once upon a time, Len Trigg wrote:
> Thomas Down wrote:
> >One worry, though, is that if you watch memory usage during
> >an 'ant runtests', it creeps up pretty rapidly even when
> >running tests which only use a few small objects.  I'm wondering
> >how much of the problem is actually data, and how much is due
> >to keeping lots of classes in memory.  BioJava is a pretty big
> 
> I suppose that could account for a lot. How much memory does it take? As 
> a data point, our main src module here has about 10% more classes than 
> biojava, and runs around 4000 tests. We give the JVM 200MB when running 
> the tests. I think we had problems with 128MB. Our stuff can be very 
> memory intensive while running (text categorization, lots of documents 
> and classifiers etc), so I would say has a higher memory requirement 
> than biojava.

Okay, I think I've got to the bottom of this now.  When running
in fork="no" mode, the Ant JUnit task normally creates a new
ClassLoader for every test.  Now, the core of BioJava is
pretty interdependant, so significant parts of the core are
needed by many other classes.  In practice, this means that
a test run ends up loading about 50 copies of AlphabetManager
and friends, so it's not surprising that it runs out of memory!

Ant 1.6.0 has added a new option which allows all the tests
to be run in a single ClassLoader.  That solves most of the
memory issues, and also makes the tests run faster (factor of >2).

I've just checked in a build.xml patch which enables this mode
by default when you do an "ant runtests".  I've added an extra
target, "runtests-fork", which gives the old behaviour.  If, for
some reason, you can't upgrade to ant 1.6, use "runtests-fork".

(In principle, sharing a ClassLoader is a little bit risky,
since one test could modify static data and affect the result
of a later test.  However, we don't have many static variables
in BioJava, so I don't think this is a serious risk.  Something
to be aware of, though).

    Thomas.