[Bioperl-l] Network tests overhaul

[Sendu Bala]

When the test suite runs currently, most (the intent is all) tests skip 
if the test would require network (internet) access. This is to avoid 
tests failing not due to bugs in Bioperl code, but due to temporarily 
inaccessible servers. This is also to make running the test suite faster.

To do a complete test you currently have to set BIOPERLDEBUG to true, 
which activates the network test but also increases verbosity. This 
actually causes a problem, since when running the entire test suite the 
additional debug information is more a hindrance than a help, since the 
reams of printed information can hide significant warnings that may also 
get printed. Its also ugly.

The solution is to divorce activation of network tests from the request 
for verbosity. The obvious implementation is to have another environment 
variable, perhaps BIOPERLNETWORK. However, there is an opportunity to do 
something more appropriate. The running of networking tests should be a 
choice given to every end-user installing Bioperl. Debugging 
information, on the other hand, is only of interest to the developer 
working on a specific module under test, so can be left as a 'hidden' 
env var.


I have just committed one possible implementation along these lines.

You say:
perl Build.PL
as normal, and if you seem to have internet access it asks you if you'd 
like to run network tests. The default answer is no. If you answer yes, 
network tests will be enabled.

You can alternatively say:
perl Build.PL --network
and if you seem to have internet access, network tests will be enabled.

Then you run the tests:
./Build test
Any tests written to support the new system will then skip network tests 
if they haven't been enabled.

The only test I've written to support the new system is t/RemoteBlast.t:
./Build test --test_files t/RemoteBlast.t --verbose


Adding support to test scripts consists of the following changes:

+ use Module::Build;
+ my $build = Module::Build->current(get_options => { network => {} });
+ my $do_network_tests = $build->notes('network');

! if (!$ENV{'BIOPERLDEBUG'}) { # skip network tests
---
! if (!$do_network_tests) { # skip network tests


I propose adding this support to all test scripts that carry out network 
tests. Does anyone have objections? Does anyone have alternate 
implementations that may be superior?

I specifically suggest we don't use an env var in addition to the above, 
because the multiple ways of doing things could lead to confusion. Which 
takes priority? Did a user really have the networking tests turned on 
when he reported his test results?


The one thing I need help with is identifying which tests attempt to 
access the internet. I think we caught most of them for the 1.5.2 
release, but I think there are more lurking around. Can anyone offer a 
way to systematically find at least the test scripts which access the 
internet, if not the specific tests within?

Cheers,
Sendu.