[Bioperl-l] Network tests overhaul

[Hilmar Lapp]

Sounds like a great idea to me. -hilmar

On Jun 18, 2007, at 6:44 AM, Sendu Bala wrote:

> When the test suite runs currently, most (the intent is all) tests  
> skip
> if the test would require network (internet) access. This is to avoid
> tests failing not due to bugs in Bioperl code, but due to temporarily
> inaccessible servers. This is also to make running the test suite  
> faster.
>
> To do a complete test you currently have to set BIOPERLDEBUG to true,
> which activates the network test but also increases verbosity. This
> actually causes a problem, since when running the entire test suite  
> the
> additional debug information is more a hindrance than a help, since  
> the
> reams of printed information can hide significant warnings that may  
> also
> get printed. Its also ugly.
>
> The solution is to divorce activation of network tests from the  
> request
> for verbosity. The obvious implementation is to have another  
> environment
> variable, perhaps BIOPERLNETWORK. However, there is an opportunity  
> to do
> something more appropriate. The running of networking tests should  
> be a
> choice given to every end-user installing Bioperl. Debugging
> information, on the other hand, is only of interest to the developer
> working on a specific module under test, so can be left as a 'hidden'
> env var.
>
>
> I have just committed one possible implementation along these lines.
>
> You say:
> perl Build.PL
> as normal, and if you seem to have internet access it asks you if  
> you'd
> like to run network tests. The default answer is no. If you answer  
> yes,
> network tests will be enabled.
>
> You can alternatively say:
> perl Build.PL --network
> and if you seem to have internet access, network tests will be  
> enabled.
>
> Then you run the tests:
> ./Build test
> Any tests written to support the new system will then skip network  
> tests
> if they haven't been enabled.
>
> The only test I've written to support the new system is t/ 
> RemoteBlast.t:
> ./Build test --test_files t/RemoteBlast.t --verbose
>
>
> Adding support to test scripts consists of the following changes:
>
> + use Module::Build;
> + my $build = Module::Build->current(get_options => { network =>  
> {} });
> + my $do_network_tests = $build->notes('network');
>
> ! if (!$ENV{'BIOPERLDEBUG'}) { # skip network tests
> ---
> ! if (!$do_network_tests) { # skip network tests
>
>
> I propose adding this support to all test scripts that carry out  
> network
> tests. Does anyone have objections? Does anyone have alternate
> implementations that may be superior?
>
> I specifically suggest we don't use an env var in addition to the  
> above,
> because the multiple ways of doing things could lead to confusion.  
> Which
> takes priority? Did a user really have the networking tests turned on
> when he reported his test results?
>
>
> The one thing I need help with is identifying which tests attempt to
> access the internet. I think we caught most of them for the 1.5.2
> release, but I think there are more lurking around. Can anyone offer a
> way to systematically find at least the test scripts which access the
> internet, if not the specific tests within?
>
> Cheers,
> Sendu.
> _______________________________________________
> Bioperl-l mailing list
> Bioperl-l at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/bioperl-l

-- 
===========================================================
: Hilmar Lapp  -:-  Durham, NC  -:-  hlapp at gmx dot net :
===========================================================