[Bioperl-l] Tests involving remote databases

[Chris Fields]

On Oct 2, 2006, at 4:40 AM, Sendu Bala wrote:

> Chris Fields wrote:
>>
>> The idea is to retain current behavior (remote DB access will not be
>> run unless BIOPERLDEBUG is set to 1) and apply it to all tests
>> requiring such access.  Otherwise, just those tests are skipped (and
>> not the rest of the tests, which occurs currently).  If BIOPERLDEBUG
>> is set, the next tests would check the URL, which passes/fails (based
>> on the specific value of $@), and runs/skips tests based on the mere
>> presence of $@, which indicates some URL issue.  You can do this with
>> Test::More, but I'm not sure this can be done with Test.pm or
>> Test::Simple.
>
> Firstly, BIOPERLDEBUG should not be abused; it should be used only  
> when
> you want to see extra debugging messages. There should be another
> variable that you can set to choose if network-requiring tests are  
> run,
> and it should also be a configurable choice when you run perl  
> Makefile.PL.
>
> (But changing this isn't going to happen for 1.5.2)
>
> When the server problem is ambiguous we should not fail the test. Just
> make the skip message visible and pass all ok...

I agree, as well as with your assessment of BIOPERLDEBUG (which I  
alluded to in a previous post).  Torsten suggested creating a new  
env. variable for network tests.

It's obvious this won't be done before 1.5.2, but we can make plans  
towards the next release.

>> The current behavior just skips all tests based on a single failed
>> URL.  Then, Test::Harness, as currently set, shows skipped tests as
>> passed.  The last run I posted previously where XEMBL_DB.t remote DB
>> tests failed, I also ran all tests (make test) and get this, which
>> doesn't tell us that the remote URL failed:
>>
>> -----------------------------------------
>>
>> ...
>> t/WABA.......................ok
>> t/XEMBL_DB...................ok
>> t/ztr........................Bio::SeqIO::staden::read of bioperl-ext
>> is not installed or is installed incorrectly - skipping ztr.t tests
>> ok
>> All tests successful, 5 subtests skipped.
>
> All you have to do to make it visible is start the skip message  
> with the
> work 'Skip':
>
> skip('Skip server may be down',1);
>
> ...
> t/WABA.......................ok
>
> t/XEMBL_DB...................ok
>
>          1/9 skipped: server may be down
> t/ztr........................Bio::SeqIO::staden::read of bioperl- 
> ext is
> not installed or is installed incorrectly - skipping ztr.t tests
> t/ztr........................ok
>
>
> It's nicer when using Test::More.

Okay, if Test::Harness picks that up it would be okay.  We could use  
skip blocks to skip subsets of tests that require remote access (like  
SeqFeature.t) as opposed to skipping all tests.

I think we want to avoid promoting running tests with BIOPERLDEBUG  
(or similar) upon installation for everyday installation anyway (such  
as from CPAN, which Hilmar points out).  It's not something everybody  
installing a new BioPerl should be running unless they run into  
problems.

Christopher Fields
Postdoctoral Researcher
Lab of Dr. Robert Switzer
Dept of Biochemistry
University of Illinois Urbana-Champaign