[Bioperl-l] Bioperl-run: Testing alignments generated externally

[Nathan S. Haigh]

Chris Fields wrote:
> I have been running into similar issues with EUtilities tests.  Since the
> data on the server is constantly updated I have to try an future-proof the
> tests so they don't constantly fail.  
>
> I have been using Test::More and like/unlike or cmp_ok to get around some of
> those 'fuzzy data' issues.  If some methods consistently return a particular
> type of value, such as an integer, you could use:
>
> like($foo->get_value, qr{^\d+$}, 'value test'); #integer
>
> or similar.
>
> Chris
>
> Christopher Fields
> Postdoctoral Researcher - Switzer Lab
> Dept. of Biochemistry
> University of Illinois Urbana-Champaign 
>
>   
>> Nathan -
>>
>> I agree - the values tend to change with different versions of the
>> applications unfortunately.  It would make sense to just test that
>> you get out sequences that are in valid alignment format and perhaps
>> have as many ending sequences as you started with.   The more
>> restrictive tests probably aren't reliable with mixing and matching
>> versions.
>>
>> One thing we do for PAML is condition tests on the version used - but
>> of course when a new version comes out we have to add more stuff to
>> the tests (or just have some code that skips those tests).
>>
>> -jason
>> On Oct 26, 2006, at 3:33 AM, Nathan Haigh wrote:
>>
>>     
I think it makes sense to test that data of the expected type was
returned by the xternal resource but not to test the specifics of what
was retured. If specifics are tested we are then in the realm of testing
whether we believe the data returned by the external resource or not. We
should assume that the domain experts for these resources know what they
are doing - in some cases this might not be true :-)  but I think we
should stick to testing that the objects created hold the expected type
of data.

I like what Chris had to say (above) but wonder whether tests
would/should be tested for in the module itself - i.e. testing that a
stored value is an integer and warn/throw if not?

Nath