[Bioperl-l] Bioperl-run: Testing alignments generated externally

[Chris Fields]

...
> I think it makes sense to test that data of the expected type was
> returned by the xternal resource but not to test the specifics of what
> was retured. If specifics are tested we are then in the realm of testing
> whether we believe the data returned by the external resource or not. We
> should assume that the domain experts for these resources know what they
> are doing - in some cases this might not be true :-)  but I think we
> should stick to testing that the objects created hold the expected type
> of data.
> 
> I like what Chris had to say (above) but wonder whether tests
> would/should be tested for in the module itself - i.e. testing that a
> stored value is an integer and warn/throw if not?
> 
> Nath

Yeah, sorry about the top post (stupid Outlook always sticks the sig at the
top of the page!).  

Testing in the module would be best but can be tricky for the very same
reasons that writing tests entail, even more so.  For instance, for NCBI
esummary data, I parse the data in a very generic way in order to have
access to as much data as possible.  

For tests, I have to assume that NCBI will always return a particular type
of value (string, integer, date).  I can test for each of those with a regex
in the module fairly simply and throw/wanr, as you indicate.  However, if
they decide to add new data with a data tag other that the ones I test for
in the module (i.e. String, Integer, Date), I suddenly have warns/throws
showing up and cluttering/clobbering the code for perfectly valid data.  

However, if these are caught in tests and the tests fail, no big loss.  The
actual module still works, even if the tests are failing based on an new
unknown value being returned.  

For me, failed tests are sort of a warning light to let me know that
something has changed, but it doesn't necessarily mean a module doesn't
work.  I generally use throw/warn for something truly catastrophic, like no
response from the server or an error in the XML, which affects downstream
methods.  

Christopher Fields
Postdoctoral Researcher - Switzer Lab
Dept. of Biochemistry
University of Illinois Urbana-Champaign