[Bioperl-l] Smoke test aggregator - Buildbot + Smolder ?

[Jay Hannah]

On May 5, 2010, at 4:40 PM, Dave Messina wrote:
> Very few of BioPerl's tests rely on outside servers, and those that do have to be turned on explicitly with a network-tests flag. So hopefully that won't be an issue.

I said "no" to the network tests for my smoke runs. Haven't really examined the results enough to know if the failures are my fault or what. Since I always use bioperl-live out of SVN (soon git) I may not be following the ./Build.PL procedure correctly.

> Agreed, not really. Except for some of the GMOD stuff. And anyway this could always be done later if desired. Probably much later. :)

Ya. Some day http://smolder.open-bio.org hosting jillions of projects would be dreamy!   :)

Any open-bio.org projects using TAP other than BioPerl? Smolder can host anything TAP, and TAP producers are available in at least 17 languages:

   http://testanything.org/wiki/index.php/TAP_Producers

> Would the reporter need to have any special setup to do this?

LWP::UserAgent or Smolder's smolder_smoke_signal are the two methods I've successfully executed so far:

   http://jays.net/wiki/Smolder

> Could this kind of reporting be written into the BioPerl Build.PL as a user-settable option (just like the options for installing scripts or running network tests)?
> 
> If so, then we could get lots of feedback on trunk (master) commits and not just releases.

Ya, wow. I've never built BioPerl "the right way" (I'm an SVN/git junkie) so I'm not sure how this would get put into Build.PL. Would you prompt the user, something like "Since you just installed BioPerl, we'd like to connect to the Internet and report in your test results. Is this ok? [yes] "  ?

It would be very cool to collect and trend thousands of reports, assuming it can be 100% automated for the user.

Thanks for the feedback!  :)   Time to putter my motorcycle home before it gets too cold.  G'night,

Jay Hannah
http://biodoc.ist.unomaha.edu/wiki/User:Jhannah