[Fwd: Re: [Biopython-dev] Bio.CAPS module submission]

[Iddo Friedberg]

Jonathan Taylor floated an idea to add unit testing in addition to 
regression testing. Seems useful, and the change to setup.py is minor. 
There should be no changes to the test_MODULE.py files, or the current 
behavior of run_tests.py


Does anybody see any problem with this? I don't.

Best,

Iddo

-------- Original Message --------
Subject: 	Re: [Biopython-dev] Bio.CAPS module submission
Date: 	Tue, 15 Feb 2005 17:58:38 -0500
From: 	Jonathan Taylor <jonathan.taylor at utoronto.ca>
To: 	Iddo Friedberg <idoerg>
References: 
<Pine.SGI.4.10.10502142149040.21921303-100000 at pines2.ljcrf.edu> 
<421264EF.70903 at utoronto.ca> <421269FE.50407 at burnham.org>



Ah I see after looking what this is.  The regression tests provide an 
easy way to provide sanity tests.  I agree that simply having my 
unittests run and have the output "All tests ok" in output/test_CAPS is 
a little less than useful if a problem does arise.  Of course it does at 
least give warning if there is a problem.  So I spent the last little 
while batting around some ideas to allow biopython developers use both 
"regression tests" and regular unittest's.  Heres what I came up with.

By changing the testing procedure a bit as below to search each 
test_MODULE for a suite() method, we can allow developers aggregate some 
real unittests in their files by simply providing a suite() method. 
(SEE BELOW FOR A UNIFIED DIFF OF run_tests.py).

Thus I added:

def suite():
  suite = unittest.TestSuite()
  suite.addTest(UnevenAlignment("test"))
  return suite

to my test_CAPS.py file.  That way we get these tests run in addition to 
the standard regression test as below:

nullset at coder:~/projects/biopython/Tests $ python run_tests.py test_CAPS
test (test_CAPS.UnevenAlignment) ... ok
test_CAPS ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.054s

OK

Clearly I still need to add the rest of my tests to the suite() method. 
 What do you think of this idea?  Maybe I should post this to the list. 
 I am working against biopython cvs here so when we are all done and 
agreed I can send you a patch file to the main tree. ;)

Cheers.
Jon.

nullset at coder:~/projects/biopython/Tests $ cvs diff -u run_tests.py
Index: run_tests.py
===================================================================
RCS file: /home/repository/biopython/biopython/Tests/run_tests.py,v
retrieving revision 1.7
diff -u -r1.7 run_tests.py
--- run_tests.py        2 Jul 2003 15:04:18 -0000       1.7
+++ run_tests.py        15 Feb 2005 22:49:18 -0000
@@ -117,7 +117,12 @@
     test_suite = unittest.TestSuite()

     for test in all_tests:
-        # add the test to the test suite
+        # add any provided test cases
+        m = __import__(test)
+        if 'suite' in dir(m):
+          test_suite.addTest(m.suite())
+
+        # add the regression test to the test suite
         test_suite.addTest(RegressionTest(test))

     return test_suite
@@ -313,6 +318,7 @@
         line = line.replace(newline_to_replace, "\\n")

     return line
+

 if __name__ == "__main__":
     sys.exit(main(sys.argv))





Iddo Friedberg wrote:
> Jonathan Taylor wrote:
> 
>> The tests are mostly for developers though.  If the rest of biopython 
>> works on a users machine, then this will.  Maybe I am misunderstanding 
>> something.
>>
> 
> The regression test module, run_tests.py depends on some sort of output 
> comparison with a test file (in your case, test_CAPS.py). You can also 
> place your input in a file as well. Actually, the tests are run 
> (optionally) upon install by the users. We have discovered several bugs 
> in the past by having users run the tests and receiving different 
> outputs than expected, due to the vagaries of their OS and or Python 
> version. Also, it serves as a sanity check for the user. You can keep 
> the format of your internal testing, and just generate a file which says 
> "Tested OK". Although something more verbose would be nice. If, for some 
> reason, the module breaks on a user's machine, the test will fail.
> 
> Thanks,
> 
> Iddo
> 




-- 
Iddo Friedberg, Ph.D.
The Burnham Institute
10901 N. Torrey Pines Rd.
La Jolla, CA 92037 USA
Tel: +1 (858) 646 3100 x3516
Fax: +1 (858) 713 9930
http://ffas.ljcrf.edu/~iddo
==========================
The First Automated Protein Function Prediction SIG
Detroit, MI June 24, 2005
http://ffas.burnham.org/AFP