[MOBY-dev] Dead services and how to figure them out

Sergio Ramirez Ramirez srramirez at uma.es
Mon Nov 10 16:54:32 UTC 2008 

Hi Eddie,

The idea is to define a list of status for the services, direct 
accessible from the list of them, that represents the results of the 
tests make over it.
If there aren't samples available, the only test that can be done is a 
ping to see is the service responds. In this case the status proposed are:

    * online: The service responds properly to the ping 
    * Timeout: If the time limit is got when trying to connect to the
      service.
    * NotFound: The service url is not found in the server
    * InternalServiceError: The service responds with an error related
      with the implementation of the service.
    * Disconnected: The connection with the service couldn't be established.

In addition, the NonAvailable status represents those services that have 
been down manually by the service provider for performing administrative 
tasks; and the unknown status when cannot be determine the status of the 
service.

When an example was available (defined in the RDF or in the extended 
registry of the INB, etc) can be make more specific test to the service. 
In this case, the set that we proposed is:

    * Tested: The service has been tested with the inputs given and no
      errors have been reported.
    * BadSample: The service respond with errors related with the input
      given.
    * NoSample: There aren't example for testing the service.

The output can be also used for test the behavior of the service. In 
this case we discussed two ways for testing:

    * TypeCorrect: The type of the data returned is the same or
      compatible with the expected one.
    * TypeIncorrect: If the data type is not compatible with the
      expected one.

And if  a way is provided for check the output content (regular 
expression or xpath) them there are two more status to contemplate:

    * OutputTested: The output content is check and fit with the results
      expected.
    * OutputFailed: in other case.

As you can see Eddie, the proposal is compatible with the one you 
mention. The only difference is the way to query the status of the 
service, that know can  be accessible using the functions of the MOBY 
Central..

Edward Kawas wrote:
> Hello,
>
> I wanted to chime in with my 2 cents!
>
> Back in February during the Biohackathon, we (Canadian/Spanish/German
> groups) came up with some predicates that we could use to add unit testing
> information to a service[1]. 
>
> Currently, JAVA can support parsing the RDF to get this information as well
> as comparing the unit test information with the output for the service. I
> have yet to add Perl support.
>
> In addition, I have been working on a Dashboard panel that will help users
> fill in unit test information for a particular service. In addition to
> filling in the information, a service user can test the selected service and
> save the unit test (in the service RDF).
>
> Is the current proposal for testing services based on what we came up with
> in February?
>
> Thanks,
>
> Eddie
>
> [1] http://lists.open-bio.org/pipermail/moby-dev/2008-June/005061.html
>
>
> -----Original Message-----
> From: moby-dev-bounces at lists.open-bio.org
> [mailto:moby-dev-bounces at lists.open-bio.org] On Behalf Of Sergio Ramirez
> Ramirez
> Sent: November-07-08 6:12 AM
> To: Core developer announcements
> Subject: Re: [MOBY-dev] Dead services and how to figure them out
>
> Hello all,
>
> The only difference of the proposal with the list 
> "http://moby.ucalgary.ca/moby/ValidateService"  is to include the 
> information of this list in the service repository so it can be accessed 
> using the API of MOBY.
> It also provides the advantage of filter those services that haven't 
> been validate when the user ask for a list of services.
>
> When the service hasn't and example we only can test is the service is 
> answering or not, so that is what means "work properly". In the INB we 
> have examples associated with the services so is possible to test the 
> services with more detail.
>
> For non-sample services the status could be: Online, Timeout is the 
> service is not answering and BadName when the service name is wrong. 
> When an example was available is possible to check if the example is 
> correct and the service doesn't returns with errors;  if the output type 
> is the expected and/or if the output itself is correct using some 
> mechanism to compare it with the one registered as example.
>
> This is the new proposal in big details, but the main idea is to have 
> the information that is now in the list available directly for clients.
>
> Best regards,
> Sergio.
>
>
> Sebastien Carrere wrote:
>   
>> Hi,
>>
>> my list contains the DeadServices provided on biomoby.org but also 
>> services where the following Perl test fails:
>>
>> eval {$service_instance->execute(XMLinputlist =>[[]])};
>>
>> The most frequent error is "Service execution failed: 500 Can't 
>> connect to localhost:8080 (connect: Connection refused)"
>>
>> I don't know what is exactly done by 
>> http://moby.ucalgary.ca/moby/ValidateService .
>>
>> Nevertheless, if we want to test if a webservice "works properly", we 
>> should first ask service developpers to register with test data.
>> This is already what we do using the PlayMoby framework.
>> By this way we survey our services ('not tested', 'ok', 'down') and 
>> generate a daily report.
>>
>>
>> Sebastien
>>
>>
>>
>>
>> groscurt wrote:
>>     
>>> Mhm i missed the intial email to that respond.
>>>
>>> so i'm wondering - what is different from your list to the list from 
>>> the one
>>> from mobycentral http://moby.ucalgary.ca/moby/ValidateService ?
>>> And to the discussion about "working proper" services - it might be 
>>> in the
>>> document you will see, Sergio, but i'm really curios about the 
>>> definition of
>>> the phrase "work properly". So when does a service work properly ?
>>> of course, in my point of view, if it fulfills its registrated 
>>> defintion -
>>> so the question is how to test that ?
>>> I guess i wait for the document sergio was talking about ;-)
>>>
>>> Cheers
>>> andreas
>>>
>>> ----------------original message-----------------
>>> From: "Pierre-Yves Chibon" To: "Core developer announcements" 
>>> moby-dev at lists.open-bio.org
>>> Date: Thu, 06 Nov 2008 17:28:34 +0100
>>> -------------------------------------------------
>>>  
>>>       
>>>> Sebastien Carrere wrote:
>>>>    
>>>>         
>>>>> Here is my list of Dead services (Ihave got also list for the 3 
>>>>> past days for testRegistry and Inab registry) :
>>>>>
>>>>>       
>>>>>           
> http://lipm-bioinfo.toulouse.inra.fr/remora/sessions/blacklist.mobycentral 
>   
>>>  
>>>       
>>>> Just by curiosity :
>>>>   - do you have a list of working services ?
>>>>   - do you have available your results from the testing repository ? 
>>>> (by the way, why are you testing it ?)
>>>>
>>>> Thanks in advance,
>>>>
>>>> Regards,
>>>>
>>>> Pierre
>>>> _______________________________________________
>>>> MOBY-dev mailing list
>>>> MOBY-dev at lists.open-bio.org
>>>> http://lists.open-bio.org/mailman/listinfo/moby-dev
>>>>
>>>>     
>>>>         
>>>
>>> _______________________________________________
>>> MOBY-dev mailing list
>>> MOBY-dev at lists.open-bio.org
>>> http://lists.open-bio.org/mailman/listinfo/moby-dev
>>>   
>>>       
>> _______________________________________________
>> MOBY-dev mailing list
>> MOBY-dev at lists.open-bio.org
>> http://lists.open-bio.org/mailman/listinfo/moby-dev
>>   
>>     
>
>
>   


-- 
Sergio Ramírez Ramírez
Instituto Nacional de Bioinformática (INB)
Integrated Bioinformatics Node (GNV-5)
Dpto. de Arquitectura de Computadores
Campus Universitario de Teatinos, despacho 2.3.9a
29071 Málaga (Spain) +34 95 213 3387


"Short-term decisions tend to fail in the long-term."
            Frank Herbert, God Emperor of Dune

More information about the MOBY-dev mailing list