[Bioperl-pipeline] multiple pipelines

[Elia Stupka]

> Yes, these ideas seem great. I would personally put in a vote for at
> least having  pure cgi as an option (as opposed to only having a Java
> based client for example).

You hit home ground with this, since we are a strong perl-shop. We are 
thinking of going SOAP/XML in terms of the actual protocol to ship data 
between client and server, and then we can implement CGI/Perl 
shell/applet on top of that.

> But, for now, how safe is it to run two pipelines at once?

See Shawn's answer, they will not get mixed up, no worries. We 
currently do it quite often.

> I don't know if you have any numbers or not, but I wonder what the 
> approximate percent speed gain is from doing this... any idea? That is 
> obviously a very aggressive setup... the type of setup I would expect 
> on a heavily used/publicly accessible resource.

I can't get you numbers off the top of my head, but basically you are 
killing BLAST if you read the database from a remote NFS mounted 
location.

About the agressive setup.... it is high-performance, no doubt, but not 
particularly expensive. You can buy pretty cheap processors, buy EIDE 
disks rather than SCSI, and achieve this "aggressive" setup, while 
often newbies dish out money on 3Ghz Xeons and then put in a single, 
tiny, SCSI hard disk. Ensembl, for example, runs on Blades running 
Celerons 800Mhz with mirrored EIDE disks, cheap, and fantastic 
performance. Of course the fact of distributing and mirroring databases 
only makes sense if you have at least a few nodes, though one could 
advocate that as soon as you have more than 2 processors accessing that 
data you should do it. The other option is having a dedicated SAN or 
NAS though it will never beat local access (which is still the cheapest 
solution).

Elia

---
Bioinformatics Program Manager
Temasek Life Sciences Laboratory
1, Research Link
Singapore 117604
Tel. +65 6874 4945
Fax. +65 6872 7007