[Biojava-l] file i/o with ArrayList

[Khalil El Mazouari]

Hi Stefan,

I recently had a similar problem. Object serialisation was OK. However, deserialization of huge sequence list may consume a lot of memory => less performance ... 

I highly recommend to take a look at ChronicleMap (CM): A low latency Key Value Store, with consistency, persistence and performance. Data can be stored and reloaded from disk, in a single file. You can also use CM as off-heap store.

With CM we managed to persist, load and process 1.000.000 sequences with just 4G RAM.  

Best Regards,

Khalil

On 30 Jan 2015, at 13:00, biojava-l-request at mailman.open-bio.org wrote:

> Today's Topics:
> 
>   1. file i/o with ArrayList (stefan harjes)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Fri, 30 Jan 2015 11:01:33 +0000 (UTC)
> From: stefan harjes <stefanharjes at yahoo.de>
> To: "biojava-l at mailman.open-bio.org" <biojava-l at mailman.open-bio.org>
> Subject: [Biojava-l] file i/o with ArrayList
> Message-ID:
> 	<1067364125.3039846.1422615693955.JavaMail.yahoo at mail.yahoo.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Hi biojava-l
> 
> 
> I have a huge number of small sequences in an Array (ListArray<Sequence<?>>) which for server start and stop I would like to store on disk. Unfortunately Sequence is not serilizable, so I searched and found that GenbankWriterHelper.writeSequences(OutputStream os, Collection<Sequence<?>> seqs) should be able to do the job. 
> However when looking at GenbankReaderHelper, there are no methods which correspond to the above writer method. Am I on the wrong track completely? 
> 
> When looking at the writer/reader helpers, I think I remember reading that they are rudimentary and save only the sequence (fasta)? I would expect in such an advanced verision of biojava (4.0 is being prepared?) that there must be a standard way to serialize rich sequences/arrays of them in order to send them around on streams/Json etc?
> 
> Any help would be appreciated
> CheersStefan
> 
> 
> 
> 
> 
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://mailman.open-bio.org/pipermail/biojava-l/attachments/20150130/7b1260cf/attachment-0001.html>
> 
> ------------------------------
> 
> _______________________________________________
> Biojava-l mailing list  -  Biojava-l at mailman.open-bio.org
> http://mailman.open-bio.org/mailman/listinfo/biojava-l
> 
> End of Biojava-l Digest, Vol 143, Issue 2
> *****************************************


-----
Confidentiality Notice: This e-mail and any files transmitted with it are private and confidential and are solely for the use of the addressee. It may contain material which is legally privileged. If you are not the addressee or the person responsible for delivering to the addressee, please notify that you have received this e-mail in error and that any use of it is strictly prohibited. It would be helpful if you could notify the author by replying to it.