[BioPython] Biopython object serialization

[Patrick K. O'Brien]

On Tuesday 12 November 2002 04:29 am, Estienne Swart wrote:
> I recently came across an article about object persistence
> (http://www-106.ibm.com/developerworks/linux/library/l-pypers.html),
> by Patrick O'Brien (the name should ring a bell to those of you that
> read his O'Reilly article on Bioinformatics). He advocates the use of
> his own solution to persistence,
> PyPerSyst <http://sourceforge.net/projects/pypersyst/>, which is
> supposedly faster than ZODB, and simpler to implement too.

Thanks for mentioning the article. I hope you liked it.

I think a couple of caveats are in order concerning the PyPerSyst 
approach. First, there is only a basic implementation available right 
now. I think it needs more layers of abstraction to be truly useful to 
support persistence in a completely robust and transparent fashion. 
That will take a while. I'm a perfectionist and I want to do it right.

Second, the entire approach relies on keeping all objects in memory. 
That's why it is faster and simpler than a database and ZODB. Once you 
decide you need to support more objects than can fit in RAM, you need 
to add quite a lot of overhead to check if an object is in memory and 
what state it is in, swapping objects to and from disk and various 
states, etc. All those mechanisms are necessary but also add complexity 
and slow things down. That's why ZODB is bigger and slower.

So while I am an advocate of the PyPerSyst/Prevayler approach, I'm also 
targeting applications that can fit within the primary constraint of 
physical RAM, and Python developers who recognize that this is a new 
project that doesn't yet come with all the bells and whistles that we'd 
like it to have some day. Of course, how long that takes depends in 
part on getting other people involved with the project, so I have to do 
some amount of promotion.

I hope that helps clarify where PyPerSyst might be useful and where it 
might not.

-- 
Patrick K. O'Brien
Orbtech      http://www.orbtech.com/web/pobrien
-----------------------------------------------
"Your source for Python programming expertise."
-----------------------------------------------