[MOBY-dev] RFC - Synchronization of Biomoby secondary repositories

[Andreas Groscurth]

>The amount of RSS-RDF  
>we would have to maintain on MOBY Central in order to have a complete  
>history that would allow a mirror to reliably re-construct the current  
>state of the database is... well... large!  At the moment, I keep only the  
>last... 100?... changes.  If you don't pick-up the feed for a day, or if  
>someone registers 1000 new services, you wont see them in the feed.  
You are definitely right, if a secondary fails to synchronize with the central 
for a long time the number of changes are enormously large, but therefore the 
time stamp plays its role.
Each secondary stores the time stamp of its last synchronization - furthermore 
in the RSS the latest 30 / 100 (or whatever) changes are written also with 
the time stamp of the actual change. So with each RSS fetching, the secondary 
is able to determine whether it is out of sync.
And if so, the secondary calls the central to retrieve a specific RSS for the 
changes it lacks. So this is independent from the 'normal' RSS. 
Also it is discussable whether to force the secondary to fetch the RSS feed at 
least once a day or any other frequency. So one can shorten that frequency 
but not extend it.

I'm not quite sure whether i said something completely inappropriate according  
your concerns - Heiko will clear that out I guess if so ;-)

Cheers
Andreas