[Bio-packaging] Distributed file system and experimental cluster

[Francesco Strozzi]

Hi Pjotr,
I've used GlusterFS in the past and I think it's reasonably simple to
install and configure for testing purposes. We had nice experiences with
GlusterFS in mirroring mode on production systems. We tested also the
striping mode although in that case we saw some bad behaviour from the
server which resulted in files becoming not accessible after some time. But
that was ~2 years ago so hopefully those bugs should be solved now.

Cheers
Francesco

On Wed, 10 Feb 2016 at 13:37 Pjotr Prins <pjotr.public66 at thebird.nl> wrote:

> On Wed, Feb 10, 2016 at 10:47:38AM +0100, Ricardo Wurmus wrote:
> >
> > Pjotr Prins <pjotr.public66 at thebird.nl> writes:
> >
> > > Anyone experience with distributed file systems, such as moosefs,
> > > glusterfs etc.? I want to try something reasonably easy to install and
> > > manage and has striping or chunking.
> >
> > What do you want to achieve with a distributed file system that could
> > not be achieved with centralised storage?  In the configurations that I
> > worked with distributed file systems are only used for volatile scratch
> > space.  Everything else (e.g. user software, home directories) uses
> > either centralised storage or is node-local (e.g. the centrally managed
> > system state).
> >
> > That said, I only ever worked on distributed file systems as a user.  I
> > was never in charge of setting up a distributed file system.
>
> With large data and many nodes the central storage quickly is the IO
> bottleneck on a cluster. At least in more conventional designs.
>
> Pj.
>
> --
> _______________________________________________
> bio-packaging mailing list
> bio-packaging at lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/bio-packaging
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.open-bio.org/pipermail/bio-packaging/attachments/20160210/3cea6355/attachment.html>