[Bio-packaging] Distributed file system and experimental cluster

[Pjotr Prins]

On Wed, Feb 10, 2016 at 10:47:38AM +0100, Ricardo Wurmus wrote:
> 
> Pjotr Prins <pjotr.public66 at thebird.nl> writes:
> 
> > Anyone experience with distributed file systems, such as moosefs,
> > glusterfs etc.? I want to try something reasonably easy to install and
> > manage and has striping or chunking.
> 
> What do you want to achieve with a distributed file system that could
> not be achieved with centralised storage?  In the configurations that I
> worked with distributed file systems are only used for volatile scratch
> space.  Everything else (e.g. user software, home directories) uses
> either centralised storage or is node-local (e.g. the centrally managed
> system state).
> 
> That said, I only ever worked on distributed file systems as a user.  I
> was never in charge of setting up a distributed file system.

With large data and many nodes the central storage quickly is the IO
bottleneck on a cluster. At least in more conventional designs.

Pj.

--