[ale] distributed network file system

[Ed Cashin]

Robert, if you really meant to spell it the way you did, I apologize and
humbly request a link to the project so that I can learn more about it.
Otherwise...

Not really a nit but just a pointer for folks who want to google for RLH's
nice suggestion: I always mix up the letters in DRBD, but it helps to
remember that the last two stand for "block device".

I think it's worth mentioning that in addition to distributed filesystems,
there have been a lot of new systems that are more like big storage
containers, where you look up a value by providing a key.  They came out of
the distributed hash table research in the early 2000s, so you can find
them by following the DHT links on wikipedia.  If you don't need full
filesystem semantics and you want the number of nodes and locations to grow
a lot, they might be interesting.

On Tue, Jun 26, 2012 at 1:35 PM, Robert L. Harris <robert.l.harris at gmail.com
> wrote:

>
> Check out DRDB.  I'm using it to sync two 750G filesystems and it works
> spectacularly.  You should be able to do multiple nodes syncing 1
> filesystem and it just generates additional mirrors.
>
> Robert
>
>
> On Tue, Jun 26, 2012 at 11:24 AM, John Heim <john at johnheim.net> wrote:
>
>> From: "Derek Atkins" <warlord at MIT.EDU>
>> To: "Atlanta Linux Enthusiasts" <ale at ale.org>
>> Sent: Tuesday, June 26, 2012 8:04 AM
>> Subject: Re: [ale] distributed network file system
>>
>>
>> > Hi,
>> >
>> > "John Heim" <john at johnheim.net> writes:
>> >
>> >> I would like to set up a distributed network file system in my
>> >> department.
>> >> There is a dizzying array of possibilities, gfarmfs, ceph, glusterfs,
>> >> just
>> >> to name a few.
>> >>
>> >> Needs:
>> >> 1. Should work on a large number of small nodes, 100Gb each.
>> >> 2. Parallelism & striping.
>> >> 3. Prefer debian package, GPL.
>> >> 4. Meta data in mysql would be nice.
>> >>
>> >> Any experience and/or recommendations?
>> >
>> > What are your requirements for usage of the space?  Are you trying to
>> > get a distributed SAN array?  Or are you just trying to get a
>> > distributed file space?
>> >
>> > If the latter you might also want to look at OpenAFS.  It is F/OSS,
>> > although it's not GPL.  Oh, and the metadata isn't stored in MySQL.
>>
>> We have a 2Tb SAN for users to use for files space. But we have about 300
>> users so each gets only 6Gb.  That's just not enough for some users.
>> Mostly,
>> its enough on a long term basis but sometimes they need to generate 50Gb -
>> 100Gb of data. We have all kinds of disk space on each users workstation
>> but
>> they can't get to it. This is deliberate. We don't want them saving files
>> where they won't be backed up.  And we want to be able to re-image a
>> machine
>> at a moment's notice w/o having to have the user back up his stuff.
>>
>> I got the brilliant idea of using the 100Gb (or so) of free space on each
>> workstation for a distributed network file system.  So we'd need to be
>> able
>> to wipe out a node w/o losing anything.  I could make sure we copy the
>> data
>> off before we re-image a workstation. But an end-user might simply turn
>> their workstation off. Whatever we use would have to deal with that.
>>
>> The mysql thing was just a preference (over postgres). I have nothing
>> against other DBMSes. Its just that we already have mysql.
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>>
>
>
>
> --
> :wq!
> ---------------------------------------------------------------------------
> Robert L. Harris
>
> DISCLAIMER:
>       These are MY OPINIONS             With Dreams To Be A King,
>        ALONE.  I speak for                      First One Should Be A Man
>        no-one else.                                     - Manowar
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>


-- 
  Ed Cashin <ecashin at noserose.net>
  http://noserose.net/e/
  http://www.coraid.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.ale.org/pipermail/ale/attachments/20120626/193b84e0/attachment.html