[ale] ZFS on Linux

Mon Apr 1 07:14:46 EDT 2013

A gotcha on ZFS:
If it ever fills up a drive array, for instance the backup process starts
writing recursively, you can't remove anything, since it uses disk space to
write a cache of what you delete.  You have to copy the files to a larger
array and reformat your stuffed ZFS array.  Ask me how I know?

On Sat, Mar 30, 2013 at 1:49 AM, gcs8 <gcsviii at gmail.com> wrote:

> David, my whole rack takes under 6 amps to power, I am pretty good at
> hardware design..
>
>
> On Sat, Mar 30, 2013 at 12:11 AM, Jim Kinney <jim.kinney at gmail.com> wrote:
>
>> Heh, heh.
>>
>> Informatics research eats drive space like it's free.
>>
>> You should see the nifty new tape system that UPS delivered today.
>>
>> dual LTO6 with 48 slots. It will be managed by a head end connected over
>> 10G ethernet to second server with 2 arrays (1 has 15 2TB drive and the
>> other is 12 2TB drives.)
>>
>> The 16 4TB drive system actually has dual 2.5 SAS drives for OS drives
>> plus the main 16 and a honkin' LTO6 tape system with 24 slots. This
>> connected over a dedicated 10G ethernet to the main number cruncher with
>> serious CPU and RAM and x16 2TB drives. There's an option to add an
>> additional 12 drives in the back of the server.
>>
>> http://www.supermicro.com/products/chassis/3U/837/SC837E16-RJBOD1.cfm
>>
>>
>> On Fri, Mar 29, 2013 at 10:41 PM, dev null zero two <
>> dev.null.02 at gmail.com> wrote:
>>
>>> rainbow tables ofc :-)
>>>
>>> what sort of case are you using for that many hdds?
>>>
>>> Sent from my mobile. Please excuse the brevity, spelling, and
>>> punctuation.
>>> On Mar 29, 2013 10:36 PM, "David Tomaschik" <david at systemoverlord.com>
>>> wrote:
>>>
>>>> I can only imagine the power bill for 20-30 hard drives & all the
>>>> controllers and CPU needed to spin that all up.  (And unfortunately, I have
>>>> good guesses as to what its used for.)
>>>>
>>>>
>>>> On Fri, Mar 29, 2013 at 3:13 PM, gcs8 <gcsviii at gmail.com> wrote:
>>>>
>>>>> i run raidz2, if i could chang anything it would be adding SSD caching
>>>>> and dual SSD ZIL.
>>>>>
>>>>>
>>>>> On Fri, Mar 29, 2013 at 5:07 PM, Jim Kinney <jim.kinney at gmail.com>wrote:
>>>>>
>>>>>> That sounds encouraging. I'll have probably 2 arrays. One with 15 1TB
>>>>>> data and a 2nd with 16 4TB SAS2. thinking raidz with a spare drive for each.
>>>>>>  On Mar 29, 2013 4:43 PM, "gcs8" <gcsviii at gmail.com> wrote:
>>>>>>
>>>>>>> I run a 20 2TB disk array at home and have rather fond of ZFS, its
>>>>>>> been stable for me for ~3 years now.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Mar 29, 2013 at 3:10 PM, Ed Cashin <ecashin at noserose.net>wrote:
>>>>>>>
>>>>>>>> Thanks in advance for the notes you take.  I'm interested to hear
>>>>>>>> what you find.
>>>>>>>>
>>>>>>>> By the way, work sent me to a Nexenta course in 2010 where I
>>>>>>>> learned ZFS stuff from Richard Elling:
>>>>>>>>
>>>>>>>>   http://www.richardelling.com/
>>>>>>>>
>>>>>>>> ... and one thing that kept coming up is how users usually reach
>>>>>>>> for the RAID-Z options when stripes of mirrors would perform better and
>>>>>>>> offer more flexibility.  I thought that was notable.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Mar 29, 2013 at 2:44 PM, Jim Kinney <jim.kinney at gmail.com>wrote:
>>>>>>>>
>>>>>>>>> There was an update to this that just hit /.
>>>>>>>>>
>>>>>>>>> WooHoo!
>>>>>>>>>
>>>>>>>>> I have not used ZFS (minimal solaris experience) before but I have
>>>>>>>>> a situation where raidz looks like a very good choice. An external 16 bay
>>>>>>>>> JBOD box hanging off a SAS HBA looks like a prime candidate. The big issue
>>>>>>>>> for me is the support for native NFSv4 ACLs which solves some issues.
>>>>>>>>>
>>>>>>>>> As I go through this exercise, I'll keep some notes and maybe look
>>>>>>>>> at a presentation on doing this.
>>>>>>>>>
>>>>>>>>> hints, gotcha's, are very welcome.
>>>>>>>>>
>>>>>>>>> I did find a pdf of the zfsadmin.pdf
>>>>>>>>> http://pdfhome.org/zfsadmin.pdf
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> --
>>>>>>>>> James P. Kinney III
>>>>>>>>> *
>>>>>>>>> *Every time you stop a school, you will have to build a jail.
>>>>>>>>> What you gain at one end you lose at the other. It's like feeding a dog on
>>>>>>>>> his own tail. It won't fatten the dog.
>>>>>>>>> - Speech 11/23/1900 Mark Twain
>>>>>>>>> *
>>>>>>>>> http://electjimkinney.org
>>>>>>>>> http://heretothereideas.blogspot.com/
>>>>>>>>> *
>>>>>>>>> _______________________________________________
>>>>>>>>> Ale mailing list
>>>>>>>>> Ale at ale.org
>>>>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>   Ed Cashin <ecashin at noserose.net>
>>>>>>>>   http://noserose.net/e/
>>>>>>>>   http://www.coraid.com/
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Ale mailing list
>>>>>>>> Ale at ale.org
>>>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Charles Selfridge
>>>>>>>
>>>>>>> PBYC  IT director
>>>>>>>
>>>>>>> (404) 910-3409
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Ale mailing list
>>>>>>> Ale at ale.org
>>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Ale mailing list
>>>>>> Ale at ale.org
>>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>>> http://mail.ale.org/mailman/listinfo
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Charles Selfridge
>>>>>
>>>>> PBYC  IT director
>>>>>
>>>>> (404) 910-3409
>>>>>
>>>>> _______________________________________________
>>>>> Ale mailing list
>>>>> Ale at ale.org
>>>>> http://mail.ale.org/mailman/listinfo/ale
>>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>>> http://mail.ale.org/mailman/listinfo
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> David Tomaschik
>>>> OpenPGP: 0x5DEA789B
>>>> http://systemoverlord.com
>>>> david at systemoverlord.com
>>>>
>>>> _______________________________________________
>>>> Ale mailing list
>>>> Ale at ale.org
>>>> http://mail.ale.org/mailman/listinfo/ale
>>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>>> http://mail.ale.org/mailman/listinfo
>>>>
>>>>
>>> _______________________________________________
>>> Ale mailing list
>>> Ale at ale.org
>>> http://mail.ale.org/mailman/listinfo/ale
>>> See JOBS, ANNOUNCE and SCHOOLS lists at
>>> http://mail.ale.org/mailman/listinfo
>>>
>>>
>>
>>
>> --
>> --
>> James P. Kinney III
>> *
>> *Every time you stop a school, you will have to build a jail. What you
>> gain at one end you lose at the other. It's like feeding a dog on his own
>> tail. It won't fatten the dog.
>> - Speech 11/23/1900 Mark Twain
>> *
>> http://electjimkinney.org
>> http://heretothereideas.blogspot.com/
>> *
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>>
>>
>
>
> --
> Charles Selfridge
>
> PBYC  IT director
>
> (404) 910-3409
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>

-- 
Wolf Halton
This Apt Has Super Cow Powers - http://sourcefreedom.com
Open-Source Software in Libraries - http://FOSS4Lib.org
Advancing Libraries Together - http://LYRASIS.org
Apache Open Office Developer wolfhalton at apache.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130401/21f8258b/attachment-0001.html>