[clug] Cloud servers, ZFS, SnapRAID+MergerFS (was Re: Topic for Feb meeting)

Tue Feb 21 05:41:18 UTC 2017

Define 'cloud'.  Do you mean compute+storage for home?  I'm at the 
burn-in test stage for a new home server for just this. Indulge me...

My requirement is for a robust NAS that can grow (lots of disk bays), 
but could also have enough grunt for the home network, some general 
web-ish services, and some VMs.

I bought a Dell C6105 cloud server off eBay.  It is actually three 
servers in one, and I believe is used by cloud providers to use or sell 
as cloud services.  In 2RU I got 144GB of RAM, 6 x 6-core Opteron 2419 
processors, and 12 x 1TB 3.5" SATA drives.

Each separate server has 1/3 of these, so 48GB RAM, 2x CPUs and 4x 
drives, all x 3.

Power is non-trivial.  There's a single PSU that powers the lot. Zero 
cards on = 50W of idle power draw.  Turn one processor card on and it 
takes a total of 190-250W depending on how hard the CPUs are pushed.  
That's more than I would like, but probably less power than the 
equivalent Xeon rack servers of their age.  Fully running, it's about 
400W, but I don't anticipate running more than one server card 
regularly, unless I need to heat the garage.

Price was compelling.  The nearest I could price a new JBOD chassis plus 
decent RAM, CPU, PSU, motherboard was north of $2500, and then disks on 
top of that.  I got the whole server including the disks, delivered from 
the US, for about $1200.  That justifies extra power consumption for me.

I will use Proxmox for virtualisation, giving me LXC and KVM.

I was all set for using ZFS, either through a virtualised FreeNAS 
instance, or natively on Debian, but am now seriously considering 
SnapRAID + MergerFS.  See here: 
https://www.linuxserver.io/2016/02/02/the-perfect-media-server-2016/ .  
For home use, this seems like a good compromise.  For me this is where 
it gets interesting.

ZFS is enterprise-grade, but does not grow easily.  You can't add a 
single drive and have it restripe like you can with MDADM.  To grow your 
ZFS pool once you outsize it, you have two options: add a whole new vdev 
(another bunch of disks, requiring their own bays etc), or you replace 
each disk one by one, and once you have replaced them all, by ZFS magic 
you get the extra disk space.  I was looking at 8 disks in my RAID 
array, and so when the time came to upgrade, I would have to buy a new 
batch of 8 disks.  No point in anything less.

But SnapRAID and MergerFS allow me to use mismatched drives, and to add 
and remove disks at will (with some caveats and careful handling).  The 
main downside I see is that when you restore, you lose file ownership 
and permissions.  This is a grievous oversight IMO, but I should be able 
to work around it by only really storing media files on there, where 
permissions are less of an issue.

I'll do some testing with SnapRAID+MergerFS and get a feel for it.

Tony

On 21-Feb-17 3:37 PM, jhock at iinet.net.au wrote:
> Just for self interest, if anyone wanted to show me how to set up an open source cloud system, on a Linux system like Ubuntu, that allowed me to backup my entire file storage on my Android phone, wife's PC and my Xubuntu netbook, via my router's wifi, I would definitely like that to be a CLUG topic. But that's my self interest. :--)
>
> Oh, and the cloud could be a Raspberry Pi with an attached 2Tb hard disk plugged into the router to reduce the power supply, would also be perfect. :--)
>
> John.
>
> On 21 February 2017 3:17:53 PM AEDT, Steven Hanley <sjh at svana.org> wrote:
>> On Sun, Feb 05, 2017 at 08:51:52AM +1100, Stephen Hocking wrote:
>>> This part of the group would be, which reminds me...
>>
>> An oh look it is CLUG week, if you have a talk that does sound like a
>> pretty
>> cool topic for everyone to learn about.
>>
>> 	See You
>> 	    Steve
>>
>>> On 5 February 2017 at 00:53, Brenton Ross <rossb at fwi.net.au> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I have just finished putting together a presentation on testing for
>> my
>>>> VICI project.
>>>>
>>>> It covers a testing library for unit tests, asynchronous testing
>> for
>>>> threads and child processes, testing of modules and programs and
>>>> simulating user interactions with GUI programs.
>>>>
>>>> Would the group be interested ?
>>>>
>>>> Brenton
>>>>
>>>>
>>>>
>>>> --
>>>> linux mailing list
>>>> linux at lists.samba.org
>>>> https://lists.samba.org/mailman/listinfo/linux
>>>>
>>>
>>>
>>> -- 
>>>
>>>    "I and the public know
>>>    what all schoolchildren learn
>>>    Those to whom evil is done
>>>    Do evil in return"		W.H. Auden, "September 1, 1939"
>>> -- 
>>> linux mailing list
>>> linux at lists.samba.org
>>> https://lists.samba.org/mailman/listinfo/linux
>> -- 
>> Steven Hanley sjh at svana.org http://svana.org/sjh/diary
>> i can see her features begin to blur, as she pours herself
>> into the mold he made for her, and for everything he does
>> she has a way to rationalize
>>    Fixing Her Hair - Imperfectly - Ani
>>
>> -- 
>> linux mailing list
>> linux at lists.samba.org
>> https://lists.samba.org/mailman/listinfo/linux