[clug] Cloud servers, ZFS, SnapRAID+MergerFS (was Re: Topic for Feb meeting)
jhock at iinet.net.au
jhock at iinet.net.au
Tue Feb 21 06:50:13 UTC 2017
Wow. Lots of technical stuff there that has gone over my head. Hence, the ask for help in probably a more simple system.
It all started when I received a Western Digital Technologies (WDT) Mycloud, 4Tb hardware. I did manage to set it up but it didn't have a client for Linux, it only backed up media, it wasn't encrypted and it was supposed to send my data to WDT which I definitely don't want. I want my data in my hands. Anyway, I took it back to the shop and got my money back. I thought there must be something like this using Linux. Hence my post.
On 21 February 2017 4:41:18 PM AEDT, Tony Lewis <tony at lewistribe.com> wrote:
>Define 'cloud'. Do you mean compute+storage for home? I'm at the
>burn-in test stage for a new home server for just this. Indulge me...
>My requirement is for a robust NAS that can grow (lots of disk bays),
>but could also have enough grunt for the home network, some general
>web-ish services, and some VMs.
>I bought a Dell C6105 cloud server off eBay. It is actually three
>servers in one, and I believe is used by cloud providers to use or sell
>as cloud services. In 2RU I got 144GB of RAM, 6 x 6-core Opteron 2419
>processors, and 12 x 1TB 3.5" SATA drives.
>Each separate server has 1/3 of these, so 48GB RAM, 2x CPUs and 4x
>drives, all x 3.
>Power is non-trivial. There's a single PSU that powers the lot. Zero
>cards on = 50W of idle power draw. Turn one processor card on and it
>takes a total of 190-250W depending on how hard the CPUs are pushed.
>That's more than I would like, but probably less power than the
>equivalent Xeon rack servers of their age. Fully running, it's about
>400W, but I don't anticipate running more than one server card
>regularly, unless I need to heat the garage.
>Price was compelling. The nearest I could price a new JBOD chassis
>decent RAM, CPU, PSU, motherboard was north of $2500, and then disks on
>top of that. I got the whole server including the disks, delivered
>the US, for about $1200. That justifies extra power consumption for
>I will use Proxmox for virtualisation, giving me LXC and KVM.
>I was all set for using ZFS, either through a virtualised FreeNAS
>instance, or natively on Debian, but am now seriously considering
>SnapRAID + MergerFS. See here:
>For home use, this seems like a good compromise. For me this is where
>it gets interesting.
>ZFS is enterprise-grade, but does not grow easily. You can't add a
>single drive and have it restripe like you can with MDADM. To grow
>ZFS pool once you outsize it, you have two options: add a whole new
>(another bunch of disks, requiring their own bays etc), or you replace
>each disk one by one, and once you have replaced them all, by ZFS magic
>you get the extra disk space. I was looking at 8 disks in my RAID
>array, and so when the time came to upgrade, I would have to buy a new
>batch of 8 disks. No point in anything less.
>But SnapRAID and MergerFS allow me to use mismatched drives, and to add
>and remove disks at will (with some caveats and careful handling). The
>main downside I see is that when you restore, you lose file ownership
>and permissions. This is a grievous oversight IMO, but I should be
>to work around it by only really storing media files on there, where
>permissions are less of an issue.
>I'll do some testing with SnapRAID+MergerFS and get a feel for it.
>On 21-Feb-17 3:37 PM, jhock at iinet.net.au wrote:
>> Just for self interest, if anyone wanted to show me how to set up an
>open source cloud system, on a Linux system like Ubuntu, that allowed
>me to backup my entire file storage on my Android phone, wife's PC and
>my Xubuntu netbook, via my router's wifi, I would definitely like that
>to be a CLUG topic. But that's my self interest. :--)
>> Oh, and the cloud could be a Raspberry Pi with an attached 2Tb hard
>disk plugged into the router to reduce the power supply, would also be
>> On 21 February 2017 3:17:53 PM AEDT, Steven Hanley <sjh at svana.org>
>>> On Sun, Feb 05, 2017 at 08:51:52AM +1100, Stephen Hocking wrote:
>>>> This part of the group would be, which reminds me...
>>> An oh look it is CLUG week, if you have a talk that does sound like
>>> cool topic for everyone to learn about.
>>> See You
>>>> On 5 February 2017 at 00:53, Brenton Ross <rossb at fwi.net.au> wrote:
>>>>> Hi all,
>>>>> I have just finished putting together a presentation on testing
>>>>> VICI project.
>>>>> It covers a testing library for unit tests, asynchronous testing
>>>>> threads and child processes, testing of modules and programs and
>>>>> simulating user interactions with GUI programs.
>>>>> Would the group be interested ?
>>>>> linux mailing list
>>>>> linux at lists.samba.org
>>>> "I and the public know
>>>> what all schoolchildren learn
>>>> Those to whom evil is done
>>>> Do evil in return" W.H. Auden, "September 1, 1939"
>>>> linux mailing list
>>>> linux at lists.samba.org
>>> Steven Hanley sjh at svana.org http://svana.org/sjh/diary
>>> i can see her features begin to blur, as she pours herself
>>> into the mold he made for her, and for everything he does
>>> she has a way to rationalize
>>> Fixing Her Hair - Imperfectly - Ani
>>> linux mailing list
>>> linux at lists.samba.org
>linux mailing list
>linux at lists.samba.org
More information about the linux