FYI: ZFS on Fusion Drive actually works.

Moderators: jhartley, MSR734, nola

Re: FYI: ZFS on Fusion Drive actually works.

Post by grahamperrin » Fri Nov 02, 2012 2:14 pm

As far as I recall: whilst it's easy enough to give a CoreStorage logical volume to ZEVO, it was less easy to remove that type of thing from CoreStorage world. Something like that … in retrospect, the issue might have been the dynamic nature of ZEVO.


Got it: diskutil coreStorage deleteLVG fails where there's ZFS
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: FYI: ZFS on Fusion Drive actually works.

Post by grahamperrin » Sat Nov 03, 2012 6:43 am

grahamperrin wrote:You might get more milage from using part of the SSD as L2ARC or ZIL (not moving the startup volume away from that SSD).


Rewind! For some use cases, that may be bad advice. My apologies.

In Server Fault, quoted in full from the accepted answer to ZFS - how to partition SSD for ZIL or L2ARC use? –

It's not a good idea to share an SSD between pools for reasons of data integrity and performance.

First, ZFS needs to be able to trigger the device's onboard cache to flush when a synchronous write is requested, to ensure that the write is really on stable storage before returning to the application. It can only do this if it controls the whole device. If using a slice, ZFS cannot issue the cache flush and you risk losing data during an unexpected shutdown.

Second, the SSD, while very fast, is still a finite resource. Sharing that resource between pools means that the expected performance of the device from either pool's perspective could be vastly different if the drive is busy serving IOPS from the other pool. The end result is that you could have worse performance than if you didn't use the SSD at all.

Dedicated ZIL and L2ARC devices per pool is the way to go.


So: maybe not a good idea to use one part (a slice) of an SSD for L2ARC or ZIL, if another part is for a different purpose.
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: FYI: ZFS on Fusion Drive actually works.

Post by si-ghan-bi » Sat Nov 03, 2012 9:02 am

I think it depends in the goal and system and load: using a SSD for both OS and L2ARC (let's say 200 GB and 40 GB, but less L2ARC is possible) is probably a very good compromise for all systems where the SSD is not being heavily used (very heavily: it takes a lot of activity to slow the SSD access time down to HDD performances) and the RAM is limited (below 8 GB).
The same applies to ZIL, but it is probably more critical.
si-ghan-bi Offline


 
Posts: 145
Joined: Sat Sep 15, 2012 5:55 am

Re: FYI: ZFS on Fusion Drive actually works.

Post by grahamperrin » Sat Nov 03, 2012 5:02 pm

Thanks … conscious that I took this way off-topic from the opening post (sorry, Patrick). I had some other questions about L2ARC and ZIL, these might fit under What is best use of spare SSD: read or write cache? or Benchmark/check L2ARC performances.

Back to Fusion Drive … ZEVO on CoreStorage with auto-tiering … 

If a routine scrub will demote data, then might we mitigate that negativity by carefully timing an incremental zfs send to occur after (not during) a scrub?
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Thanks!

Post by grahamperrin » Sun Nov 04, 2012 5:13 am

Thanks again, Patrick … richly deserved! :D
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: FYI: ZFS on Fusion Drive actually works.

Post by TomUnderhill » Sun Nov 04, 2012 5:14 am

Following-up to my best use of a spare SSD post that grahamperrin referenced. Because I don't leave my MacPro running 24/7, using a spare SSD as L2ARC gives minimal performance improvement.

My current rig has changed slightly since my posting:

MacPro3,1
  • 24GB RAM
  • 1x 128GB SSD (boot)
  • 3x 2TB HDD
  • 1x 64GB SSD
  • a whole bunch of 1TB drives stacked around my office, with various FW, USB and eSATA mounts available

I currently have the 3x 2TB HDDs in a 4TB RAIDZ filled just over 50%. Of that ~2.25TB of data, I can easily remove almost 1TB from that equation for a real dataset about 1.5TB in size.

I want to maximize throughput (read and write) for photo-editing/graphic design/book publishing/MySQL and PHP programming. I don't need video editing speeds. I realize there is a leap from playing with hypothetical setups with thumb drives to real, live, working data that pays the bills--but let's throw caution (not quite!) to the wind and see what happens. I've always been more dare-devil than wallflower, and I'm not achieving the throughput I want with my current set, but those are issues for different posts. Patrick, given these requirements, how would you configure the storage, and would you bring Fusion Drive into the mix? Same question for anyone else who would like to chime in.

Sometime in Q1 I'm planning on using the 1TB drives in an eSATA configuration to add additional redundancy and backup to the mix, replacing my stand-alone 5TB NAS. Looking ahead, any suggestions for a PCIe card for driving the external box of drives? PCIe x4 or x8 are available.

Let's have fun!
TomUnderhill Offline


 
Posts: 36
Joined: Wed Oct 10, 2012 8:06 am
Location: Southern California

Re: FYI: ZFS on Fusion Drive actually works.

Post by grahamperrin » Sun Nov 04, 2012 7:00 am

In Ask Different, under One mac, Two or more storage devices - how to divide things? alongside the accepted answer I added:


Some of it is dragon territory (!) but the opening poster Mike Bradshaw is extraordinarily good at teasing things into shape for consumption by the masses. Let's see how this fans out …
grahamperrin Offline

User avatar
 
Posts: 1596
Joined: Fri Sep 14, 2012 10:21 pm
Location: Brighton and Hove, United Kingdom

Re: FYI: ZFS on Fusion Drive actually works.

Post by ntinti » Sun Dec 16, 2012 1:06 pm

jollyjinx wrote:To spare someone the time to test:
I've been fiddling with Apples core storage layer (Fusion drive on old Macs - http://jollyjinx.tumblr.com) and now test ZFS on a fusion drive (FUD). I got ZFS using the FUD core storage and it is working by using first the SSD and then the HDD. Accessing data on the HDD will bring them eventually onto the SSD.
This might be a workaround of the limitation that a ZFS SSD cache disk is empty after reboot.


If you want more security than a striped volumen gives you, it is possible to use Raid1 mirrored fusion drives. Both appleraid and zpool mirror works with two fusion drive volumes

diskutil cs create FDrive1 disk2 disk1
diskutil cs create FDrive2 disk3 disk4
diskutil cs createVolume 2F... jhfs+ fd1sys 256G
diskutil cs createVolume 2F... jhfs+ fd1data 100%
diskutil cs createVolume 6E... jhfs+ fd2sys 256G
diskutil cs createVolume 6E... jhfs+ fd2data 100%
diskutil ar create mirror MacOS JHFS+ disk6 disk8
zpool create -f Data mirror /dev/disk7 /dev/disk9
ntinti Offline


 
Posts: 1
Joined: Tue Dec 11, 2012 4:47 pm

Re: FYI: ZFS on Fusion Drive actually works.

Post by TomUnderhill » Wed Dec 19, 2012 6:37 pm

Wanted to try ZFZ on Fusion, so I built a performance beast proof-in-concept, and it's pretty impressive. Started by hitting the good ole 'nearly dead and mostly obsolete' parts bin.

Grabbed my first SSD. 64GB. Then pulled the 160GB hard drive from a dead 2007 or 2008 MacBook Pro and dropped the drive onto a spare SATA to USB sled.

Before I cobbled things together, I ran a few iostat benchmarks. I'm waiting to pick up one of my kids from school and don't have the actual numbers handy.

    SSD, JHFS+ ~60GB/s
    HDD, JHFS+ ~28GB/s
    FUD, JHFS+ ~60GB/s

    FUD, ZFS ~60GB/s
I'll toss some more data at the drive and see what happens with performance over the next couple of days.

One other point to make: so much talk of Fusion Drives have been with much larger HDDs, and the SSD contributes a minor percentage to the aggregate drive. My 220GB Fusion Drive was roughly 1/3 SSD. Cool.
TomUnderhill Offline


 
Posts: 36
Joined: Wed Oct 10, 2012 8:06 am
Location: Southern California

Re: FYI: ZFS on Fusion Drive actually works.

Post by raattgift » Mon Mar 04, 2013 2:54 am

zevo always creates labels on unlabelled devices it's told to use; that includes CoreStorage LVs, which will usually contain a (J)HFS+ volume without a label.

The CoreStorage framework and some ancillary software is different in different 10.8.2 builds. The 10.8.2 builds that run (pretty much only) on 2012 Mac Minis will log a line like :

Mar 3 09:24:48 localhost kernel[0]: thr 0xffffff8028624aa0 Composite Disk alg="bloomclock" unit_nbytes=131072

for any Composite Disk (CPDK, which is any LVG spanning multiple PVs) where there is exactly one PV that is recognized as Solid State. If you don't see a line like that, your system will not migrate blocks between the various PVs in your CPDK.

Only drives attaching via IOKIt's IOAHCIBlockStorageDevice class can be recognized as Solid State without heavy patching in /System/Library. However, the shipped code will match most SATA-3 SSDs installed on the internal bus or via Thunderbolt and a SATA-3 adaptor (like Seagate's).

Finally, a lot of the relevant operation of CoreStorage can be dtraced.
raattgift Offline


 
Posts: 98
Joined: Mon Sep 24, 2012 11:18 pm

Previous

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 0 guests

cron