[clug] Storage and networking for Storage as used for supporting [Enterprise] virtualized environments

[George at Clug]

Hi,

I am looking for comments on my  below research on  "real world, a opposed to theoretical"  data speeds for various storage and network technologies;

Below are the figures from my research (created for mono spaced font).
Device                Realistic data throughput 
ADSL                 100KB/s, 200KB/s,400K/s, 1MB/s
7200 RPM HD        30-80MB/s
7200 RPM SATA        100MB/s
10K  RPM     100-130MB/s
15K  RPM         150-190MB/s
Ethernet  100BASE-T   10MB/s 
Ethernet 1000BASE-T  116MB/s
Ethernet 1000BASE-T (Jumbo Frames) 123MB/s
SATA II             300MB/s
SSDs                530/500MB/s
SATA III            600MB/s
RAID 6 x6 HD        600MB/s (4x read speed, no write speed gain)
Ethernet 10GBASE-T 1.25GB/s
RAID 6 x24 HD       2.2GB/s (22x read speed, no write speed gain)
NVMe  (M.2)             3.4GB/s (3500MB/s )
iSCSI                1GB/s, 10GB/s, and 40GB/s
Fibre Channel        1GB/s, 2GB/s, 4GB/s, 8GB/s, 10GB/s, 16GB/s, 32GB/s and 128GB/s

Not having much practical experience in Enterprise storage I am curious about any comments people can or would like to make.

Regards,

George.




<B>Sources</B>

https://forum.huawei.com/enterprise/en/some-differences-between-scsi-iscsi-fcp-fcoe-fcip-nfs-cifs-das-nas-san/thread/229549-891
Some differences between SCSI, ISCSI, FCP, FCoE, FCIP, NFS, CIFS, DAS, NAS, SAN.
Created: Sep 30, 2014 15:25:49Latest reply: Aug 6, 2019 20:20:21

http://www.raid-calculator.com/default.aspx

https://lenovopress.com/sg247986.pdf

https://forum.huawei.com/enterprise/en/some-differences-between-scsi-iscsi-fcp-fcoe-fcip-nfs-cifs-das-nas-san/thread/229549-891
Network-attached storage (NAS), in contrast to, uses file-based protocols such as NFS or SMB/CIFS where it is clear that the storage is remote, and computers request a portion of an abstract file rather than a disk block. The key difference between direct-attached storage (DAS) and NAS is that DAS is simply an extension to an existing server and is not necessarily networked. NAS is designed as an easy and self-contained solution for sharing files over the network.
Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystems in 1984, allowing a user on a client computer to access files over a network in a manner similar to how local storage is accessed. On the contrary, CIFS is its Windows-based counterpart used in file sharing.

https://www.sciencedirect.com/topics/computer-science/storage-virtualization

https://esj.com/articles/2014/05/16/fibre-channel-or-iscsi.aspx
Fibre Channel infrastructure operates at throughput speeds of 1, 2, 4, 8, 10 and 16GB per second (GB/s). Over the years, speeds have continued to increase as storage performance demands have accelerated. Even faster speeds of 32GB/s and 128GB/s are expected to hit the market in the next couple of years.
iSCSI uses standard Ethernet switches and cabling and operates at speeds of 1GB/s, 10GB/s, and 40GB/s.

https://www.enterprisestorageforum.com/storage-hardware/ssd-vs-hdd-speed.html

https://www.velocitymicro.com/blog/nvme-vs-m-2-vs-sata-whats-the-difference/

https://www.convert-me.com/en/convert/data_transfer_rate/dEtherthous.html?u=dEtherthous&v=1

https://www.cablefree.net/wireless-technology/maximum-throughput-gigabit-ethernet/
Theoretical throughput of Gigabit Ethernet with jumbo frames, and using TCP:
997Mbps – .886 – 1.33 – 1.55 – .443 – 2.21 – 2.21 – 1.33 = 987Mbps or 123MB/s.
The approximate throughput for Gigabit Ethernet without jumbo frames and using TCP is around 928Mbps or 116MB/s.

http://rickardnobel.se/actual-throughput-on-gigabit-ethernet/
Conclusion: Default Gigabit Ethernet has an impressive number of frames (about 81000 per second) possible and a high throughput for actual data (about 118 MB/s). By increasing the MTU to 9000 we could deliver even more data on the same bandwidth, up to 123 MB/s, thanks to the decreased amount of overhead due to a lower number of frames. Jumbo Frames could use the whole of 99% of Gigabit Ethernet bandwidth to carry our data.

https://www.velocitymicro.com/blog/nvme-vs-m-2-vs-sata-whats-the-difference/
Modern motherboards use SATA III which maxes out at a throughput of 600MB/s (or 300MB/s for SATA II, in which case, it’s time to upgrade). Via that connection, most SSDs will provide Read/Write speeds in the neighborhood of 530/500 MB/s. For comparison, a 7200 RPM SATA drive manages around 100MB/s depending on age, condition, and level of fragmentation. NVMe drives, on the other hand, provide write speeds as high as 3500MB/s. That’s 7x over SATA SSDs and as much as 35x over spinning HDDs!

https://photographylife.com/nvme-vs-ssd-vs-hdd-performance
In just read performance alone, my M.2 drive turned out to be a whopping 25x times faster than my enterprise-grade WD 2 TB 7200 RPM drive. That is just mind boggling, considering that SSD is only around 5x times faster in comparison. In write speed, I was able to witness up to 15x more performance, which is also a very impressive number. And that’s just for one type of sequential read/write load – if you look at the above numbers, other performance metrics indicate even larger, more noticeable gains.

https://www.zdnet.com/article/why-raid-6-stops-working-in-2019/
7200 RPM full drive writes average about 115 MB/sec

https://www.microsemi.com/product-directory/raid-controllers/4047-raid-levels#16

https://www.atto.com/software/files/techpdfs/TechnicalSpecifications_FastFrameNIC.pdf

https://www.sciencedirect.com/topics/engineering/gigabit-ethernet
Given a further 1% of overhead for TCP, this leaves a total of 118.75 MB/s for data transmission.

Below are the figures from my research (created for tabs).
Device 			Realistic data throughput 
ADSL 			100KB/s, 200KB/s,400K/s, 1MB/s
7200 RPM HD 		30-80MB/s
7200 RPM SATA 		100MB/s
10K  RPM 		100-130MB/s
15K  RPM 		150-190MB/s
Ethernet 100BASE-T 	10MB/s 
Ethernet 1000BASE-T 	116MB/s
Ethernet 1000BASE-T (Jumbo Frames)	123MB/s
SATA II			300MB/s
SSD			530/500MB/s
SATA III		600MB/s
RAID 6 x6 HD		600MB/s (4x read speed, no write speed gain)
Ethernet 10GBASE-T 	1.25GB/s
RAID 6 x24 HD 		2.2GB/s (22x read speed, no write speed gain)
NVMe  (M.2) 		3.4GB/s (3500MB/s )
iSCSI 			1GB/s, 10GB/s, and 40GB/s
Fibre Channel 		1GB/s, 2GB/s, 4GB/s, 8GB/s, 10GB/s, 16GB/s, 32GB/s and 128GB/s