This is NetApp a SAN filer and not a JBOD or an FC RAID shelf, right?
First of all, SAN servers are usually heavily optimized for IOPs and not streaming speed.
I have an IBM TotalStorage DS4300. It's a rebadged LSI unit also sold as the SGI TP9300. I think I clocked it at ~ 120MB/s streaming speed with 6x 300GB 10K disks, single 2Gb attachment. The deal with the DS4300 is that when the controller cache battery dies, it disables write caching altogether and this has a pretty disastrous effect on performance. These batteries are uncommon, quite expensive, and last only 2 or 3 years, so by now they are usually dead. Fortunately, I can overrule this behavior (and put the entire DS4300 behind a UPS if I care about integrity).
I have an SGI TP9100 as well. It's loaded with a grab bag of old disks, some even 1Gb/s models (Cheetah 73LP). But with some striping I got this:
I plan to load the TP9100 with 146GB 15K.4 disks. That should fly
Two things I like about the DS4300:
1) It supports IRIX, so failover and mutipathing 'just' works.
2) It's relatively quiet.
My experience with the QL2342 is quite good. It will deliver the expected performance. My experience with the LS driver (SAS/SATA, U320 SCSI and 4Gb FC) is more spotty:
* I use it with a pair of SATA disks in the O350. Performance is limited by the 2 SATA disks (streaming speeds ~ 240MB/s)
* I've used it with dual channel U320 SCSI and a DELL PowerVault 220S loaded two 7-way stripes of 72GB 15K.3 or 15K.4 disks and got disappointing results (200 or 250MB/s). The PowerVault was the U320 model. That was in the Tezro.
* I've got two dual 4Gb FC boards (LSI 7204XP) in the Tezro as well. That's what I used to obtain the TP9100 results above. And it didn't matter whether I attached both channels to a single LS card, or one each.
If I can get 380MB/s from a dual-2Gb attached FC array, and 200-250MB/s from a dual 320MB/s attached SCSI array on the same system there's something fishy going on. We've seen that SATA performs well earlier in this (?) thread. I have the feeling there's not much wrong with FC performance either, but that the U320 SCSI performance of the LS driver or hardware just isn't delivering. For now this is just a feeling, based on my own experience, and posts by Ian and others. I should bench some of this storage with a modern Linux/x64 server as well.
First of all, SAN servers are usually heavily optimized for IOPs and not streaming speed.
I have an IBM TotalStorage DS4300. It's a rebadged LSI unit also sold as the SGI TP9300. I think I clocked it at ~ 120MB/s streaming speed with 6x 300GB 10K disks, single 2Gb attachment. The deal with the DS4300 is that when the controller cache battery dies, it disables write caching altogether and this has a pretty disastrous effect on performance. These batteries are uncommon, quite expensive, and last only 2 or 3 years, so by now they are usually dead. Fortunately, I can overrule this behavior (and put the entire DS4300 behind a UPS if I care about integrity).
I have an SGI TP9100 as well. It's loaded with a grab bag of old disks, some even 1Gb/s models (Cheetah 73LP). But with some striping I got this:
Code: Select all
tezro 28# diskperf -W -D -n "Scratch All" -t 5 -m 8m -c2g /vol/junk
#---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name : Scratch All
# Test date : Wed May 30 23:43:29 2012
# Test machine : IRIX64 tezro 6.5 07202013 IP35
# Test type : XFS data subvolume
# Test path : /vol/junk
# Request sizes : min=16384 max=8388608
# Parameters : direct=1 time=5 scale=1.000 delay=0.000
# XFS file size : 2147483648 bytes
#---------------------------------------------------------
# req_size fwd_wt fwd_rd bwd_wt bwd_rd rnd_wt rnd_rd
# (bytes) (MB/s) (MB/s) (MB/s) (MB/s) (MB/s) (MB/s)
#---------------------------------------------------------
16384 53.17 59.62 34.28 3.03 32.44 3.61
32768 85.89 89.44 61.64 6.43 55.09 6.99
65536 117.55 121.65 80.78 13.81 83.84 12.42
131072 166.67 188.38 129.31 24.32 125.20 20.55
262144 191.93 216.40 175.22 38.56 157.68 35.47
524288 291.29 327.95 230.43 61.52 180.57 61.50
1048576 312.74 337.81 154.67 123.96 96.59 114.65
2097152 326.42 366.79 177.17 200.72 169.23 193.71
4194304 342.27 363.44 220.99 266.21 207.55 250.90
8388608 347.64 383.67 267.42 307.71 259.48 311.20
I plan to load the TP9100 with 146GB 15K.4 disks. That should fly
Two things I like about the DS4300:
1) It supports IRIX, so failover and mutipathing 'just' works.
2) It's relatively quiet.
My experience with the QL2342 is quite good. It will deliver the expected performance. My experience with the LS driver (SAS/SATA, U320 SCSI and 4Gb FC) is more spotty:
* I use it with a pair of SATA disks in the O350. Performance is limited by the 2 SATA disks (streaming speeds ~ 240MB/s)
* I've used it with dual channel U320 SCSI and a DELL PowerVault 220S loaded two 7-way stripes of 72GB 15K.3 or 15K.4 disks and got disappointing results (200 or 250MB/s). The PowerVault was the U320 model. That was in the Tezro.
* I've got two dual 4Gb FC boards (LSI 7204XP) in the Tezro as well. That's what I used to obtain the TP9100 results above. And it didn't matter whether I attached both channels to a single LS card, or one each.
If I can get 380MB/s from a dual-2Gb attached FC array, and 200-250MB/s from a dual 320MB/s attached SCSI array on the same system there's something fishy going on. We've seen that SATA performs well earlier in this (?) thread. I have the feeling there's not much wrong with FC performance either, but that the U320 SCSI performance of the LS driver or hardware just isn't delivering. For now this is just a feeling, based on my own experience, and posts by Ian and others. I should bench some of this storage with a modern Linux/x64 server as well.
Now this is a deep dark secret, so everybody keep it quiet
It turns out that when reset, the WD33C93 defaults to a SCSI ID of 0, and it was simpler to leave it that way... -- Dave Olson, in comp.sys.sgi
Currently in commercial service:
(2x)
In the museum : almost every MIPS/IRIX system.
Wanted : GM1 board for Professional Series GT graphics (030-0076-003, 030-0076-004)
It turns out that when reset, the WD33C93 defaults to a SCSI ID of 0, and it was simpler to leave it that way... -- Dave Olson, in comp.sys.sgi
Currently in commercial service:
(2x)
In the museum : almost every MIPS/IRIX system.
Wanted : GM1 board for Professional Series GT graphics (030-0076-003, 030-0076-004)
