Go Back  Bike Forums > The Lounge > Foo
Reload this Page >

Anybody here knows about RAID cards?

Foo Off-Topic chit chat with no general subject.

Anybody here knows about RAID cards?

Reply

Old 09-03-11, 01:52 AM
  #1  
RubenX 
Look! My Spine!
Thread Starter
 
RubenX's Avatar
 
Join Date: Apr 2008
Location: Kissimmee, FL
Posts: 620
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 9 Post(s)
Anybody here knows about RAID cards?

I have this RAID 5 array with 4 x 2Tb disks running on my main PC. It is working flawlessly for now but I expect to have more load soon (HTPC's mostly) and I wanted to start looking for upgrades.

The plan is to make a dedicated box, probably a headless dual core, with a *real* RAID controller. This are the features I think I need to look for:

* Memory Module - I'll be transferring 1Gb(aprox) files to the array quite often and I don't want everything to slow down. I understand that RAID cards with a memory module act like a HD with a huge cache. So I want that.

* External E-SATA connectors - At times I will be making backups and moving ghost images to and from external drives on a docking station, e-sata connected. I think it would work faster if the e-sata ports are right there on the card that controls the Array.

* Amount of SATA ports - Of course, I can't do with less than 4 ports. But 8 ports would be a lot nicier as it gives me room to expand.

I think SATA II 3Gb/sec will be sufficient, unless the price difference is not to much. In that case 6Gb/sec will win.

Finally, I have no idea about the brands. I am unfamiliar with this type of hardware. From what I have read, it seems that Areca is the way way to go.

Any input will be appreciated.

PS: No, I am not going to store/stream pron in/from this equipment.
__________________
"Hoy es un dia normal, pero yo voy a hacerlo intenso" ~ Juanes
RubenX is offline  
Reply With Quote
Old 09-03-11, 01:58 AM
  #2  
banerjek
Portland Fred
 
banerjek's Avatar
 
Join Date: Oct 2005
Posts: 11,411

Bikes: Custom Winter, Challenge Seiran SL, Fuji Team Pro, Cattrike Road/Velokit, РOS hybrid

Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 148 Post(s)
What's your usage goal? That dictates what solution would be best.
banerjek is online now  
Reply With Quote
Old 09-03-11, 06:18 AM
  #3  
rm -rf
don't try this at home.
 
rm -rf's Avatar
 
Join Date: Jan 2006
Location: N. KY
Posts: 5,008
Mentioned: 10 Post(s)
Tagged: 0 Thread(s)
Quoted: 596 Post(s)
You may not need a hardware raid controller. I think a separate computer with software raid and a decent amount of system memory would be fine. The gigabit network speed would be the bottleneck.

For instance, linux uses all it's available memory as disk buffers. So you could send a couple GB file to the server, and it would all be held in memory until it's written to the disk. And the buffers are used to send the file back out of the server, of course, so you aren't waiting for the write to finish.


10 years ago, I would get the fastest, most expensive raid controller for a new server at work. But now, even low end servers are mostly idle even when writing a lot of data to the disks. They can send faster than the disks can write.

~~~~~~~~~
I use the servers for applications like inventory or order entry, so there's a lot of random access of small sets of data. It's way more reading than writing. With 3 GB of disk buffers and 8 GB of live data, most of the currently used data is already in the disk buffers. The most disk activity is writing back data changes to the disk, not reading data off the disks. Now it's 10 times more writing than reading, where the old servers with small memory were 10-20 times more reading than writing. It's much, much faster to get a disk block from the buffer than have to get it from the disk itself.

Last edited by rm -rf; 09-03-11 at 06:41 AM.
rm -rf is offline  
Reply With Quote
Old 09-03-11, 06:29 AM
  #4  
RubenX 
Look! My Spine!
Thread Starter
 
RubenX's Avatar
 
Join Date: Apr 2008
Location: Kissimmee, FL
Posts: 620
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 9 Post(s)
Originally Posted by banerjek View Post
What's your usage goal? That dictates what solution would be best.
Not sure how I can measure that but, if I have Fiance + Daughter + Son streaming video from the Array on a Sunday afternoon and I decide to save my latest 2gb Sunday Ride Video into the thing, I don't want anybody screaming that their TV stop working (aka skipping too many frames or pausing)... lets picture it:

* Client 1 - Fiance - Watching "Sex & The City" in high def
* Client 2 - Son - Watching "The Mask" in high def
* Client 3 - Daughter - Watching "Barbie & The Diamond Castle" @ 720p
* Client 4 - Rube' downloading stuff @ 400kB/s
* Client 5 - Rube' saving 2gb movie

That sounds like something measurable?
__________________
"Hoy es un dia normal, pero yo voy a hacerlo intenso" ~ Juanes
RubenX is offline  
Reply With Quote
Old 09-03-11, 06:38 AM
  #5  
RubenX 
Look! My Spine!
Thread Starter
 
RubenX's Avatar
 
Join Date: Apr 2008
Location: Kissimmee, FL
Posts: 620
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 9 Post(s)
Originally Posted by rm -rf View Post
... The gigabit network speed would be the bottleneck....
hum... I forgot about that...
Here's the HD Tune stats of what I have right now:

__________________
"Hoy es un dia normal, pero yo voy a hacerlo intenso" ~ Juanes
RubenX is offline  
Reply With Quote
Old 09-03-11, 08:04 AM
  #6  
Wordbiker
Pwnerer
 
Wordbiker's Avatar
 
Join Date: Jun 2005
Posts: 2,909
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
As cheap as mechanical drives are these days, you may want to consider a RAID0+1 or RAID10 over a RAID5.

I've been reading that RAID5 arrays often aren't as recoverable as we're led to believe, and with the overhead needed to maintain them, potentially not as fast.
__________________
Originally Posted by ahsposo View Post
Ski, bike and wish I was gay.
Wordbiker is offline  
Reply With Quote
Old 09-03-11, 08:34 AM
  #7  
RubenX 
Look! My Spine!
Thread Starter
 
RubenX's Avatar
 
Join Date: Apr 2008
Location: Kissimmee, FL
Posts: 620
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 9 Post(s)
Originally Posted by Wordbiker View Post
As cheap as mechanical drives are these days, you may want to consider a RAID0+1 or RAID10 over a RAID5.

I've been reading that RAID5 arrays often aren't as recoverable as we're led to believe, and with the overhead needed to maintain them, potentially not as fast.
I tend to disagree on that one. On a big data center you will see hard drives going dead at least once a week. I've done it for over decade and I've yet to see an array failing because *one* drive went bad.
__________________
"Hoy es un dia normal, pero yo voy a hacerlo intenso" ~ Juanes
RubenX is offline  
Reply With Quote
Old 09-03-11, 09:41 AM
  #8  
Siu Blue Wind
BACK2ME
 
Siu Blue Wind's Avatar
 
Join Date: Apr 2005
Posts: 12,823
Mentioned: 34 Post(s)
Tagged: 0 Thread(s)
Quoted: 1465 Post(s)
Nerd alert!!!
__________________
"Foolish, selfish people are always thinking of themselves and the result is always negative. Wise persons think of others, helping them as much as they can and the result is happiness. Love and compassion are beneficial both to you and others. Through your kindness to others, your mind and heart will open to peace. " ~ Dalai Lama

Originally Posted by making View Post
Please dont outsmart the censor. That is a very expensive censor and every time one of you guys outsmart it it makes someone at the home office feel bad. We dont wanna do that. So dont cleverly disguise bad words.
Siu Blue Wind is offline  
Reply With Quote
Old 09-03-11, 03:55 PM
  #9  
dwellman
Godbotherer
 
dwellman's Avatar
 
Join Date: Jan 2011
Location: Hermitage, TN
Posts: 1,322

Bikes: 1986 Cannondale SR300 (full SRAM Apex) 1996 Cannondale R800 (Full SRAM Rival), 1997 Cannondale R200 (Shimano Tiagra), 2012 Cannondale CAAD 10-5, 1992 Bridgestone RB-1 (SRAM Force)

Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
I've just ported my entire data collection to mirrored ZFS volumes (all hail data snapshots!). So-called hardware RAID is really. . not needed, in fact it causes more problems than it solves. Especially if you're going to use a dedicated NAS / SAN, there's no reason to go HW raid, your CPU is MORE than up to the task of doing those pesky XORs and whatnot. As long as you have enough memory (at LEAST 1GB per TB, but you really want to max you RAM out) ZFS is pretty zippy.
dwellman is offline  
Reply With Quote
Old 09-03-11, 04:00 PM
  #10  
dwellman
Godbotherer
 
dwellman's Avatar
 
Join Date: Jan 2011
Location: Hermitage, TN
Posts: 1,322

Bikes: 1986 Cannondale SR300 (full SRAM Apex) 1996 Cannondale R800 (Full SRAM Rival), 1997 Cannondale R200 (Shimano Tiagra), 2012 Cannondale CAAD 10-5, 1992 Bridgestone RB-1 (SRAM Force)

Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Originally Posted by RubenX View Post
I've done it for over decade and I've yet to see an array failing because *one* drive went bad.
True that. We have suffered backplane failures, though. I hate backplane failures.
dwellman is offline  
Reply With Quote
Old 09-03-11, 04:42 PM
  #11  
AEO
Senior Member
 
AEO's Avatar
 
Join Date: Jun 2007
Location: A Coffin Called Earth. or Toronto, ON
Posts: 12,258

Bikes: Bianchi, Miyata, Dahon, Rossin

Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
as a, formerly, long time soft RAID5 user, IMO, it's much much simpler to use some offline hard drives as a form of backup, than having a massive raid 5 array that's on all the time.

RAID 5 is susceptible to file table corruption due to power outs, system freezes and whatever else ailments computers have.
RAID 5 cards, especially hardware ones, are very expensive compared to the hard drives.
The 2TB hard drives in production today are not designed for 24/7 operation, so they are more likely to cause a hiccup in the MFT.
The 2TB hard drives have spin down when not in use, but RAID5 will need to force all drives to be on all the time and that increases the chances of some form of file corruption.
soft RAID 5 cards cost at least $120 for 4 ports, and hardware raid5 cards with 8 or more ports cost upwards of $400, while 2TB hard drives only cost $60.
Motherboards with 5x SATA and 1x eSATA can easily be had for less than $100. Some of the better ones with 8x ports will only cost up to $200.

Instead, it's much simpler and easier if you just make multiple shares for each individual drive in the system and use older hard drives as a backup. You just tell every user that they have a drive of their own that they can access and since every user gets their own drive, you don't have to worry about the read speed of the drive as they don't tie each other up.

What is needed, however is a system that doesn't need to run an anti-virus or if it does, then it should be fast enough to handle all the streams in parallel with GB Ethernet.

I've found that you need at least 100MB/s to stream HD files, and GB Ethernet will max out at around 250MB/s per disk.
Another bottleneck is the router and switch. They both have to be GBE, but here is the tricky part. Anything that must go over both router and switch will get throttled to about 200MB/s if they're not up to the task. If the file transfers only have to go over 1 device, either router or switch, then you can easily achieve 500MB/s. If more than one person wants to stream over wifi, then this can also cause problems for anything less than wireless-N with powerful antennas, as wifi devices tend to jam each other's signal.
__________________
Food for thought: if you aren't dead by 2050, you and your entire family will be within a few years from starvation. Now that is a cruel gift to leave for your offspring. ;)
http://sanfrancisco.ibtimes.com/arti...ger-photos.htm

Last edited by AEO; 09-03-11 at 04:46 PM.
AEO is offline  
Reply With Quote
Old 09-03-11, 06:27 PM
  #12  
RubenX 
Look! My Spine!
Thread Starter
 
RubenX's Avatar
 
Join Date: Apr 2008
Location: Kissimmee, FL
Posts: 620
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 9 Post(s)
Originally Posted by dwellman View Post
I've just ported my entire data collection to mirrored ZFS volumes (all hail data snapshots!). So-called hardware RAID is really. . not needed, in fact it causes more problems than it solves. Especially if you're going to use a dedicated NAS / SAN, there's no reason to go HW raid, your CPU is MORE than up to the task of doing those pesky XORs and whatnot. As long as you have enough memory (at LEAST 1GB per TB, but you really want to max you RAM out) ZFS is pretty zippy.
I sense a hardcore Unix admin that bypasses Linux and goes straight to Solaris 10 x86 version.
__________________
"Hoy es un dia normal, pero yo voy a hacerlo intenso" ~ Juanes
RubenX is offline  
Reply With Quote
Old 09-03-11, 06:58 PM
  #13  
RubenX 
Look! My Spine!
Thread Starter
 
RubenX's Avatar
 
Join Date: Apr 2008
Location: Kissimmee, FL
Posts: 620
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 9 Post(s)
OK, so it looks more likely to hit the network bottleneck before anything else. I'm thinking 2 or 3 extra NICs on the SAN, aliased (SAN2, SAN3, SAN4...), going to secondary NICs of the clients via crossover cables.

*cleans crimping tool*
__________________
"Hoy es un dia normal, pero yo voy a hacerlo intenso" ~ Juanes
RubenX is offline  
Reply With Quote
Old 09-03-11, 09:25 PM
  #14  
AEO
Senior Member
 
AEO's Avatar
 
Join Date: Jun 2007
Location: A Coffin Called Earth. or Toronto, ON
Posts: 12,258

Bikes: Bianchi, Miyata, Dahon, Rossin

Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
If you're going to use all computers with wired connection, then I would just buy an 8 port GBE switch, hook it up to all computers, then use a cross over to connect the switch to a high power (as in CPU and RAM) router like a DGL-4100 or 4500.

Either that or use an 8 port router, but I'm not sure which routers have 8 ports and a good CPU/RAM. They sure aren't the cheaper ones you can buy for $30, but cost around $80 to $130

You're going need a lot of PCI-e 1x NICs and a mobo that has that many slots if you don't want your PCI bus to be saturated. PCI bus will saturate at 133MB/s and all ports are shared, so it'll be even slower if you try that.

I think I've heard of a PCI-e 4x 4 or 8 port NIC before, but I don't recall where.
__________________
Food for thought: if you aren't dead by 2050, you and your entire family will be within a few years from starvation. Now that is a cruel gift to leave for your offspring. ;)
http://sanfrancisco.ibtimes.com/arti...ger-photos.htm

Last edited by AEO; 09-03-11 at 09:28 PM.
AEO is offline  
Reply With Quote
Old 09-03-11, 11:07 PM
  #15  
Wordbiker
Pwnerer
 
Wordbiker's Avatar
 
Join Date: Jun 2005
Posts: 2,909
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Originally Posted by RubenX View Post
I tend to disagree on that one. On a big data center you will see hard drives going dead at least once a week. I've done it for over decade and I've yet to see an array failing because *one* drive went bad.
OK, I was just trying to help. No need to humiliate and belittle me.
__________________
Originally Posted by ahsposo View Post
Ski, bike and wish I was gay.
Wordbiker is offline  
Reply With Quote
Old 09-04-11, 10:36 AM
  #16  
dwellman
Godbotherer
 
dwellman's Avatar
 
Join Date: Jan 2011
Location: Hermitage, TN
Posts: 1,322

Bikes: 1986 Cannondale SR300 (full SRAM Apex) 1996 Cannondale R800 (Full SRAM Rival), 1997 Cannondale R200 (Shimano Tiagra), 2012 Cannondale CAAD 10-5, 1992 Bridgestone RB-1 (SRAM Force)

Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Originally Posted by RubenX View Post
I sense a hardcore Unix admin that bypasses Linux and goes straight to Solaris 10 x86 version.
Close: FreeBSD.

. . . except for DB server. Then its AIX or Fedora/RHEL/CentOS
dwellman is offline  
Reply With Quote
Old 09-04-11, 11:16 AM
  #17  
dwellman
Godbotherer
 
dwellman's Avatar
 
Join Date: Jan 2011
Location: Hermitage, TN
Posts: 1,322

Bikes: 1986 Cannondale SR300 (full SRAM Apex) 1996 Cannondale R800 (Full SRAM Rival), 1997 Cannondale R200 (Shimano Tiagra), 2012 Cannondale CAAD 10-5, 1992 Bridgestone RB-1 (SRAM Force)

Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Originally Posted by RubenX View Post
OK, so it looks more likely to hit the network bottleneck before anything else. I'm thinking 2 or 3 extra NICs on the SAN, aliased (SAN2, SAN3, SAN4...), going to secondary NICs of the clients via [B]crossover cables[/B
Not sure what you mean. Streaming 720p can be as low a 1.5 Mbps up to 4 Mbps worst case scenario you have 600Mbps worth of bandwidth on a typical GbE connection. Nice managed Layer 2 or Layer 3 switch (or even router based QoS/Shaping) would be a simpler and more elegant (albeit expensive) solution. BUT you eighty-six that Highpoint RocketRAID card you been eyeballing. . . yeah, that's about a price point between the switch options.

There's also link aggregation, but I don't know of a *easy* (hw restrictions abound) way to do this on your typical OS. . . its possible, but kind of a hassle.
dwellman is offline  
Reply With Quote
Old 09-04-11, 12:28 PM
  #18  
RubenX 
Look! My Spine!
Thread Starter
 
RubenX's Avatar
 
Join Date: Apr 2008
Location: Kissimmee, FL
Posts: 620
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 9 Post(s)
Oh yess.. talk dirty to mah....





...let the case twist a little @ bootup from the sheer combined torque ...
__________________
"Hoy es un dia normal, pero yo voy a hacerlo intenso" ~ Juanes
RubenX is offline  
Reply With Quote
Old 09-04-11, 02:06 PM
  #19  
dwellman
Godbotherer
 
dwellman's Avatar
 
Join Date: Jan 2011
Location: Hermitage, TN
Posts: 1,322

Bikes: 1986 Cannondale SR300 (full SRAM Apex) 1996 Cannondale R800 (Full SRAM Rival), 1997 Cannondale R200 (Shimano Tiagra), 2012 Cannondale CAAD 10-5, 1992 Bridgestone RB-1 (SRAM Force)

Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Alrighty then. . .
dwellman is offline  
Reply With Quote
Old 09-04-11, 03:48 PM
  #20  
AEO
Senior Member
 
AEO's Avatar
 
Join Date: Jun 2007
Location: A Coffin Called Earth. or Toronto, ON
Posts: 12,258

Bikes: Bianchi, Miyata, Dahon, Rossin

Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
my 3x 2TB drives > your 8x 500GB RAID 5 array

just sayin'
__________________
Food for thought: if you aren't dead by 2050, you and your entire family will be within a few years from starvation. Now that is a cruel gift to leave for your offspring. ;)
http://sanfrancisco.ibtimes.com/arti...ger-photos.htm
AEO is offline  
Reply With Quote
Old 09-05-11, 05:04 AM
  #21  
DannoXYZ 
Senior Member
 
DannoXYZ's Avatar
 
Join Date: Jul 2005
Location: Saratoga, CA
Posts: 11,739
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 101 Post(s)
10-gig network is the answer.
DannoXYZ is offline  
Reply With Quote
Old 09-05-11, 07:27 AM
  #22  
dwellman
Godbotherer
 
dwellman's Avatar
 
Join Date: Jan 2011
Location: Hermitage, TN
Posts: 1,322

Bikes: 1986 Cannondale SR300 (full SRAM Apex) 1996 Cannondale R800 (Full SRAM Rival), 1997 Cannondale R200 (Shimano Tiagra), 2012 Cannondale CAAD 10-5, 1992 Bridgestone RB-1 (SRAM Force)

Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
I'm on the "this is so much overkill" side of the fence. . . But, then again, I've subordinated my network to my power bill, so 2 x ST9320325AS on MSI Wind Board (replacing deceased Intel D945GCLF2) with 1 GB DRAM sitting in a Habey EMC-600B is more than enough for me.

What I'd really like is a SD card reader that plugs directly to an internal USB header, so I can take the USB stick off the back. Maybe PC Engines has something. . .

EDIT: Doesn't exist. Could probably make one. Like this
except a microSDHC reader on the end.

Last edited by dwellman; 09-05-11 at 07:38 AM.
dwellman is offline  
Reply With Quote

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Thread Tools
Search this Thread

Contact Us Archive Advertising Cookie Policy Privacy Statement Terms of Service