Configurations using SATA drives are all the rage, given the lower cost of these drives. I have seen SATA drives used in the enterprise initially for D2D backup, but now I am starting to hear about and see sites using SATA drives for all storage requirements because of the cost savings. SATA is starting to be used for mainline storage for large databases and for just about everything FC drives are typically used for, but that comes with some reliability concerns.
To respond to those valid concerns, RAID vendors had to develop controllers that supported RAID-6, allowing for multiple failures by having two drives for parity instead of one. Because of the additional drive, you will need more bandwidth between the controller and drives than with RAID-5. For example, using the information from the Seagate Web site, for 750 GB SATA drives the average transfer rate is 78 MB/sec. If you have RAID-5 8+1, you need 702 MB/sec to run the RAID set at rate, but with RAID-6 8+2 you need 780 MB/sec, or about 10% more bandwidth, to complete the rebuild at full rate. The same full rate issues are true for streaming I/O for video, audio or any application that requires bulk reads or writes. One area that people seem to forget requires bulk rates is database re-indexing.
Almost everyone is at least looking at SATA drives, but given the bit error rate compared to FC drives (see Restoring Data That Matters), and the fact that for IOPS they are much slower because of the lower RPM rate and longer seek time, sites are still buying FC drives, but the trend is changing and I am not sure this is a good idea.
Let's get to the details of my conjecture. I assume that there are environments that require a certain level of performance for streaming I/O, documented in MB/sec or GB/sec. To achieve this performance you need the following hardware components:
- Memory bandwidth with enough performance to meet the I/O requirements
- PCI bus with enough performance to meet I/O requirements (see A Historic Moment for Storage I/O)
- Storage connections such as FC HBAs (the most common connection in the enterprise by far), USCSI, SATA, HCAs, or a NIC
- Switch ports such as a FC switch (the most common), InfiniBand switches, or Ethernet switches
- RAID controllers which have either an FC, IB, SATA, or Ethernet interface
- Disk drives which today have an FC, SATA or up and coming SAS interface
So what is needed is a system that is balanced to meet the performance requirements. To have that type of system requires hardware at each of the areas that meets the requirements: memory, PCI bus bandwidth, HBAs, switch ports, RAID controllers and, of course, disk drives. Each of these components requires power, cooling and, which costs money, but this O&M cost is not going to be considered in my assessment since I do not have cost numbers for this area.