Storage Area Network Usage A UNIX SysAdmin’s View of How A SAN Works - PowerPoint PPT Presentation

storage area network usage a unix sysadmin s view of how a san works n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Storage Area Network Usage A UNIX SysAdmin’s View of How A SAN Works PowerPoint Presentation
Download Presentation
Storage Area Network Usage A UNIX SysAdmin’s View of How A SAN Works

play fullscreen
1 / 60
Storage Area Network Usage A UNIX SysAdmin’s View of How A SAN Works
321 Views
Download Presentation
geona
Download Presentation

Storage Area Network Usage A UNIX SysAdmin’s View of How A SAN Works

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Storage Area Network UsageA UNIX SysAdmin’s View of How A SAN Works

  2. Disk Storage • Embedded • Internal Disks within the System Chassis • Directly Attached • External Chassis of Disks connected to a Server via a Cable • Directly Attached Shared • External Chassis connected to more than one Server via a Cable • Networked Storage • NAS • SAN • others

  3. Disk Storage – 2000-2004

  4. Deficiencies of Direct Connect Storage • Single System Bears Entire Cost of Storage • Small Server in an EMC Shop • Large Server cannot easily share its unused storage • Managability • Fragmented and Isolated • Scalability • Limited • What happens when you run out of peripheral bus slots? • Availability • “SCSI Bus Reset” • Failover is a complicated add-on, if available at all

  5. DASD • Direct Access Storage Device • They still call it this in an IBM Mainframe Shop • Basic Limits of Disk Storage Recognized • Latency • Rotation Speed of the disk • Seek Time • Radial Movement of the Read/Write Heads • Buffer Sizes • Stop sending me data, I can’t write fast enough!

  6. SCSI • SCSI – Small Computer System Interface • From Shugart’s 1979 SASI implementation • SASI: Shugart Associates System Interface • Both Hardware and I/O Protocol Standards • Both have evolved over time • Hardware is source of most limitations • I/O Protocol has long-term potential

  7. SCSI - Pro • Device Independence • Mix and match device types on the bus • Disk, Tape, Scanners, etc… • Overlapping I/O Capability • Multiple read & write commands can be outstanding simultaneously • Ubiquitous

  8. SCSI - Con • Distance vs. Speed • Double the Signaling Rate • Speed: 40, 80, 160, 320 MBps • Halve the Cable Length Limits • Device Count: 16 Maximum • Low voltage Differential Ultra3 SCSI can support only 16 devices on a 12 meter cable at 160 MBps • Server Access to Data Resources • Hardware changes are disruptive

  9. SCSI – Overcoming the Con • New Hardware & Signaling Platforms • SCSI-3 Introduces Serial SCSI Support • Fibre Channel • Serial Storage Architecture (SSA) • Primarily an IBM implementation • FireWire (IEEE 1394 – Apple fixes SCSI) • Attractive in consumer market • Retains SCSI I/O Protocol

  10. Scaling SCSI Devices • Increase Controller Count within Server • Increasing Burden To CPU • Device Overhead • Bus Controllers can be saturated • You can run out of slots • Many Queues, Many Devices • Queuing Theory 101 (check-out line) - undesirable

  11. Scaling SCSI Devices • Use Dedicated External Device Controller • Hides Individual Devices • Provide One Large Virtual Resource • Offloads Device Overhead • One Queue, Many Devices - good • Cost and Benefit • Still borne by one system

  12. RAID • Redundant Array of Inexpensive Disks • Combine multiple disks into a single virtual device • How this is implemented determines different strengths • Storage Capacity • Speed • Fast Read or Fast Write • Resilience in the face of device failure

  13. RAID Functions • Striping • Write consecutive logical byte/blocks on consecutive physical disks • Mirroring • Write the same block on two or more physical disks • Parity Calculation • Given N disks, N-1 consecutive blocks are data blocks, Nth block is for parity • When any of the N-1 data blocks are altered, N-2 XOR calculations are performed on these N-1 blocks • The Data Block(s) and Parity Block are written • Destroy one of these N blocks, and that block can be reconstructed using N-2 XOR calculations on the remaining N-1 blocks • Destroy two or more blocks – reconstruction is not possible

  14. RAID Function – Pro & Con • Striping • Pro: Increases Spindle Count for Increased Thruput • Con: Does not provide redundancy • Mirroring • Pro: Provides Redundancy without Parity Calculation • Con: Requires at least 100% disk resource overhead • Parity Calculation • Pro: Cuts Disk Resource Overhead to 1/N • Con: Parity calculation is expensive N-2 calculations are required If all N-1 data blocks are not in cache, they must be read

  15. RAID Types • RAID 0 • Stripe with No Parity • RAID 1 • Mirror two or more disks • RAID 0+1 • Stripe on Inside, Mirror on Outside • RAID 1+0 • Mirrors on Inside, Stripe on Outside • RAID 3 • Synchronous, Subdivided Block Access; Dedicated Parity Drive • RAID 4 • Independent, Whole Block Access; Dedicated Parity Drive • RAID 5 • Like RAID 4, but Parity striped across multiple drives

  16. RAID 0 RAID 1

  17. RAID 3 RAID 5

  18. RAID 1+0 RAID 0+1

  19. Breaking the Direct Connection • Now you have high performance RAID • The storage bottleneck has been reduced • You’ve invested $$$ to do it • How do you extend this advantage to N servers without spending N x $$$? • How about using existing networks?

  20. How to Provide Data Over IP • NFS (or CIFS) over a TCP/IP Network • This is Network Attached Storage (NAS) • Overcomes some distance problems • Full Filesystem Semantics are Lacking • …such as file locking • Speed and Latency are problems • Security and Integrity are problems as well • IP encapsulation of I/O Protocols • Not yet established in the marketplace • Current speed & security issues

  21. NAS and SAN • NAS – Network Attached Storage • File-oriented access • Multiple Clients, Shared Access to Data • SAN – Storage Area Network • Block-oriented access • Single Server, Exclusive Access to Data

  22. NAS: Network Attached Storage • File Objects and Filesystems • OS Dependent • OS Access & Authentication • Possible Multiple Writers • Require locking protocols • Network Protocol: i.e., IP • “Front-end” Network

  23. SAN: Storage Area Network • Block Oriented Access To Data • Device-like Object is presented • Unique Writer • I/O Protocol: SCSI, HIPPI, IPI • “Back-end” Network

  24. A Storage Area Network • Storage • StorageWorks MA8000 (24), EVA (2) • HDS is 2nd Approved Storage Vendor • 9980 Enterprise Storage Array – EMC class storage • Switches • Brocade 12000 (8), 3800 (20), & 2800 (34) • 3900’s are being deployed – 32 port • UNIX Servers on the SAN • Solaris (56), IRIX (5), HP-UX (5), Tru64 (1) • Storage Volume Connected to UNIX Servers • 13000 GB as of May, 2003 • Windows Servers • Windows 2000 (74), NT 4.0 (16)

  25. SAN Implementations • FibreChannel • FC Signalling Carrying SCSI Commands & Data • Non-Ethernet Network Infrastructure • iSCSI • SCSI Encapsulated By IP • Ethernet Infrastructure • FCIP – FibreChannel over IP • FibreChannel Encapsulated by IP • Extending FibreChannel over WAN Distances • Future Bridge between Ethernet & FibreChannel • iFCP - another gateway implementation

  26. NAS & SAN in the Data Center

  27. FCIP In The Data Center

  28. FibreChannel • How SCSI Limitations are Addressed • Speed • Distance • Device Count • Access

  29. FibreChannel – Speed • 266 Mbps – ten years ago • 1063 Mbps – common in 1998 • 2125 Mbps – available today • 4 Gbps – near future products • Backward compatible to 1 & 2 Gbps • 10 Gbps – 2005? • Not backward Compatible with 1/2/4Gbps • But 10 Gig Ethernet will compete • Remember FDDI & ATM

  30. Why I/O Protocols are Coming to IP • IP Networking is ubiquitous • Gigabit ethernet is here • 10Gbps ethernet is just becoming available • Don’t have to invest in a second network • Just upgrade the one you have • IP & Ethernet software is well understood • Existing talent pool for vendors to leverage • Developers, not end-user Network Engineers

  31. FibreChannel – Distance • 1063 Mbps • 175m (62.5 um – multi-mode) • 500m (50.0 um – multi-mode) • 10 km (9 um – single-mode) • 2125 Mbps • 500m (50.0 um – multi-mode) • 2 km (9 um – single-mode)

  32. FibreChannel – A Network • Layer 1 – Physical (Media: fiber, copper) • Fibre: 62.5, 50.0, & 9.0 um • Copper: Cat6, Twinax, Coax, other • Layer 2 – Data Link (Network Interface & MAC) • WWPN: World Wide Port Name • WWNN: World Wide Node Name • In a single port node, usually WWPN = WWNN • 64-bit device address • Comparable to 48-bit Ethernet device addresses • Layer 3 – Network (IP & SCSI) • 24-bit fabric address • Comparable to an IP address

  33. FibreChannel Terminology: Port Types • N_Port • Node port – Computer, Disk, or Storage Node • F_Port • Fabric port – Found only on a Switch • E_Port • Expansion Port – Switch to Switch port • NL_Port • Node port with Arbitrated Loop Capabilities • FL_Port • Fabric port with Arbitrated Loop Capabilities • G_Port • Generic Switch Port: Can act as any of F_Port, E_Port, or FL_Port

  34. FibreChannel - Topology • Point-to-Point • Arbitrated Loop • Fabric

  35. FibreChannel – Point-to-point • Direct Connection of Server and Storage Node • Two N_Ports and One Link

  36. FibreChannel - Arbitrated Loop • Up to 126 Devices in a Loop via NL_Ports • Token-access, Polled Environment (like FDDI) • Wait For Access Increases with Device Count

  37. FibreChannel - Fabric • Arbitrary Topology • Requires At Least One Switch • Up to 15 million ports can be concurrently logged in with the 24-bit address ID. • Dedicated Circuits between Servers & Storage • via Switches • Interoperability Issues Increase With Scale

  38. FibreChannel – Device Count • 126 devices in Arbitrated Loop • 15 Million in a fabric (24-bit addresses) • Bit 0-7: Port or Arbitrated Loop addr • Bit 8-15: Area, identifies FL_Port • Bit 16-23: Domain, address of switch 239 of 256 address available • 256 x 256 x 239 = 15,663,104

  39. FibreChannel Definitions • WWPN • Zone & Zoning • LUN • LUN Masking

  40. FibreChannel - WWPN • World-Wide Port Number • A unique 64-bit hardware address for each FibreChannel Device • Analogous to a 48-bit ethernet hardware address • WWNN - World-Wide Node Number

  41. FibreChannel – Zone & Zoning • Switch-Based Access Control • Analogous to an Ethernet Broadcast Domain • Soft Zone • Zoning based on WWPN of Nodes Connected • Preferred • Hard Zone • Zoning Based on Port Number on Switch • to which the Nodes are Connected

  42. FibreChannel - LUN • Logical Unit • Storage Node Allocates Storage and Assigns a LUN • Appears to the server as a unique device (disk)

  43. FibreChannel – LUN Masking • Storage Node Based Access Control List (ACL) • LUNs and Visible Server Connections (WWPN) are allowed to see each other thru the ACL. • LUNs are Masked from Servers not in the ACL

  44. LUN Security • Host Software • HBA-based • firmware or driver configuration • Zoning • LUN Masking

  45. LUN Security • Host-based & HBA • Both these methods rely on correct security implemented at the edges • Most difficult to manage due to large numbers and types of servers • Storage Managers may not be Server Managers • Don’t trust the consumer to manage resources • Trusting the fox to guard the hen house

  46. LUN Security • Zoning • An access control list • Establishes a conduit • A circuit will be constructed thru this • Allows only selected Servers see a Storage Node • Lessons learned • Implement in parallel with LUN Masking • Segregate OS types into different Zones • Always Promptly Remove Entries For Retired Servers

  47. LUN Security • LUN Masking • The Storage Node’s Access Control List • Sees the Server’s WWPN • Masks all LUNs not allocated to that server • Allows the Server to see only its assigned LUNs • Implement in parallel with Fabric Zoning

  48. LUN - Persistent Binding • Persistent Binding of LUNs to Server Device IDs • Permanently assign a System SCSI ID to a LUN. • Ensures the Device ID Remains Consistent Across Reconfiguration Reboots • Different HBAs use different binding methods & syntax • Tape Drive Device Changes have been a repeated source of NetBackup Media Server Failure