OptIPuter Southern CaliforniaNetwork Infrastructure Philip Papadopoulos OptIPuter Co-PI University of California, San Diego Program Director, Grids and clusters San Diego Supercomputer Center January 2004
Building an Experimental Apparatus • Mod-0 Optiputer Ethernet (Packet) Based • Focused as an Immediately-usable High-bandwidth Distributed Platform • Multiple Sites on Campus ( a Few Fiber Miles ) • Next-generation Highly-scalable Optical Chiaro Router at Center of Network • Hardware Balancing Act • Experiments Really Require Large Data Generators and Consumers • Science Drivers Require Significant Bandwidth to Storage • OptIPuter Predicated on Price/performance curves of > 1GE networks • System Issues • How does one Build and Manage a Reconfigurable Distributed Instrument?
Aggregates • Year 1 (Network Build) • Chiaro Router Purchased, Installed, Working (Feb) • 5 sites on Campus. Each with 4 GigE Uplinks to Chiaro • Private Fiber, UCSD-only. • ~40 Individual nodes, Most Shared with Other Projects • Endpoint resource poor. Network Rich • Year 2 (Endpoint Enhancements) • Chiaro Router – Additional Line Cards, IPV6, Starting 10GigE Deployment • 8 Sites on Campus • h 3 Metro Sites • Multiple Virtual Routers for Connection to Campus, CENIC HPR, others • > 200 Nodes. Most are Donated (Sun and IBM) . Most Dedicated to OptIPuter • Infiniband Test Network on 16 nodes + Direct IB Switch to GigE • Enough Resource to Support Data-intensive Activity, • Slightly network poor. • Year 3 + (Balanced Expansion Driven by Research Requirements) • Expand 10GigE deployments • Bring Network, Endpoint, and DWDM (Mod-1) Forward Together • Aggregate at Least a Terabit (both Network and Endpoints) by Year 5
Web Information on the SD OptIPuter • http://web.optiputer.net • Need folks to start using resources and feedback how things can work better • Intention is give full control of resources to experiments • Experiments themselves should be of defined timeframe • This is a shared instrument • The resource endpoints are experimental • NO BACKUPS • DO NOT EXPECT 7/24 (We don’t have the staff) • THINGS WILL BREAK
High-Level Program Bullets • UCSD will complete deployment of a 150+ node distributed test bed that consists of compute, storage, visualization, and instrument endpoints. Management policies will be put in place so that experiments can be assigned physical hardware and then have specialized (experiment-specific) software loaded on assigned nodes. This forms the Core Southern California OptIPuter Test bed • In year 2, we won an IBM SURS Grant which allowed us to define a larger storewidth evaluation platform than in the original program plan. This cluster, deployed in January 2004 is 48 nodes with 6 spindles per node. This enables middleware and applications to understand how lambdagrids enable applications. • As part of the SURS grant a small IB test fabric was purchased. The Topspin switch includes 4 gigE uplinks to allow investigation of IB to Ethernet communication.