1 / 17

Shown many data transfers (few are simultaneously going on)

Kolkata Tier-2 and Network. Shown many data transfers (few are simultaneously going on). Site Name :- IN-DAE-VECC-02 Monalisa Name:- Kolkata-Cream VO :- ALICE City:- KOLKATA Country :- INDIA. ALICE Collaboration ~ 1/2 ATLAS, CMS, ~ 2x LHCb ~1100 people 30 countries, 80 Institutes.

mkimble
Download Presentation

Shown many data transfers (few are simultaneously going on)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Kolkata Tier-2 and Network Shown many data transfers (few are simultaneously going on) Site Name :- IN-DAE-VECC-02 Monalisa Name:- Kolkata-CreamVO :- ALICE City:- KOLKATA Country :- INDIA

  2. ALICE Collaboration • ~ 1/2 ATLAS, CMS, ~ 2x LHCb • ~1100 people • 30 countries, • 80 Institutes Total weight 10,000t Overall diameter 16.00m Overall length 25m Magnetic Field 0.4Tesla 8 kHz (160 GB/sec) level 0 - special hardware 200 Hz (4 GB/sec) level 1 - embedded processors 30 Hz (2.5 GB/sec) level 2 - PCs Introduction to ALICE 30 Hz (4 GB/sec) data recording & offline analysis Lots of data to Process:- Solution GRID COMPUTING Slide from Latchezar Betev PPT

  3. Tier 0 ALICE Grid Structure CERN computer centre, Geneva CAF (T1 +T2) Central Analysis Facility Experimental site 10 -100 Gbps links ASIA (KISTI) TAIWAN Germany Italy France Tier 1 8 centres 1 -10 Gbps links Tier 2 India China JAPAN Pakistan Logical, geographical association of regional groups IN_DAE_VECC_02 100 -1000 Mbps links Panjab U. Jaipur U. Jammu U. EHEPAG (VECC) Tier 3 Different Universities, Institutes Individual scientist’s PC

  4. Kolkata Tier-2 at Monalisa Consistently 800 ALICE Jobs are running. Vikas Singhal, VECC, INDIA

  5. Grid Computing Facility Vikas Singhal, VECC, INDIA

  6. 3 M ALICE Job completed Successfully during last 5 Years • Total :- 888 Cores DELL Quad Core Blades 38 * 2 * 4 *2 = 608 (HT) • HP Quad Core Blades 8 * 2 * 4 * 2 = 128 (HT) • HP Dual Core Blades 38 * 4 = 156 • Equivalent 9K HEPSpec 2006 Computing Resources. • Storage :- 146 TB • XrootD Redirector • 3 Disk Servers SAN Based • 78TB (IBM) + (36 + 32)TB (HP EVA) • 1Gbps WAN Network speed. • 10Gbps Backbone Network Vikas Singhal, VECC, INDIA

  7. NKN @ INDIA

  8. NKN TOPOLOGY April 22, 2009 Developing e-inftrastructures in India

  9. Physical Network Connectivity upto Kolkata CERN LHCONE 10 Gbps TEIN-3 Network 4 Gbps/10 Gpbs Shared TIFR Mumbai 10 Gbps 10 Gbps (Under going) Internet 1 Gbps POP in Mumbai 10 Gbps 10 Gbps Core Network 10 Gbps SINP Kolkata POP in Kolkata 1 Gbps 10 Gbps 1 Gbps 1 Gbps 4-10 Gbps 1 Gbps VECC Kolkata 1 Gbps Kolkata Tier-2 x Gbps

  10. 10 Gigabit Fiber Backbone NKN Router at VECC 1Gb optical Fiber Link Grid Computing Facility 1 Gigabit Ethernet Link XrootD Redirector gridce02 Grid-peer grid DNS Server XrootD Disk Servers 10 Gigabit Fiber Link HP IBM SAN Storage ALICE Tier-2 Worker nodes Tier-3 Nodes ALICE Centre

  11. Kolkata TIER-2 centre Network diagram Internet 1Gbps Copper Router 1Gbps Fiber Backbone SINP 144.16.112.xx/27 Switch naamak Dache-server Installer grid01 Backup-server Grid-peer grid gridse001 gridce02 4 – Xrootd Disk Servers 130 TB Backup wn001 wn045 Switch-1 wn002 wn046 Switch-2 10G Backbone Switch 10G Backbone Switch 25 Nodes Dell and Wipro Blades Cluster with 25 TB of As Tier-3 DELL and HP Blade Server with Multi Core Xeon 3.0 GHz Computing Nodes Computing Nodes 192.168.x.x (Stand by) 10 Gbps wn024 wn122 10 Gbps wn025 wn123 IN-DAE-VECC-02 Site with 64 bit machine GRID-PEER Tier-3 cluster with 32 & 64 bit machine Vikas Singhal, VECC, INDIA

  12. From 128Kbps to 1Gbps WAN Network Started with ----128Kbps shared link 2002 ----512Kbps 2003 ----2Mbps Dedicated Link 2004 ----4Mbps from Bharti 2006 ----30Mbps from Reliance 2008 ----100Mbps from VSNL (ERNET) 2009 ----300 Mbps from NKN 2011 ----1Gpbs 2014 Vikas Singhal, VECC, INDIA

  13. Kolkata SE Traffic # cat route-eth0default via 192.168.1.12 dev eth0144.16.112.16 via 192.168.1.16 dev eth0144.16.112.15 via 192.168.1.15 dev eth0144.16.112.14 via 192.168.1.14 dev eth0144.16.112.9 via 192.168.1.9 dev eth0 Vikas Singhal, VECC, INDIA

  14. Tier-3 (grid-peer) cluster HP c7000e Blade enclosure. 8 Number of HP ProLiant DL585 G7 AMD Blade Each blade has 2 * 16 Core 2.5GHz AMD CPU Each blade has 256 GB RAM. Installed SLC 6.5 x86_64 OS 1U Sliding LCD Monitor with 16 port KVM Dell PowerEdge M1000e Blade Server Chassis. 16 Number of Dell PowerEdge M610 High Performance Intel Blade Each blade has Nehalem based 2 * Intel Quad Core E5530 Xeon 2.4GHz CPU with 8MB cache. Each blade has16GB RAM. Each blade has 2 * 146GB Mounted as RAID1. Installed SLC 6.5 x86_64 OS IBM SAN Storage 3 * 16 * 2TB SAS hard disks. 3 * 26 TB = 78 TB Usable space after RAID5 and Hot Spare. Total 14 Nodes for VECC users and Collaborating Institutes. 13 Collaborating Institutes. 64bit computing nodes. 7 Non-interactive AMD nodes. 7 Interactive AMD + Intel nodes. 78 TB IBM SAN storage. Storage mounted via NAS on all the Compute Nodes. 10Gbps Network backbone. Managing via ROCKS6. Using PBS Job Scheduler. 50 + active users (across India.) 30 + active users (in VECC.) Quota implemented for the users. Root, Geant3, Aliroot, Alien, Fairsoft, CBMroot, gFortran etc user specific software installed.

  15. Future Plan and Decisions • Decided to increase Kolkata Tier-2 resources as per ALICE Fair Share rule. • 10 Cooled Rack Space allotted. • Indent raised to procure more than 30K HEP Spec 2006 equivalent computing Power. • Will procure 1200 Physical cores or 2400 Hyperthreaded core. • 12 Core processors are the best suitable as per price performance point. • Plan to purchase only bare disks as per requirement. • 3 * 2U servers comprising 12*4TB disks configured under Open Source EOS storage, will be included into KOLKATA SE.

  16. Network Backbone Planning 10Gb backbone, 40Gb core switch connection. Both Fiber and Copper connectivity. Connection via DAC Cable. Dual path, Full Redundancy. Vikas Singhal, VECC, INDIA

  17. Thank You DURGA POOJA Festival of GODESS DURGA. Celebrated in Month of October. Idol will be submerged into the Water. Vikas Singhal, VECC, INDIA

More Related