1 / 33

Belle Computing System

Belle Computing System. Ichiro Adachi KEK representing for computing & DST/MC production group ACAT03, KEK, 2003.Dec.02. Outline. Introduction Software Computing Model DST/MC Processing Data Management Plan & Summary. Introduction. Belle Experiment. B-factory experiment at KEK.

Download Presentation

Belle Computing System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Belle Computing System Ichiro Adachi KEK representing for computing & DST/MC production group ACAT03, KEK, 2003.Dec.02 ACAT03, 2003.Dec.02

  2. Outline • Introduction • Software • Computing Model • DST/MC Processing • Data Management • Plan & Summary ACAT03, 2003.Dec.02

  3. Introduction ACAT03, 2003.Dec.02

  4. Belle Experiment • B-factory experiment at KEK • Asymmetric energy e+e- collider: KEKB • Explore CP violation and flavor physics in B mesons KEKB ring Belle detector ACAT03, 2003.Dec.02

  5. Belle Detector Silicon Vertex Detector 3 layers of DSSD for vertexing ToF counter P-ID Aerogel Chrenkov Counter π/K seperation Central Drift Chamber tracking & dE/dx CsI(Tl) Calorimeter photon and electrons KLM muon & KL catcher Superconduncting solenoid of 1.5T ACAT03, 2003.Dec.02

  6. CPU power Network Storage system Management computing environment Belle’s Achievements 2001: Observation of large CP violation in B meson system 2002: Evidence of CP-violating asymmetries in B0π+π- 2003: Indication of new physics from B0Ks lots of B mesons ACAT03, 2003.Dec.02

  7. More data • KEKB has achieved design luminosity of 1034cm-2sec-1 • 10 B meson pairs/sec • Accumulated more than 160fb-1 of data so far 160fb-1 Largest B meson data sample at  energy region ACAT03, 2003.Dec.02

  8. Basic Requirements • Beam data should be available for physics analyses in a couple of months • Software updates can be reflected onto physics analysis immediately • Physics outputs in a timely manner • MC sample at least 3 times larger than beam data • Analysis technique gets matured with a large statistics of beam data • Systematic study needs more statistics in MC sample ACAT03, 2003.Dec.02

  9. Basic Requirements(cont’d) • Software • stable & robust • Computing Model • efficient • expandable • Performance • data availability for analyses ACAT03, 2003.Dec.02

  10. Software ACAT03, 2003.Dec.02

  11. Event flow Input with panther shared object unpacking calibration module tracking vertexing loaded dynamically B.A.S.F. clustering particle ID diagnosis Output with panther Software Tools • Home-made kits • “B.A.S.F.” for framework • Belle AnalySis Framework • unique framework for any step of event processing • event-by-event parallel processing on SMP • “Panther” for I/O package • unique data format from DAQ to user analysis • bank system with zlib compression • reconstruction & simulation library • written in C++ • Other utilities • CERNLIB/CLHEP… • PostgreSQL for database ACAT03, 2003.Dec.02

  12. Computing Model ACAT03, 2003.Dec.02

  13. 2002 2003 Mar Super-SINET to Univ’s Disk Storage (60TB) User PC farms KEKB computers from 2001 Feb 2003 Jun 10 login servers for User PC Farms Computing Model Overview Fast network (1~10Gbps) 4Gbps ACAT03, 2003.Dec.02

  14. PC farms Sun computing server Fujitsu GbE switch tape library 500TB Computing network for batch jobs and DST/MC production 500MHz*4 38 hosts Compaq online tape server GbE switch work group server super-sinet 500MHz*4 HSM server 1Gbps 9 hosts Tokyo Nagoya Tohoku GbE switch file server 8TB University resources user PC disk 4TB 1GHz 100hosts User analysis & storage system HSM library 120TB KEKB Computers ACAT03, 2003.Dec.02

  15. 2002 2003 1999 2001 PC farms • heterogeneous system from various vendors • cost effectiveness • 3 types of CPU(Pen3/Xeon/Athlon) Dell 36PC’s 0.5GHz@P3 Fujitsu 127PC’s 1.26GHz@P3 NEC 84PC’s 2.8GHz@Xeon Compaq 60PC’s 0.7GHz@Xeon Appro 113PC’s “1.67GHz”@Athlon processor clock speed ACAT03, 2003.Dec.02

  16. Sun CPU 9 servers(0.5GHz*4CPU) 38 computing servers(ibid.) tape drives(2 each for 20hosts) Linux CPU 60 Compaq servers(Intel Xeon, 0.7GHz*4CPU) 127 Fujitsu servers(P3, 1.26GHz dual) 113 Appro servers(Athlon, 1.67GHz dual) 84 NEC servers(P3, 2.8GHz dual) Disk servers & storage Tape library DTF2 tape(200GB), 24MB/s IO 500TB total 40 tape drives 8TB NFS file servers 120TB HSM servers 4TB staging disk 2 servers for 60TB disk CPU & Disk Storage ACAT03, 2003.Dec.02

  17. User PC farm & Disk Storage PC farm data job Disk storage notice LSF scheduler public beam data MC data 84 PC’s with dual Xeon 2.8GHz CPUs Local disk CPU utilization 6TB user data histograms Login servers debugging user code ACAT03, 2003.Dec.02

  18. Super-SINET at Belle • Disks located at Nagoya (~350km away from KEK) are NFS-mounted to the KEK host • Directly write data onto such disks from batch jobs running at KEK computer • superSINET also used for copying a various type of data full recon bs J/ inclusive D*s KEK hadronic sample Nagoya 350km ACAT03, 2003.Dec.02

  19. DST/MC Processing ACAT03, 2003.Dec.02

  20. DST Production & Skimming Scheme 1. Production(reproduction) raw data data transfer Sun DST data PC farm disk histograms log files 2. Skimming disk or HSM user analysis skims such as hadronic data sample Sun histograms log files DST data disk ACAT03, 2003.Dec.02

  21. Processing Power & Failure Rate • Processing power • Processing ~1fb-1 per day with 180GHz • Allocate 40 PC hosts(0.7GHzx4CPU) for daily production to catch up with DAQ • 2.5fb-1 per day possible • Processing speed(in case of MC) with 1GHz one CPU • Reconstruction: 3.4sec • Geant simulation: 2.3sec • Failure rate for one B meson pair ACAT03, 2003.Dec.02

  22. Reprocessing Snapshot Lprocessed/day(pb-1) exp27 exp25 gap: waiting for constants ACAT03, 2003.Dec.02

  23. Performance:beam data processing • All data including a final bit of beam data have been always processed and been used for analyses 2003 summer 159fb-1 3months 2002 summer 78fb-1 2.5months 2001 summer 30fb-1 ACAT03, 2003.Dec.02

  24. MC Production • MC sample • 3 times bigger statistics • Run dependence taken into account min. set of generic MC Run# xxx 3 files B0 MC data Run# xxx beam data file B+B- MC data run-dependent background IP profile charm MC data light quark MC ACAT03, 2003.Dec.02

  25. MC Production(cont’d) • PC farms at KEK shared with DST processing • Switching to MC production can be made easily • MC samples for 159 fb-1 has been completed in November 2003 2.2 billion events 2003 2002 Library minor update Library major update Library minor update ACAT03, 2003.Dec.02

  26. MC Production at Remote Sites GHz • Total CPU resources at remote sites amounts to ~600GHz • 14% of MC events has been produced outside KEK • All data have been transferred to KEK via network ~600GHz 14% at remote sites ACAT03, 2003.Dec.02

  27. Data Management ACAT03, 2003.Dec.02

  28. Data Management • 20K files for beam runs • 240K files for run-dependent MC data User has to go through those to get final results File information are stored in postgreSQL database ~”meta data” data files SQL database inquire read inquire command in batch job user access job submit Web based interface answer ACAT03, 2003.Dec.02

  29. trouble Data Management(cont’d) • File info centralized and uniquely managed • Easy to change if necessary • Disk failure etc SQL database administrator ACAT03, 2003.Dec.02

  30. Plan & Summary ACAT03, 2003.Dec.02

  31. Software Plan for 2004 • Belle detector upgraded this summer • Silicon vertex detector • 4 layers( from 3 layers ) DSSD • smaller inner radius & wider acceptance • New inner chamber • Cathode part replaced into new chamber • Real-time processing • Refer to talk by Itoh san(Dec/4 session2) • Need to update reconstruction software • Calibration constants newly determined • Tuning is underway • Reprocess Belle phase-I data of 159fb-1 • Under discussion ACAT03, 2003.Dec.02

  32. Prospects • We are in the stage of O(PB) scale • More than 2500 DTF2 tapes • Will record another 100fb-1 by 2004 summer • Obviously increase data • Super B-factory project? • Data storage as well as management can be a big issue ACAT03, 2003.Dec.02

  33. Summary • Our computing system has been working well • Processing beam data as well as MC data have been successfully done • Proven up to 160fb-1 ACAT03, 2003.Dec.02

More Related