1 / 17

The Computing System for the Belle Experiment

The Computing System for the Belle Experiment . Ichiro Adachi KEK representing the Belle DST/MC production group CHEP03, La Jolla, California, USA March 24, 2003. Introduction: Belle Belle software tools Belle computing system & PC farm DST/MC production Summary. Introduction.

shanae
Download Presentation

The Computing System for the Belle Experiment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Computing System for the Belle Experiment Ichiro Adachi KEK representing the Belle DST/MC production group CHEP03, La Jolla, California, USA March 24, 2003 • Introduction: Belle • Belle software tools • Belle computing system & PC farm • DST/MC production • Summary

  2. Introduction • Belle experiment • B-factory experiment at KEK • study CP violation in B meson system. start from 1999 • recorded ~120M B meson pairs so far • KEKB accelerator is still improving its performance 120fb-1 The largest B meson data sample at (4s) region in the world Ichiro Adachi, CHEP03

  3. Belle detector example of event reconstruction fully reconstructed event Ichiro Adachi, CHEP03

  4. Belle software tools Event flow • Home-made kits • “B.A.S.F.” for framework • Belle AnalySis Framework • unique framework for any step of event processing • event-by-event parallel processing on SMP • “Panther” for I/O package • unique data format from DAQ to user analysis • bank system with zlib compression • reconstruction & simulation library • written in C++ • Other utilities • CERNLIB/CLHEP… • Postgres for database Input with panther shared object unpacking calibration module tracking vertexing loaded dynamically B.A.S.F. clustering particle ID diagnosis Output with panther Ichiro Adachi, CHEP03

  5. PC farms Sun computing server Fujitsu GbE switch tape library 500TB Computing network for batch jobs and DST/MC production 500MHz*4 38 hosts Compaq online tape server GbE switch work group server super-sinet 500MHz*4 HSM server 1Gbps 9 hosts Tokyo Nagoya Tohoku GbE switch file server 8TB University resources user PC disk 4TB 1GHz 100hosts User analysis & storage system HSM library 120TB Belle computing system Ichiro Adachi, CHEP03

  6. Computing requirements Reprocess entire beam data in 3 months Once reconstruction codes are updated or constants are improved, fast turn-around is essential to perform physics analyses in a timely manner MC size is 3 times larger than real data at least Analyses are getting matured and understanding systematic effect in detail needs large MC sample enough to do this Added more PC farms and disks Ichiro Adachi, CHEP03

  7. Will come soon Delivered in Dec.2002 boost up CPU power for DST & MC productions PC farm upgrade Total CPU = CPU processor speed(GHz)  # of CPUs  # of nodes Total CPU(GHz) ~1500GHz Total CPU has become 3 times bigger in recent two years 60TB(total) disks have been also purchased for storage Ichiro Adachi, CHEP03

  8. Belle PC farm CPUs • heterogeneous system from various vendors • CPU processors(Intel Xeon/PenIII/Pen4/Athlon) Dell 36PCs (Pentinum-III ~0.5GHz) NEC 84PCs (Pentium4 2.8GHz) will come soon 168GHz 470GHz Compaq 60PCs (Intel Xeon 0.7GHz) Appro 113PCs (Athlon 2000+) 320GHz 380GHz setting up done Fujitsu 127PCs (Pentium-III 1.26GHz) Ichiro Adachi, CHEP03

  9. DST production & skimming scheme 1. Production(reproduction) raw data data transfer Sun DST data PC farm disk histograms log files 2. Skimming disk or HSM user analysis skims such as hadronic data sample Sun histograms log files DST data disk Ichiro Adachi, CHEP03

  10. hadronic mini-DST full recon J/ inclusive D*s bs Output skims • Physics skims from reprocessing • “Mini-DST”(4-vectors) format • Create hadronic sample as well as typical physics channels(up to ~20 skims) • many users do not have to go through whole hadronic sample. • Write data onto disk at Nagoya(350Km away from KEK) directly using NFS(thanks to super-sinet link of 1Gbps) mini-DST reprocessing output 1Gbps Nagoya ~350Km from KEK KEK site Ichiro Adachi, CHEP03

  11. Processing power & failure rate • Processing power • Processing ~1fb-1 per day with 180GHz • Allocate 40 PC hosts(0.7GHzx4CPU) for daily production to catch up with DAQ • 2.5fb-1 per day possible • Processing speed(in case of MC) with 1GHz one CPU • Reconstruction: 3.4sec • Geant simulation: 2.3sec • Failure rate for one B meson pair Ichiro Adachi, CHEP03

  12. Reprocessing 2001 & 2002 • Reprocessing • major library & constants update in April • sometimes we have to wait for constants • Final bit of beam data taken before summer shutdown always reprocessed in time 3months For 2002 summer 78fb-1 2.5months For 2001 summer 30fb-1 Ichiro Adachi, CHEP03

  13. MC production • Produce ~2.5fb-1 per day with 400GHz PenIII • Resources at remote sites also used • Size 15~20GB for 1 M events. • 4-vector only • Run dependent min. set of generic MC Run# xxx B0 MC data Run# xxx beam data file mini-DST B+B- MC data run-dependent background IP profile charm MC data light quark MC Ichiro Adachi, CHEP03

  14. MC production 2002 • Keep producing MC generic samples • PC farm shared with DST • Switch from DST to MC production can be made easily • Reached 1100M events in March 2003. 3 times larger samples of 78fb-1 completed minor change major update Ichiro Adachi, CHEP03

  15. MC production at remote sites GHz CPU resource available • Total CPU resources at remote sites is similar to KEK • 44% of MC samples has been produced at remote sites • All data transferred to KEK via network • 6~8TB in 6 months ~300GHz MC events produced 44% at remote sites Ichiro Adachi, CHEP03

  16. Future prospects • Short term • Software:standardize utilities • Purchase more CPUs and/or disks if budget permits… • Efficient use of resources at remote sites • Centralized at KEK  distributed over Belle-wide • Grid computing technology… just started survey & application • Date file management • CPU usage • SuperKEKB project • Aim 1035(or more) cm-2s-1 luminosity from 2006 • Phys.rate ~100Hz for B-meson pair • 1PB/year expected • New computing system like LHC experiment can be a candidate Ichiro Adachi, CHEP03

  17. Summary • The Belle computing system has been working fine. More than 250fb-1 of real beam data has been successfully (re)processed. • MC samples with 3 time larger than beam data has been produced so far. • Will add more CPU in near future for quick turn-around as we accumulate more data. • Grid computing technology would be a good friend of ours. Start considering its application in our system. • For SuperKEKB, we need much more resources. May have rather big impact in our system. Ichiro Adachi, CHEP03

More Related