Cardiff University’s Condor Pool

1. Cardiff University’s Condor Pool Industrial Perspectives James Osborne

2. Contents • Introduction • Application Diversity • English Heritage • Velindre Cancer Centre • Full Economic Costing • Charging Models • Questions

3. Introduction • 1 Central Manager RHEL 3 • 800 Windows XP SP2 • 30 Submit Nodes Max (IP Address) • 200 Execute Nodes (Execute Always) • 600 Execute Nodes (Execute Idle) • Condor Installed Via Application Object • Condor 6.6.11 ~ 60% + 6.6.2 6.6.9 6.6.10

4. Architecture Energy Plus Biosciences DB, BLAST PAUP, Structure Chemistry Dmarel Computer Science Image Processing Radiotherapy Engineering Travelling Salesman History and Archaeology Artefacts Longbarrows Application Diversity

5. Mathematics CFD of Polymers Optometry Macromolecules Application Diversity

6. English Heritage

7. English Heritage • Collaboration with HISAR • Alex Bayliss from English Heritage • Very difficult to track her down • Artefacts ~ 300BC • Prof John Hines • Longbarrows ~ 3700BC • Prof Alasdair Whittle

8. Condorised Application • Oxcalc • Developed by Christoper Ramsey • Developed at University of Oxford • We use version 3.10 • Oxcalc interprets radiocarbon data using Bayesian statistics to date archaeological artefacts based on their carbon isotope ratios

9. Data Collection • Scientist takes a sample of the artefact • The sample is heated until it vaporises • The vapour is then analysed • The ratio of C12 to C14 is measured

10. Artefacts

11. Artefacts – Sensitivity • English Heritage still collecting the data • Data will be available in September 2006 • 300 – 400 artefacts (jewellery, pottery, tools) • 1 hour per job on a P4 • 12 – 16 days on a single workstation (E) • 1 hour using the entire pool (E)

12. Longbarrows

13. Longbarrows - Stability • English Heritage have collected the data • 6 Longbarrows in Southern England • 80 – 100 models per Longbarrow • 1 hour per job on a P4 • 20 – 25 days on a single workstation (E) • 1 hour using the entire pool (E) • ~1.5 hours total (A)

14. The Future • English Heritage have already paid us • EH + HISAR share an internal budget code • Via TRF into our internal condor account • Paid £500 for 25,000 CPU Hours • Currently used < 10% of those CPU Hours • EH work with other institutions • EH intend to use us in the future with others

15. Velindre Cancer Centre

16. Velindre Cancer Centre • Collaboration with COMSC • Mary Chin and Geraint Lewis from Velindre • Radiotherapy • Prof David Walker • Jon Giddy

17. Condorised Applications • BeamNRC • Simulates x-rays emitted from radiotherapy machines based on plan files • DosXYZNRC • Splits radiotherapy plan files (pre Condor) • Joins BeamNRC results files (post Condor)

18. Radiotherapy Machines

19. Condorised Application • Radiotherapist provides plan file • Radiotherapist runs Perl script • Perl script calls DosXYZNRC to split plan file • Perl script writes 18 submit files • Radiotherapist runs condor_submit • 18 beams, 90 pieces, 500,000 events • 1620 jobs per patient per plan • 1.5 hours per job on a P4 (A)

20. Condorised Application • 3.5 months on a single workstation (E) • 3 hours using the entire pool (E) • ~12 hours total (A) • ~10 Gigabytes of results • Radiotherapist runs another Perl script • Perl script calls DosXYZNRC to join results • Radiotherapist evaluates the results

21. The Future • Velindre have agreed to paid us • V + COMSC share an internal budget code • Via TRF into our internal condor account • Will pay £500 for 25,000 CPU Hours • Currently used ~ 30% of those CPU Hours • Future potential for 1400 patients / year • 3.4 million hours using the entire pool (E) • 6 months using the entire pool (E)

22. Full Economic Costing

23. Background • Full economic costing is a hot topic • Some guidelines available from • EPSRC, HEFCE, HEFCW JISC… • Some guidelines from other universities • No guidelines for HPC or HTC • Funding bodies require FEC in new grants • FEC of Condor was a pre-emptive strike

24. Categories • Buildings • Equipment • Maintenance • Power • Staff

25. FEC for OAWS • Buildings • Equipment • Execute and submit nodes and networking • Maintenance • Equipment warranties • Power • Staff

26. FEC for Condor • Equipment • Central manager • Maintenance • Equipment warranty • Power • Additional power consumed • Staff • Additional staff required

27. Equipment • Dell 1750 • Equipment cost £2240.37 • Racking cost £1000.00 pa • Rack space, UPS connection, 100mbit network connection, power, and air conditioning • Total cost £1560.10 pa • Assuming a 4 year update cycle

28. Additional Power • Measure power consumption • When Idle • When running a job using max CPU • When running a job using max DISK • Calculate additional power consumed • Max Condor = • (Max CPU – Idle) + • (Max DISK – Idle)

29. Watts Up Pro • Measures • Watts, Volts, Amps, WattHrs, Cost, Avg Kwh, Mo Cost, Max Wts, Max Vlt, Max Amp, Min Wts, Min Vlt, Min Amp, Pwr Fct, Dty Cyc, Pwr Cyc • Freq – 1 second • Duration – 15 minutes

30. Power Consumption

31. Pool Power Consumption

32. Additional Staff • Condor support engineer £ 24.26 ph • Administrative assistant £ 21.08 ph

33. Charging Models

34. Internal Charging

35. External Charging

36. Conclusions • Still early days • Only two industrial partners • Only one EH has actually paid us • Full economic costing complete • Charging policy 90% complete

37. Questions ?