1 / 8

Weekly Report LSF API Timing Tests

Weekly Report LSF API Timing Tests. Summary of the week. Meeting with Tim Bell (IT) about accounting systems, allocation requests

elsie
Download Presentation

Weekly Report LSF API Timing Tests

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Weekly ReportLSF API Timing Tests P. Nilsson

  2. Summary of the week • Meeting with Tim Bell (IT) about accounting systems, allocation requests • Possible to use (e.g.) LSF, Condor, Torque-Maui for PROOF accounting without using the full machinery of the batch systems. The PROOF master can then query the “batch system” master for resource information • Bell recommended us to start with LSF. IT can help us set up LSF on the PROOF cluster (1-2 days of work, cluster already under LSF license) • LSF API • How to use it for allocation requests (see next few slides) P. Nilsson

  3. LSF API Example • How to request a number of hosts for a task • Example: Ask for two “suitable” machines with atlas user environment interactively The equivalent API function is ls_placereq() > lsplace -R atlas -n 2 lxb5509 lxb5510 char resreq[MAXLINELEN]; // resource requirement expression int num = 2; sprintf(resreq,"%s","atlas"); char **host = ls_placereq(resreq, &num, 0, NULL); P. Nilsson

  4. Resource Names (Used to build the resource requirement expression) RESOURCE_NAME TYPE ORDER DESCRIPTION r15s Numeric Inc 15-second CPU run queue length r1m Numeric Inc 1-minute CPU run queue length (alias: cpu) r15m Numeric Inc 15-minute CPU run queue length ut Numeric Inc 1-minute CPU utilization (0.0 to 1.0) pg Numeric Inc Paging rate (pages/second) io Numeric Inc Disk IO rate (Kbytes/second) ls Numeric Inc Number of login sessions (alias: login) it Numeric Dec Idle time (minutes) (alias: idle) tmp Numeric Dec Disk space in /tmp (Mbytes) swp Numeric Dec Available swap space (Mbytes) (alias: swap) mem Numeric Dec Available memory (Mbytes) lftm Numeric Dec Minutes until next shutdown pool Numeric Dec Free MB in work directories maxpool Numeric Dec Max size of pool disk cms Boolean N/A CMS DC04 tests atlas Boolean N/A ATLAS CD alice Boolean N/A ALICE MDC … … … … + many more (full list with the lsinfo command) See separate word document P. Nilsson

  5. Timing of request calls • Timing performed with external time command Requesting one host [lxplus006] > time lsplace -R atlas -n 1 lxb5510 0.000u 0.010s 0:00.02 50.0% 0+0k 0+0io 197pf+0w Requesting 20 hosts [lxplus006] > time lsplace -R atlas -n 20 lxb5510 lxb5510 lxb5504 lxb5504 lxb5501 lxb5501 lxb5507 lxb5507 lxb5506 lxb5506 lxb5508 lxb5508 lxb5502 lxb5502 lxb5503 lxb5503 lxb5505 lxb5505 lxb5509 lxb5509 0.000u 0.000s 0:00.02 0.0% 0+0k 0+0io 197pf+0w P. Nilsson

  6. Timing of request calls • Timing performed with Gerri’s TMeasure class Requesting 20 hosts with same resource requirement expression (“atlas”) using the API [lxplus006] > hostreq lxb5510 lxb5510 lxb5504 lxb5504 lxb5503 lxb5503 lxb5507 lxb5507 lxb5502 lxb5502 lxb5506 lxb5506 lxb5509 lxb5509 lxb5501 lxb5501 lxb5505 lxb5505 lxb5508 lxb5508 #-------------------- TMeasure ----------------------# # place: ResourceQuery # # results (seconds): # 0.01 Done requesting #----------------------------------------------------# P. Nilsson

  7. Timing of request calls • Varying the number of requests (same simple requirement resource expression) No of requests Time (s) 1 0.01 100 1.9 500 10.0 1000 18.1 P. Nilsson

  8. Timing of request calls • More complex requirement resource expression resreq = "atlas:type=local:cpu=.1:ls=2:mem=1024“ resreq = atlas && (type == local) && (cpu <= .1) && (ls <= 2) && (mem >= 1024) No of requests Time (s) 1 0.02 100 2.5 500 13.1 1000 25.0 P. Nilsson

More Related