1 / 43

The 3 rd Pan-Galactic

The 3 rd Pan-Galactic. Workshop. Schedule. 9:00 – 10:00 BOINC status report 10:00 – 10:15 Break 10:15 – 11:30 BOINC plans 11:30 – 12:30 Discussion 12:30 – 1:30 Lunch 1:30 – 2:15 Breakout session 1 2:30 – 3:15 Breakout session 2 3:30 – 4:00 reports

sona
Download Presentation

The 3 rd Pan-Galactic

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The 3rd Pan-Galactic Workshop

  2. Schedule 9:00 – 10:00 BOINC status report 10:00 – 10:15 Break 10:15 – 11:30 BOINC plans 11:30 – 12:30 Discussion 12:30 – 1:30 Lunch 1:30 – 2:15 Breakout session 1 2:30 – 3:15 Breakout session 2 3:30 – 4:00 reports 4:00 – 5:00 Wish list, discussion, wrap-up

  3. BOINC status reportDavid P. Andersondavea@ssl.berkeley.eduSept. 6, 2007

  4. Outline • The state of volunteer computing • The state of BOINC • What’s happened since the last workshop • and what hasn’t happened • What’s in the works • What’s in the future

  5. The state of volunteer computing • Folding@home leads the way • 200 TFLOPS from PCs, 50 TFLOPS from GPUs, 600 TFLOPS from PS3s • Storage@home • BOINC-based activity • ~40 projects • 500 TFLOPS from PCs • No big discoveries so far • Grid.org shuts down April 07 • IBM WCG transitions to BOINC • Volunteer population: stagnant • No major (> 10K hosts) projects in last year

  6. The state of BOINC • Über-goals • use all computers in the world for important work • enable new science • involve the public in science • BOINC’s role • provide software technology • catalyze/enable communities • Partners: projects, BAM!, GridRepublic, translators, alpha testers, volunteers, ... • Staff: me, Rom Walton, Charlie Fenton 50% • Contributors • Kevin Reed, John McLeod, Rytis Slatkevicius, Nicolas Alvarez, David Barnard, Frank Weiler, many others • Funding: new NSF grant, 9/07-8/10

  7. Requests for BOINC projects • If you need a new feature: • talk to me about it first • develop it yourself but stay in synch • Upgrade server software frequently • Participate in email lists • Include some BOINC funding in your grant proposals • Tell me about publications

  8. What’s happened since the last workshop?

  9. BOINC now uses trac • Trac bug-tracking system replaces BOINCzilla • Subversion replaces RCS • sandboxes • Milestones • Documentation is Wiki-fied • please participate!

  10. Facilitating project creation • VMware BOINC server • run on Windows/Mac/Linux • all software pre-installed, compiled • Wrapper application • handles sequence of “legacy” apps, which use stdin/stdout or named files • checkpointing at “sub-job” level • make_project improvements • reasonable config.xml defaults

  11. Client software releases • 5.6 (Sept 06) • new CPU scheduler • Mac sandbox • 5.8 (Dec 06) • Simple view • Snooze button • CPU throttling • memory usage prefs • 5.10 (May 07) • Project selection menu • Advanced prefs dialog • Grid-based views

  12. Multi-platform support • Some hosts can run >1 type of binary supported platforms: win64 win32 client scheduler application version number platform

  13. Homogeneous redundancy • Multiple HR types • each classifies hosts based on numerical properties • per-application HR types • Feeder reserves shared-memory slots per HR class in proportion to RAC • “Census” program computes ratios • Feeder DB queries scans all unsent results feeder shared mem win32 linux/x86 mac new scheduler . . . scheduler

  14. Load-aware scheduler job deadline CPU time remaining 1 17 4 2 35 12 ... client scheduler For each candidate job J: if run J with existing jobs, are any new deadlines missed (or missed by more)? If not, accept J

  15. Client simulator • Simulates 1 client and N projects • Uses actual client code • Web interface • Uses: • study new scheduling policies • debug client • optimize preferences for a particular client project description options graphical timeline host description message log simulator global_prefs.xml figures of merit: idle CPU wasted CPU share violation monotony cc_config.xml

  16. Recruitment and retention • Newsletter support • Reminder email support • dropouts • initial failures • Grass-roots PC magazine publicity campaign • http://boinc.berkeley.edu/trac/wiki/BoincPr

  17. BOINC-wide teams • http://boinc.berkeley.edu/teams • create BOINC-wide teams here • offers list of teams/founders by Web RPC • team_update.php (periodic task) • fetches team list • creates users and teams as needed • BOINC-wide teams are automatically created on new projects

  18. Skype-based “customer support” • Volunteer registration • languages • projects • Server polls Skype status • Users can call or email • Users can rate volunteers

  19. Scheduler and daemons 2006 2007 Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug EDF simulation based on workload use reliable hosts for retries homogeneous redundancy changes multi-platform support abort, abort-if-unstarted

  20. Client 2006 2007 Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug security-related changes “suspend if idle” preference memory-aware scheduling log availability info auto-update improved slot dir management improved network BW estimates

  21. Manager 2006 2007 Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug preferences dialog, day-of-week prefs per-application slide show project selection menu grid-based views complete skin mechanism

  22. API and runtime system 2006 2007 Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug multi-job wrapper message-passing more robust support memory-aware apps kill unresponsive apps

  23. Web features 2006 2007 Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug team foundership transfer mass host merge show all projects on user page certificate of computation forum moderation features Paypal donation system private messages

  24. Web RPCs and AMS support 2006 2007 Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Google gadget AMS can specify prefs, venue, resource share BOINC-wide teams team history delta

  25. Simulators 2006 2007 Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug SIMBA (global simulator) client simulator

  26. What did we not get around to? • Document app development using Cygwin, MinGW, FORTRAN • Disk space management • Revised credit system • better FLOPS accounting • generalized credit system • Community features • buddy lists • cross-project profiles, message boards • BOINC protocols as Internet standards

  27. What’s in the works?

  28. Version 6 application graphics • Old: graphics in separate thread • New: graphics in separate program • takes –fullscreen option • has logical name “v6graphics” • Library support: supply render function, input-handling functions • Support for shared memory between app and graphics BOINC manager, screensaver BOINC client graphics app main app shmem

  29. Other V6 changes • Windows • Sandboxing: create unprivileged account for BOINC • Move data directory out of C:Program Files • Wix-based installer • Mac/Linux • Use memory-mapped files instead of shmat()

  30. V5/V6 transition • V6 client is compatible only with V6 apps • Scheduler change: • API version is attribute of app version • Can have both V5 and V6 app versions • Scheduler will send only V5 apps to pre-V6 clients • Projects can maintain backwards compatibility as long as they want

  31. What’s in the future?

  32. Berkeley@home • Campus-level “meta-project” • Applications • 6 pilot apps: climate, geophysics, nanotechnology, genetics, phylogenomics, scientific decision systems • Volunteers • 1,000 instructional PCs • 5,000 faculty/staff • 30,000 students • 400,000 alumni • general public • NSF proposal submitted

  33. Multi-threading support • What’s in a $1000 PC? • 2007: dual-core CPU, 4 GFLOPS, 1 GB RAM • 2010: 80-core CPU, 100 GFLOPS, 8 GB RAM • Volunteer computing provides a use for all those cores, but you may run out of RAM • BOINC support for multi-thread apps • Languages/libraries for parallel programming • Open MP • Titanium, Cilk, ... Try to use N cores core client app OK, I’m using M cores

  34. Non-PC platforms • Sony Playstation 3 • Support from Sony: on hold • PS3Grid.org • Other game consoles • TV set-top boxes • Hand-held devices • not clear if feasible

  35. Energy and volunteer computing • Computing == energy usage [== carbon emission ] • Energy-aware computing • dynamic clock rate • shut-off when idle • disk spin-down • BOINC must be energy-aware friendly • “compute while in use” by default • allow auto-shutoff • don’t assume constant clock rate • Make sure volunteers are aware of energy cost • Computer as space heater

  36. How to validate results and credit? • Redundancy • homogeneous redundancy: messy • heterogeneous: must write fuzzy validator • wastes computing power • Non-redundancy (CPDN, Rosetta) • result consistency checking (application-specific) • granted credit: constant, or based on project-wide distribution • Variable redundancy? • hosts start with 2X redundancy, asymptote to 1X redundancy as trust increases • (doesn’t solve comparison problem)

  37. Skill aggregation (human computing) • Web-based vision tasks • Stardust@home, Clickworkers, galaxy classification • Amazon “Mechanical Turk” • Validation • Formulation as multi-person game • Louis von Ahn: image tagging • Motivational axes: competition community

  38. Rosetta@home plan • Protein structure prediction • low-res: combinatorial, spatial, intuitive; humans do better than computers • high-res: computers do better • Interactive “protein manipulation” program • Teams as management structures • tasks are given to (possibly multiple) teams • managers organize and schedule sub-groups with particular skills or resources • communication paths between sub-groups

  39. Berkeley Open System for Skill Aggregation (BOSSA) • DB-driven web-based system • Tools for community, competition, collaboration, and organization • Teams, groups, sub-groups • Communication mechanisms • Tasks, validation, credit • Integration with BOINC for tasks that involve large computation or storage

  40. How to verifiably train volunteers? lesson exercise exercise lesson exercise • Interspersed lessons and exercises • Optimal refresh for long-term retention • Analytics for content assessment/refinement • Adaptive teaching • Volunteer computing and skill aggregation provide an ideal venue for education research (3 weeks) exercise (1 week) exercise lesson exercise lesson exercise lesson lesson

  41. Berkeley Open Learning Technology (BOLT) • DB-driven CMS and analytics engine for web-based teaching content (lessons, exercises) teaching engine (PHP) Sequencing, navigation Students student info, interaction DB course structure (XML) analytical tools Educators

  42. DB and web integration BOINC hosts applications jobs BOLT lessons courses BOSSA tasks Accounts, teams and groups Communication Credit and competition

  43. Background utility compatibility • Background utilities • disk defrag • disk indexing • virus scanning • web pre-fetch • disk backup • Most run only when computer is idle • volunteer computing ==> they never run • A) ignore zero-priority CPU activity • B) Background manager • intelligent decision about when to run activities

More Related