user patterns from skyserver
Download
Skip this Video
Download Presentation
User Patterns from SkyServer

Loading in 2 Seconds...

play fullscreen
1 / 5

User Patterns from SkyServer - PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on

User Patterns from SkyServer. 1/f power law of session times and request sizes No discrete classes of users!! Users are willing to learn SQL for advantage Quickly adopted to server-side MyDB environment 1600 active users of MyDB Start with small ad-hoc requests

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' User Patterns from SkyServer' - blythe


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
user patterns from skyserver
User Patterns from SkyServer
  • 1/f power law of session times and request sizes
    • No discrete classes of users!!
  • Users are willing to learn SQL for advantage
  • Quickly adopted to server-side MyDB environment
    • 1600 active users of MyDB
  • Start with small ad-hoc requests
  • Once template finalized, run on whole dataset
  • Share results with others

Through the web services interface

  • Frequent use of crawlers
  • Often bucketize requests (checkpointing)
mydb casjobs
MyDB/CasJobs

Nolan Li, JHU

  • Server side user databases
  • Can perform joins with main databases
  • Users can create their own UDFs etc
  • Every user action is recorded
  • Can share tables, do simple graphics server side

MyDB

DR4

DR5

DR6

S4

jhu petascale archive
JHU Petascale Archive
  • We are building a distributed SQL Server cluster exceeding 1 Petabyte
  • Just becoming operational
  • 40x8-core servers with 22TB each, 6x16-core servers with 33TB each, connected with Infiniband
  • 10Gbit lambda uplink to StarTap
  • Funded by Moore Foundation, Microsoft and the Pan-STARRS project
  • Dedicated to eScience,will provide public access
amdahl s laws
Amdahl’s Laws

Gene Amdahl (1965) Laws for a balanced system

  • Parallelism: max speedup is S/(S+P)
  • One bit of IO/sec per instruction/sec (BW)
  • One byte of memory per one instruction/sec (MEM)
  • One IO per 50,000 instructions (IO)

Modern multi-core systems move farther away from Amdahl’s Laws (Bell, Gray and Szalay 2006)

For a Blue Gene the BW=0.013, MEM=0.471.

For the JHU cluster BW=0.664, MEM=1.099

components
Components
  • Data must be heavily partitioned
  • It must be simple to manage
  • Distributed SQL Server cluster
  • Management tools
  • Configuration tools
  • Workflow environment for loading/system jobs
  • Workflow environment for user requests
  • Provide crawler framework
    • Both SQL and procedural languages
  • User workspace environment (MyDB)
ad