User patterns from skyserver
This presentation is the property of its rightful owner.
Sponsored Links
1 / 5

User Patterns from SkyServer PowerPoint PPT Presentation


  • 63 Views
  • Uploaded on
  • Presentation posted in: General

User Patterns from SkyServer. 1/f power law of session times and request sizes No discrete classes of users!! Users are willing to learn SQL for advantage Quickly adopted to server-side MyDB environment 1600 active users of MyDB Start with small ad-hoc requests

Download Presentation

User Patterns from SkyServer

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


User patterns from skyserver

User Patterns from SkyServer

  • 1/f power law of session times and request sizes

    • No discrete classes of users!!

  • Users are willing to learn SQL for advantage

  • Quickly adopted to server-side MyDB environment

    • 1600 active users of MyDB

  • Start with small ad-hoc requests

  • Once template finalized, run on whole dataset

  • Share results with others

    Through the web services interface

  • Frequent use of crawlers

  • Often bucketize requests (checkpointing)


Mydb casjobs

MyDB/CasJobs

Nolan Li, JHU

  • Server side user databases

  • Can perform joins with main databases

  • Users can create their own UDFs etc

  • Every user action is recorded

  • Can share tables, do simple graphics server side

MyDB

DR4

DR5

DR6

S4


Jhu petascale archive

JHU Petascale Archive

  • We are building a distributed SQL Server cluster exceeding 1 Petabyte

  • Just becoming operational

  • 40x8-core servers with 22TB each, 6x16-core servers with 33TB each, connected with Infiniband

  • 10Gbit lambda uplink to StarTap

  • Funded by Moore Foundation, Microsoft and the Pan-STARRS project

  • Dedicated to eScience,will provide public access


Amdahl s laws

Amdahl’s Laws

Gene Amdahl (1965) Laws for a balanced system

  • Parallelism: max speedup is S/(S+P)

  • One bit of IO/sec per instruction/sec (BW)

  • One byte of memory per one instruction/sec (MEM)

  • One IO per 50,000 instructions (IO)

    Modern multi-core systems move farther away from Amdahl’s Laws (Bell, Gray and Szalay 2006)

    For a Blue Gene the BW=0.013, MEM=0.471.

    For the JHU cluster BW=0.664, MEM=1.099


Components

Components

  • Data must be heavily partitioned

  • It must be simple to manage

  • Distributed SQL Server cluster

  • Management tools

  • Configuration tools

  • Workflow environment for loading/system jobs

  • Workflow environment for user requests

  • Provide crawler framework

    • Both SQL and procedural languages

  • User workspace environment (MyDB)


  • Login