your results from disk to device preview n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Your Results From Disk to Device (Preview) PowerPoint Presentation
Download Presentation
Your Results From Disk to Device (Preview)

Loading in 2 Seconds...

play fullscreen
1 / 24

Your Results From Disk to Device (Preview) - PowerPoint PPT Presentation


  • 101 Views
  • Uploaded on

Your Results From Disk to Device (Preview). Amit Chourasia Mona Wong- Barnum Michael L. Norman San Diego Supercomputer Center, UCSD Presented at: XSEDE 13, San Diego, CA, Jul 23, 2013. S tream E ncode E xplore D isseminate M y E xperiments. Computation Cycle. Abstract ?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Your Results From Disk to Device (Preview)' - vega


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
your results from disk to device preview

Your Results From Disk to Device(Preview)

Amit Chourasia

Mona Wong-Barnum

Michael L. Norman

San Diego Supercomputer Center, UCSD

Presented at: XSEDE 13, San Diego, CA, Jul 23, 2013

computation cycle
Computation Cycle

Abstract

?

Monitor?

Validate?

slide4

Monitoring

  • DerivedProducts
  • Abstraction
  • Validation
slide5

Direct Access

  • Not All Members Have HPC Access
  • Direct Sharing
  • Security Implications On HPC
  • DerivedProducts
slide6

Concise derived products

  • Plots & Text Documents
  • Image Sequences & Videos

Share with the research group

  • Download derived products
  • Share with collaborators
      • Email snippets
      • Set up a webpage
        • Add content
        • Email
derived visual content survey
Derived Visual Content Survey

Derived content for a single research group in respective domain

pitfalls in sharing d erived c ontent
Pitfalls in sharing derived content
  • Download Upload Round Trip
  • Video Encoding Complexity
  • Web Setup
  • Process Duplication
  • Scalability
slide9

Direct Transfer

  • Easy Sharing
  • Reusable
  • Ubiquitous
Gaps

SeedMe.org

  • Download Upload
  • Video Encoding
  • Web Setup
  • Process Duplication
  • Scalability
  • Direct Access
  • Not All Members Have HPC Access
  • Direct Sharing
  • Security Implications On HPC
seedme architecture
SeedMe Architecture

Ingestion (Atomic or Recurring)

  • Instant Processing
  • (Text, Images)
  • Notification
  • Responsive Content Access
  • Batch Processing
  • (Videos)
seedme hardware
SeedMe Hardware

Ingestion (Atomic or Recurring)

  • Instant Processing
  • (Text, Images)
  • Notification
  • Responsive Content Access
  • Batch Processing
  • (Videos)

20

gbe

Web Cluster (2-8 nodes) : Drupal

20

gbe

20

gbe

Phones

Ingest

Temp

Storage (10TB)

Mirrored Storage (50 TB) : ZFS

Tablets

Encode Cluster (16 nodes) :

Apple Compressor + ffmpeg

Desktops

seedme collection
SeedMe Collection

Ticker*

  • Plots
  • Image Sequences*
  • Videos

Access: Public, Group, Private

Collaborators

Meta Data

* Recurring: periodic update

seedme interaction process
SeedMe: Interaction Process

Create New Collection

Set Access Control (Public, Group, Private)

Specify collaborators* and meta data

Upload/Update content

http/https, ftp, Globus Online

Interactively on Web, Command Line or REST Services

View Collection on Web

seedme interaction modality
SeedMe: Interaction Modality
  • Command Line Interface (using curl)
  • Web Interface (WYSIWYG)
  • RESTful API (integrate your tool chain)
    • JSON
slide17

{

"email":”jdoe@sdsc.edu",

"api_key":"XXXXXXXXXXX",

”view_permission":"public",

”viewer_emails":["a1@a.com","b1@b.com"],

"title":"simulation XYZ",

"description":"input conditions",

"credits":"Visservices group, SDSC",

"key_values":{“key1”: "v1”,"key2":"v2”},

"expire_date":"2013-05-01",

"tags":["x","y","z"],

"ticker":["time step 100","time step 200"],

"files":{

”docs":["a.pdf",”b.txt"],

"plots":["a.png","b.jpg","c.tiff"],

"image_sequences":{

"ash":["ash1.jpg","ash2.jpg","ash3.jpg"],

"air":["air1.jpg","air2.jpg","air3.jpg"]

},

"movies":["m1.mov","m2.mov »]

}

}

Authentication

Permissions

Meta Data

Content

Text Ticker

Doc Files

Plot Files

Image Sequences

Videos

application scenarios
Application Scenarios
  • Monitor progress with text tickers and images
  • Encode and stream videos from image sequences
  • Share & collect visualization results
  • Use the collections for education and outreach

Content accessible on most devices

work in progress
Work in progress

How it Works

Examples #1 & #2

Periodic update at simulation run time from a Magneto Hydrodynamics Simulation

Courtesy of Mahidhar Tatineni and Homa Karimabadi, UCSD

future work
Future Work
  • Production SeedMe Service
  • Example Scripts& Documentation
  • Globus integration with SeedMe
acknowledgements
Acknowledgements

Andrew Ferbert, Michael Dwyer, Kristen Levy & Doug Weimer, SDSC :Hardware/software setup

Mahidhar Tatineni, SDSC & Homa Karimabadi, UCSD: in-situ tests

Apple Inc. : Provided test hardware/software on loan during project conceptualization phase

National Science Foundation

This material is based upon work supported by the National Science Foundation under Grant No. OCI-1235505

slide22

Sign up Invitation

For notification and early access

(we won’t spam)

Contact: amit @ sdsc.edu