large scale machine learning using dryadlinq
Download
Skip this Video
Download Presentation
Large-scale Machine Learning using DryadLINQ

Loading in 2 Seconds...

play fullscreen
1 / 17

Large-scale Machine Learning using DryadLINQ - PowerPoint PPT Presentation


  • 144 Views
  • Uploaded on

Large-scale Machine Learning using DryadLINQ. Mihai Budiu Microsoft Research, Silicon Valley Ambient Intelligence: From Sensor Networks to Smart Environments and Social Media Workshop Stanford, June 11, 2019. Goal of DryadLINQ. Software Stack. Applications. . Net + LINQ. DryadLINQ.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Large-scale Machine Learning using DryadLINQ' - randi


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
large scale machine learning using dryadlinq

Large-scale Machine Learning using DryadLINQ

Mihai Budiu

Microsoft Research, Silicon Valley

Ambient Intelligence: From Sensor Networks to Smart Environments and Social Media Workshop

Stanford, June 11, 2019

software stack
Software Stack

Applications

.Net + LINQ

DryadLINQ

Dryad

Cluster storage

Cluster services

Windows Server

Windows Server

Windows Server

Windows Server

dryad execution layer
Dryad = Execution Layer

Job (application)

Pipeline

Dryad

Unix Shell

Cluster

Machine

linq data model
LINQ Data Model

.NET objects of type T

Collection

linq language summary
LINQ Language Summary

Input

Where (filter)

Select (map)

GroupBy

OrderBy (sort)

Aggregate (fold)

Join

slide7
LINQ

=> DryadLINQ

Dryad

dryadlinq data model
DryadLINQ Data Model

.Net objects

Partition

Collection

dryadlinq linq dryad
DryadLINQ = LINQ + Dryad

Collection<T> collection;

static boolIsLegal(Key c);

var results = from c in collection where IsLegal(c.key) select new { Hash(c.key), c.value};

Code

Dryad job

Data

collection

C#

C#

C#

C#

results

natal problem
Natal Problem
  • Recognize players from depth map
  • At frame rate
  • Low resource usage
learn from data
Learn from Data

Rasterize

Training examples

Motion Capture

(ground truth)

Machine learning

Classifier

cluster based training
Cluster-based training

Classifier

Training examples

Machine learning

DryadLINQ

Dryad

you can have it
You can have it!
  • Dryad+DryadLINQ available for download
    • Academic license
    • Commercial evaluation license
  • Runs on Windows HPC platform
  • Dryad is in binary form, DryadLINQ in source
  • Requires signing a 3-page licensing agreement
  • http://connect.microsoft.com/site/sitehome.aspx?SiteID=891
ad