Chuck parts 3 and 4 plork and machine learning
This presentation is the property of its rightful owner.
Sponsored Links
1 / 27

Chuck! Parts 3 and 4: PLOrk and machine learning PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on
  • Presentation posted in: General

Chuck! Parts 3 and 4: PLOrk and machine learning. Rebecca Fiebrink Princeton University. Part 3: The Princeton Laptop Orchestra. 4 years old, first of its kind Now laptop orchestras in Stanford U., Tokyo, London, Oslo, and many others 15+ students

Download Presentation

Chuck! Parts 3 and 4: PLOrk and machine learning

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Chuck parts 3 and 4 plork and machine learning

Chuck!Parts 3 and 4: PLOrk and machine learning

Rebecca Fiebrink

Princeton University


Part 3 the princeton laptop orchestra

Part 3: The Princeton Laptop Orchestra

  • 4 years old, first of its kind

    • Now laptop orchestras in Stanford U., Tokyo, London, Oslo, and many others

  • 15+ students

    • Learn ChucK, some Max/MSP, and digital music composition in two classes per week

    • Rehearse once per week

  • Pieces composed by students, faculty, and guest artists

    • Very hard compositional issues!


Dan trueman silicon carbon

Dan Trueman: Silicon/Carbon

  • http://www.youtube.com/watch?v=YyD96blI4EU

  • http://www.youtube.com/watch?v=o71j1xGvvfA&feature=related


Equipment

Equipment


Techniques

Techniques

  • Network to synchronize computers

  • Sensors & game controllers

  • Incorporate voice & acoustic instruments

  • Often improvisational within a predetermined structure

  • Unique conducting gestures


Part 4

Part 4

Machine learning for live performance


Machine learning for live performance

Machine learning for live performance

  • Problem: there is a semantic gap between the raw data that computers use and the musical, cultural, aesthetic meanings that humans perceive and assign.


One solution a lot of code

One solution: A lot of code

  • What algorithm would you design to tell a computer whether a picture contains a human face?


The problem

The problem

  • If your algorithm doesn’t work, how can you fix it?

  • You can’t easily reuse it to do a similar task (e.g., recognizing monkey faces that are not human)

  • There’s no “theory” for how to write a good algorithm

  • It’s a lot of work!


Another solution machine learning classification

Another solution: Machine learning (Classification)

  • Classification is a data-driven approach for applying labels to data. Once a classifier has been trained on a training set that includes the true labels, it will predict labels for new data it hasn’t seen before.


Chuck parts 3 and 4 plork and machine learning

Train the classifier on a labeled dataset

Data Set: A feature vector and class for every data point

Classifier


Chuck parts 3 and 4 plork and machine learning

Run the trained classifier on new data

Classifier

NO!


Candidates for machine learning

Candidates for machine learning

  • Which gesture did the performer just make with the iCube?

  • Which instruments are playing right now?

  • Who is singing? What language are they singing?

  • Is this chord major or minor?

  • Is this dancer moving quickly or slowly?

  • Is this music happy or sad?

  • Is anyone standing near the camera?


An example algorithm knn

An example algorithm: kNN

  • The features of an example are treated as its coordinates in n-dimensional space

  • To classify an new example, the algorithm looks for its k (maybe 10) nearest neighbors in that space, and chooses the most popular class.


Knn space basketball or sumo

kNN space: Basketball or Sumo?

Feature 2: Height

Feature 1: Weight


Knn space basketball or sumo1

kNN space: Basketball or Sumo?

?

Feature 2: Height

Feature 1: Weight


Knn space basketball or sumo2

kNN space: Basketball or Sumo?

K=3

?

Feature 2: Height

Feature 1: Weight


Knn space basketball or sumo3

kNN space: Basketball or Sumo?

S

Feature 2: Height

Feature 1: Weight


Another technique neural networks

Another technique: Neural networks

  • One of the first machine learning methods

  • Model functions as complex connections of interconnected nodes (neurons)

    • Inspired by the human brain

    • An input causes a cascade of neuron firings, resulting in an output value (e.g., a class label, or a real number)

  • Results in highly non-linear functions from input to output

    • Interesting for performance!


Combining techniques

Combining Techniques

Processing: Extract webcam features

Java: Train a neural network to map features to sounds

OSC

OSC

ChucK: Pass features to Java, receive results back and use them to make sound

Example: Samson recognizer


Review

Review

  • Machine learning is often easier than the alternatives

    • Use standard algorithms (exiting libraries) to do hard work

    • Present examples of inputs (features) with their outputs (desired labels)

    • Requires you to choose which features to use (different for audio, video, sensor)

  • Appropriate for camera, audio, sensors, and many other types of data

  • Live, interactive performance is a very interesting application area

    • E.g., can re-train classifiers as you get new data

  • See my handout: Classification_handout.pdf


  • Login