cc lab proce55ing with image sound and video
Download
Skip this Video
Download Presentation
CC Lab: Proce55ing with image, sound and video

Loading in 2 Seconds...

play fullscreen
1 / 58

CC Lab: Proce55ing with image, sound and video - PowerPoint PPT Presentation


  • 143 Views
  • Uploaded on

CC Lab: Proce55ing with image, sound and video. By the way, apparently we should now use “Processing”, not “Proce55ing”. (Eventually we’ll go in and correct it in these PPTs.). The goals of this week module are to:.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'CC Lab: Proce55ing with image, sound and video' - edita


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide2
By the way, apparently we should now use “Processing”, not “Proce55ing”. (Eventually we’ll go in and correct it in these PPTs.)
the goals of this week module are to
The goals of this week module are to:

Learn how to do some basic image, video and sound processing with proce55ing.

slide4
There’s a lot of info in todays presentation.(We may work on this over two weeks. We’ll see how it goes.)
slide6

- Images must be in the "data" directory to load correctly.

  • Processing currently works with GIF, JPEG, and Targa images.
  • So, in order to put the images in the “Data” directory, go to Add File… from the Sketch Menu.
slide7

Code

PImage b;

// Images must be in the "data" directory to load correctly

b = loadImage(”dt.jpg");

image(b, 0, 0);

slide8

What’s happening in this code?PImage b PImage is the function name and b is the declared variable name for image.// followed by comment.b = loadImage(“dt.jpg”) calls the image from the data directory to be display in the sketchImage(b, 0, 0) displays the defined b image, followed by x and y co-ordinates of the upper left hand corner of the image.

slide10

There are a bunch of image proce55ing functions. We will go over them in this presentation. Also look at the proce55ing reference to check the syntax while practicinghttp://www.proce55ing.org/reference/index.html

image mode
Image mode

PImage b;

b = loadImage(”dt.jpg");

imageMode(CORNER);

image(b, 35, 35, 50, 50);

Modifies the location from which images draw.

image(b,35, 35, 50, 50) where 35,35 are the x and y co-ordinates of upper left hand side of the image and 50, 50 are the width and height of the image.

image mode12
Image mode

PImage b;

b = loadImage(”dt.jpg");

imageMode(CORNERS); there’s a difference between CORNER and CORNERS

image(b, 35, 35, 50, 50);

Modifies the location from which images draw.

image(b,35, 35, 50, 50) where 35,35 are the x and y co-ordinates of upper left hand side of the image and 50, 50 are the x and y co-ordinates of lower right hand side corner of the image.

background
Background

PImage b;

b = loadImage(”dt.jpg");

background(b);

So, you can also set the image as a background and then draw shapes and then top.

tint and alpha
Tint and Alpha

PImage b;

b = loadImage(”dt.jpg");

// Tint blue

tint(0, 153, 204);

image(b, 0, 0);

So, if you remember your RGB channels, you will

get this straight. Tint(0,153,204); so our Red

color is 0, Green is 153 and Blue is 204. So, R, G

and B takes 8 bits each, a maximum value of 255,

now notice blue is 204 which is closest to 255

and so is taking on the tint.

tint and alpha15
Tint and Alpha

PImage b;

b = loadImage("dt.jpg");

image(b, 0, 0);

// Tint blue and set transparency

tint(0, 153, 204, 126);

image(b, 50, 0);

So, the 32-bit graphics are completed by the 4-channels, the last one being Alpha, RGBA. Notice the last number in tint value is 126, which brings the opacity close to 50%. Just to show the variation the image is loaded again with x-cord.on 50 pixels.

filters
Filters

PImage b;

b = loadImage(“1.jpg”);

Image (b,0,0);

//Hit play to preview the image

// insert the filter

filter(THRESHOLD);

//hit play now.

By Default processing applies

.5 threshold Value. You can apply

any value between 0 and1,

integer or float.

For example

Filter (THRESHOLD, 0.4);

before

after

lets try some more filters
Lets try some more Filters

filter(GRAY);

filter(INVERT);

filter(POSTERIZE, 4);

filter(BLUR, 6);

slide19
Get

PImage myImage = loadImage("1.jpg");

image(myImage, 0, 0);

PImage cp = get(10,20,20,20);

image(cp, 30, 30);

Reads the color of any pixel or grabs a section of an image.

In this example,

PImage cp = get (10,20,20,20) goes to 10x and 20y cords of the image and grab 20 px width and 20 px height from that co-ordinate.

image(cp,30,30) maps the grabbed pixels on 30x and 30y.

slide20
Get

PImage myImage = loadImage(”1.jpg"); image(myImage, 0, 0);

color cp = get(30, 20);

fill(cp);

rect(30, 20, 55, 55);

Reads the color of any pixel or grabs a section of an image.

In this example, color cp = get(30, 20) picks up the color of pixel located on 30x and 20y of the image.

fill(cp) Fills the shape with the picked up color from that particular pixel.

slide21
Set

PImage myImage = loadImage("1.jpg");

image(myImage, 0, 0);

color white = color(255);

set(30, 20, white);

set(85, 20, white);

set(85, 75, white);

set(30, 75, white);

Changes the color of any pixel or writes an image directly into the display window.

In this example, color white = color (255) selects the white color and set (30, 20, white) plants white color on the specific pixel co-ordinate.

slide22
Copy

PImage img = loadImage("eames.jpg");

image(img, 0, 0);

copy(15, 25, 10, 10, 35, 25, 50, 50);

Copies a region of pixels from the display window to another area of the display window.

In this example, copy(15,25, 10,10 are x, y cords, width and height of the source, 35, 25, 50, 50 are x, y cords, width and height of the destination. Notice the source is 10 x 10 pixels and destination is 50 x 50 pixels, which results in 5 times scaling of pixels pasted.

blend
Blend

background(loadImage("22.jpg"));

PImage img = loadImage("1.jpg");

image(img, 0, 0);

blend(img, 0, 0, 33, 100, 67, 0, 33, 100, ADD);

Blends two color values and copies a single pixel or region of pixels from one image into another.

We can use two images, to see one blending into other.

blend(img, 0, 0, 33, 100,where img is the source image, x and y cords, width and height of the source image; 67,0,33,100 are the x and y cords, width and height of blending area.

blend24
Blend

Finally the Mode, ADD, SUBTRACT, LIGHTEST, DARKEST sets the superimposition types.

blend(img, 0, 0, 33, 100, 67, 0, 33, 100, SUBTRACT);

blend(img, 0, 0, 33, 100, 67, 0, 33, 100, MULTIPLY);

blend(img, 0, 0, 33, 100, 67, 0, 33, 100, LIGHTEST);

blend(img, 0, 0, 33, 100, 67, 0, 33, 100, DARKEST);

now we used to use psound
Now, we used to use “Psound”:

Description: Class for storing and playing sounds of the .wav format.   

Methods:

play() - Plays the sound once

loop() - Plays the sound continuously

noLoop() - Stops the sound from looping

pause() - Pauses the sound playback

stop() - Stops the sound playback

volume() - Sets the volume of the sound playback

duration() - Returns the total length of the sound in seconds

time() - Returns the current position of sound playback in seconds

slide29
So, we are looking into a couple of alternatives:the Sonia and Ess librariesYou can go to:http://processing.org/reference/libraries/
things we want to do with sound31
Things we want to do with sound:
  • Play control (play, stop, FF, reverse)
  • Sampling
  • Sound Looping
slide32

For today, let’s browse through the possibilities and check out what the different sound libraries are capable of.Form 4-5 groups and go do a bit of research…

slide33
Loading

and

Capturing

Video

the processing video library
The Processing Video Library

Video can be captured from:

USB cameras.

IEEE 1394 (Firewire) cameras.

Video cards with composite/s-video.

Video can be loaded from:

Any QuickTime file on your computer

or on the Internet

software requirements for video
Software Requirements for Video

Both Mac and PC’s need this software installed to work with video:

Proce55ing version 0093

(0095 wont work due to a few bugs)

Apple QuickTime

pc webcams
PC Webcams

If your PC Webcam does not use QuickTime...

(Such as Logitech webcams.)

you need to install

WinVDIG

With WinVDIG QuickTime applications can capture video on Windows. It is free to use.

http://www.vdig.com/WinVDIG/

if you run windows
If you run Windows...

And if you get this error in Proce55ing:

“This version of QTJava has expired”

trying to load videos…

You need to upgrade your Apple QuickTime 7 software.

Apple forgot to remove the expiration date from the QuickTime 7 beta code base when it went into to production.

http://www.apple.com/quicktime

movie
Movie

Description: Datatype for storing and playing movies in Apple\'s QuickTime format.   

Methods:

read() - Reads the current frame from the movie

available() - Returns true when a new frame is available

play() - Plays the movie once

pause() - Pauses the movie playback

stop() - Stops the movie playback

loop() - Plays the movie continuously

noLoop() - Stops the movie from looping

jump() -Jumps to a specific location in the movie

duration() - Returns the total length of the movie

time() - Returns the current playback position

speed() - Sets a multiplier for how fast/slow a movie should play

framerate() - Sets how often new frames are read from the movie

slide39
Movies must be located in the sketch\'s data directory or an accessible place on the network to load without an error.
load a movie
Load a Movie

//Load_Quicktime.pde

import processing.video.*;

Movie myMovie;

void setup() {

size(160, 120);

background(255,0,0);

myMovie = new Movie(this, "honda_clip1.mov");

myMovie.loop();

}

void draw() {

image(myMovie, 40, 30, 80, 60);

}

// Called every time a new frame is available to read

void movieEvent(Movie m) {

myMovie.read();

}

load a movie code details
Load a Movie Code Details

import processing.video.*;  Import the Video Library.

Movie myMovie;  Movie datatype variable.

myMovie = new Movie(this, "honda_clip1.mov");  Load movie and pass it to your variable.

Constructors  Movie(parent, filename)

Movie(parent, filename, fps)

Movie(parent, url)

Movie(parent, url, fps)   

Parameters  parent PApplet: typically use "this" filename String: name of the movie fps int: frames per second

url URL: location of the file on the network

myMovie.loop();  Loop the movie.

image(myMovie, 40, 30, 80, 60);  Place movie on screen

image(img, xpos, ypos, width, height);

void movieEvent(Movie m) {  Called every time a new frame is available to read

myMovie.read();

}

loading two movies
Loading Two Movies

//Quicktime_Two.pde

import processing.video.*;

Movie myMovie, yourMovie;

void setup() {

size(200, 200);

myMovie = new Movie(this, "station.mov");

yourMovie = new Movie(this, "street.mov");

myMovie.loop();

yourMovie.loop();

}

void draw() {

image(myMovie, 0, 0, 200, 200);

image(yourMovie, 50, 50, 75, 75);

}

void movieEvent(Movie m) {

if(m == myMovie) {

myMovie.read();

} else if(m == yourMovie) {

yourMovie.read();

}

}

void mousePressed() {

myMovie.stop();

yourMovie.play();

}

void mouseReleased() {

myMovie.play();

yourMovie.stop();

}

two movie code details
Two Movie Code Details
  • Load each movie into unique variables.

 Loop both of the movies.

  • Place both movies on the screen.
  • Called when a new movie frame is available.
  • When mousePressed/mouseReleased play/stop the movies.

myMovie = new Movie(this, "station.mov");

yourMovie = new Movie(this, "street.mov");

myMovie.loop();

yourMovie.loop();

image(myMovie, 0, 0, 200, 200);

image(yourMovie, 50, 50, 75, 75);

void movieEvent(Movie m) {

if(m == myMovie) {

myMovie.read();

} else if(m == yourMovie) {

yourMovie.read();

}

}

void mousePressed() {

myMovie.stop();

yourMovie.play();

}

void mouseReleased() {

myMovie.play();

yourMovie.stop();

}

slide44

Pixel Fun

//Quicktime_Pixel.pde

import processing.video.*;

int numPixels;

int blockSize = 5;

Movie myMovie, yourMovie;

color myMovieColors[];

void setup() {

size(480, 360);

noStroke();

background(0);

myMovie = new Movie(this, "station.mov");

yourMovie = new Movie(this, "nuke.mov");

myMovie.loop();

yourMovie.loop();

numPixels = width / blockSize;

myMovieColors = new color[numPixels * numPixels];

}

void draw() {

for(int j=0; j<numPixels; j++) {

for(int i=0; i<numPixels; i++) {

fill(myMovieColors[j*numPixels + i]);

rect(i*blockSize, j*blockSize, blockSize-1, blockSize-1);

}

}

tint(255, 150);

image(yourMovie, mouseX, mouseY);

}

void movieEvent(Movie m) {

if(m == myMovie) {

myMovie.read();

for(int j=0; j<numPixels; j++) {

for(int i=0; i<numPixels; i++) {

myMovieColors[j*numPixels + i] = myMovie.get(i, j);

}

}

} else if(m == yourMovie) {

yourMovie.read();

}

}

pixel code details
Pixel Code Details

Again we are using two movies, two variables and loops like the last example.

int numPixels;  Variable for number of pixels.

int blockSize = 5;  Variable for the block size.

color myMovieColors[];  Datatype color Array.

numPixels = width / blockSize;  Width of the movie divided by blockSize to get number of pixels.

myMovieColors = new color[numPixels * numPixels];

 Fill the array with new colors.

pixel code details continued
Pixel Code Details Continued
  • For loop creates rectangles (virtual pixels) and fills them from the myMovieColors array. This is the values from myMovie.
  • tint() – Sets the tint of the image to be displayed
  • Place yourMovie on the screen positioned by the mouse

void draw() {

for(int j=0; j<numPixels; j++) {

for(int i=0; i<numPixels; i++) {

fill(myMovieColors[j*numPixels + i]);

rect(i*blockSize, j*blockSize, blockSize-1, blockSize-1);

}

}

}

tint(255, 150);

image(yourMovie, mouseX, mouseY);

pixel code details continued47
Pixel Code Details Continued

void movieEvent(Movie m) {

if(m == myMovie) {

myMovie.read();

for(int j=0; j<numPixels; j++) {

for(int i=0; i<numPixels; i++) {

myMovieColors[j*numPixels + i] = myMovie.get(i, j);

}

}

} else if(m == yourMovie) {

yourMovie.read();

}

}

The if statement of the mouseEvent() reads in the frame of myMovie and gathers the pixel values for myMovieColors array.

The else if reads in the current frame of yourMovie.

one last example with a url
One Last Examplewith a URL.

//Quiktime_Jump.pde

import processing.video.*;

Movie myMovie;

void setup() {

size(200, 200);

framerate(30);

myMovie = new Movie(this, "http://itp.nyu.edu/~dbo3/icm/choke.mov");

}

void draw() {

image(myMovie, 0, 0, 200, 200);

}

void mouseMoved() {

myMovie.jump(mouseX*(myMovie.duration()/width));

myMovie.read();

}

capture
Capture

Description: Datatype for storing and manipulating video frames from an attached capture device such as a camera.

Methods

read() - Reads the current image

available() - Returns true when a new image is available

list() - Lists the devices current attached and on

framerate() - Sets how often new frames are read

stop() - Stops capturing frames

Constructors:  Capture(parent, width, height)

Capture(parent, width, height, fps)

Capture(parent, name, width, height)

Capture(parent, name, width, height, fps)   

Parameters:  parent PApplet: typically use "this“

width int: width of the frame

height int: height of the frame fps int: number of frames to read per second name String: name of the camera

loading your camera in proce55ing
Loading your camera in Proce55ing

There are two ways for Processing to find your camera

In void()

camera = new Capture(this, 320, 240, 12);  Should find your FIRST camera that is ON and installed on your computer.

If this does not work then list any camera attached to your computer and specify it in Capture().

This is also the way to specify multiple cameras.

In void()

println(Capture.list());  List all available capture devices to the console. Use the information printed to the text area to correctly set the variable "s" below. Or multiple variables for multiple cameras.

String s = "Logitech QuickCam Zoom-WDM";  Specify your own device by the name of the capture device returned from the Capture.list() function.

camera = new Capture(this, s, width, height, 30);  Use the “s” variable in the Capture();

To open the camera’s preference application.

camera.settings();  Opens the settings for this capture device. This does not work for Apple iSight’s.

capturing video52
Capturing Video

//Video_capture.pde

import processing.video.*;

Capture camera;

void setup(){

size(320, 240);

camera = new Capture(this, 320, 240, 12);

}

void captureEvent(Capture camera){

camera.read();

}

void draw(){

image(camera, 0, 0);

}

capturing video code detail
Capturing Video Code Detail

import processing.video.*;

Capture camera;

camera = new Capture(this, 320, 240, 12);

void captureEvent(Capture camera){

camera.read();

}

void draw(){

image(camera, 0, 0);

}

  • Just like loading movies, you need to import the video library.
  • Create a variable of the datatype Capture.
  • Load the Camera and send it to your variable.
  • Called when a new camera frame is available.
  • Place the video on the screen using image().
movies and capturing
Movies and Capturing

//Video_Movie.pde

import processing.video.*;

Movie myMovie0, myMovie1, myMovie2;

Capture camera;

void setup(){

size(200,200);

background(0,0,0);

myMovie0 = new Movie(this, "nuke.mov");

myMovie1 = new Movie(this, "station.mov");

myMovie2 = new Movie(this, "street.mov");

camera = new Capture(this, 100, 100, 12);

myMovie0.loop();

myMovie1.loop();

myMovie2.loop();

}

void captureEvent(Capture camera){

camera.read();

}

void draw(){

image(camera, 0, 0);

image(myMovie0, 100, 100, 100, 100);

image(myMovie1, 100, 0, 100, 100);

image(myMovie2, 0, 100, 100, 100);

}

void movieEvent(Movie m) {

if (m == myMovie0) {

myMovie0.read();

} else if(m == myMovie1) {

myMovie1.read();

} else if(m == myMovie2) {

myMovie2.read();

}

}

code details
Code Details

Nothing new here.

Just using the concepts that we just went over.

Now on to something a lot more difficult.

tracking outlines
Tracking Outlines

//Video_Edges.pde

import processing.video.*;

Capture video;

int threshold = 25;

PImage edges;

void setup(){

video = new Capture(this,640, 480, 30);

size(640, 480);

edges = new PImage( width,height);

}

void captureEvent(Capture camera) {

camera.read();

}

void draw(){

for(int row=1; row<video.height-1; row=row+1) {

for(int col=1; col<video.width-1; col=col+1) {

int right = video.pixels[(row)*video.width+col+1];

int left = video.pixels[(row)*video.width+col-1];

if (abs(red(right)-red(left)) > threshold){

edges.pixels[row*video.width+col] = color(255,0,0);

}else{

edges.pixels[row*video.width+col] = color(0,0,0);

}

}

}

edges.updatePixels();

image(edges,0,0);

}

void keyPressed(){

if (keyCode ==38) { //up arrow

threshold++;

}else if(keyCode == 40){ //down arrow

threshold--;

}

println("key: " + keyCode + " Threshold: " + threshold);

}

tracking outlines code details
Tracking Outlines Code Details

int threshold = 25;

PImage edges;

edges = new PImage( width,height);

void draw(){

for(int row=1; row<video.height-1; row=row+1) {

for(int col=1; col<video.width-1; col=col+1) {

int right = video.pixels[(row)*video.width+col+1];

int left = video.pixels[(row)*video.width+col-1];

if (abs(red(right)-red(left)) > threshold){

edges.pixels[row*video.width+col] = color(255,0,0);

}else{

edges.pixels[row*video.width+col] = color(0,0,0);

}

}

}

edges.updatePixels();

image(edges,0,0);

}

void keyPressed(){

if (keyCode ==38) { //up arrow

threshold++;

}else if(keyCode == 40){ //down arrow

threshold--;

}

println("key: " + keyCode + " Threshold: " + threshold);

}

  • Initialize the variable threshold
  • Datatype for storing images. In this case the image is a single frame of captured video.
  • Pass the Image/frame information to the variable edges.
  • First for loop is for processing the pixels in each row of the image/frame. The second for loop is for processing the pixels in each column of the image/frame.
  • The if statement checks the threshold and then sets the color of the pixels on the edge or outline of forms.
  • Pixels[] - Array containing the values for all the pixels in the display window.
  • updatePixels() - Updates the display window with the current data from pixels[].
  • Write the image to the screen.
  • keyPressed() – Takes input from the keyboard.
  • Depending on the key pressed change the threshold.
slide58
All of the code and files presented here are on the CC Lab website:

Proce55ing_week2_stuff.zip

Many of these examples are derived right from the Processing reference guide, code examples that come with Processing and Dan O’Sullivan’s ITP website.

ad