Lecture09

1. Lecture09 Push & Immediate Mode Imaging

2. Push Imaging Model (PIM) • ImageProducer is an interface for objects which can produce the image data for Images. • ImageConsumer is an interface for objects expressing interest in image data through the ImageProducer interfaces. When a consumer is added to an image producer, the producer delivers all of the data about the image using the method calls defined in this interface.

3. Image Manipulation in Push Imaging Model • Basic single pixel level image manipulation can be done with push imaging model. • Image Scaling • Image Filtering

4. Image Scaling in PIM • getScaledInstance(int width, int height, int hints) method of the Image class can be used to scale an image and return a new Image object. hints specifies the algorithm to be used for image resampling. width and height specifies the dimensions of the new image.

5. Image Filtering in PIM • CropImageFilter : Crop out certain region of the image pixels. • RGBImageFilter : This subclass can be used to change individual pixel values. In order to use this class, public int filterRGB(int x, int y, int rgb) method of it must be defined.

6. Image Filtering • The ImageFilter class objects are ImageConsumer objects which allow them to receive data from an ImageProducer. These filters get wrapped in an ImageProducer called java.awt.Image.FilteredImageSource which sends out the filtered data. The ImageObserver is told the status of the image loading in the final ImageConsumer through calls to its imageUpdate method. // ... Image originalImg = getImage(url); ImageFilter ifil = new CropImageFilter(0,0,64,64); ImageProducer ip = new FilteredImageSource(originalImg.getSource(),ifil); Image filteredImg = createImage(ip); // ...

7. PixelGrabber • PixelGrabber is an ImageConsumer that can collect pixel data into an array such that pixels can be processed in its entirety before sending it out again. • Thus PixelGrabber marks the end of the push model and the beginning of an immediate mode model because the image data is put into an array instead of being passed to another ImageConsumer.

8. PixelGrabber PixelGrabber grabber = new PixelGrabber(originalImg, 0, 0, -1, -1,true); try { if (grabber.grabPixels()) { int width = grabber.getWidth(); int height = grabber.getHeight(); int[] originalPixelArray = (int[]) grabber.getPixels(); } else { System.err.println("Grabbing Failed"); } } catch (InterruptedException e) { System.err.println("PixelGrabbing interrupted"); }

9. PixelGrabber • Once you have pixel data in the originalPixelArray as in the above code, we can process it resulting in, say, newPixelArray. • This post-processed data can be given to a ImageProducer such as java.awt.image.MemoryImageSource so that the push imaging pipeline can be started again as follows: MemoryImageSource mis; mis = new MemoryImageSource(width, height, newPixelArray,arrayOffset,scanLength); Image filteredImage = createImage(mis);

10. Animation with MemoryImageSource • An Example code: \\ declare variables \\ create the pixels (a pixel array) source = new MemoryImageSource(width, height, pixels, 0, width); source.setAnimated(true); Image image = createImage(source); \\ do other stuff \\ Continued next slide

11. Animation with MemoryImageSource Thread me = Thread.currentThread( ); while (true) { try { thread.sleep(10); } catch( InterruptedException e ) { return;} // Modify the values in the pixels array at (x, y, w, h) // Send the new data to the interested // ImageConsumers source.newPixels(x, y, w, h); } \\ Do other stuff

12. Immediate Mode Imaging Model of Java2D • This model is centered around the subclass java.awt.image.BufferedImage of the class Image. This is achieved by having each BufferedImage contain both a Raster and a ColorModel. • Basically, this model provides memory allocation and storage of all image data, thus making it available to the programmer at all the times just as if you collected all the pixel data using a PixelGrabber in the older push model.

13. Image object into a BufferedImage 1. Make sure that all the image data is loaded. 2. Create a new BufferedImage using the image width, height and image data type (usually BufferedImage.TYPE_INT_ARGB). 3. Obtain the BufferedImage’s Graphics2D object. 4. Using this graphics object, draw the image onto the BufferedImage as done in the double buffering.

14. Image object into a BufferedImage static public BufferedImage createBufferedImage(Image imageIn, int imageType) { Label dummyComponent = new Label();// you can use any // component here MediaTracker mt = new MediaTracker(dummyComponent); mt.addImage(imageIn,0); // 0 is a given identifier of type int try { mt.waitForID(0); // 0 is the identifier } catch (InterruptedException ie) {} BufferedImage bufferedImageOut = new BufferedImage( imageIn.getWidth(null), imageIn.getHeight(null), imageType); Graphics g = bufferedImageOut.getGraphics(); g.drawImage(imageIn,0,0,null); return bufferedImageOut; }

15. Filtering with BufferedImage • When filtering with BufferedImage, you can perform more advanced filtering operations such as convolution and geometric transformations. • Filters for BufferedImages tend to give the alpha chanel special consideration whereas filters for Rasters don’t. This is because BufferedImages contain a ColorModel that allows interpretation of the color components, and with Raster no such interpretation is possible.

16. BufferedImageOp and RasterOp Interfaces • When performing BufferedImage filtering, much of the functionality of the used filter will be defined by the BufferedImageOp interface. • When performing Raster filtering, much of the functionality of the used filter will be defined by the RasterOp interface. • The following classes all implement both BufferedImageOp and RasterOp interfaces: • AffineTransformOp, RescaleOp, ConvolveOp, LookupOp • BandCombineOp only implements the RasterOp interface.

17. BufferedImageOp • The BufferedImageOp has the method filter to perform filtering: public BufferedImage filter( BufferedImage src, BufferedImage dest); • If the color models for the two images do not match, a color conversion into the destination color model is performed. If the destination image is null, a BufferedImage with an appropriate ColorModel is created. Returns the filtered BufferedImage.

18. AffineTransformOp • Affine transformations are linear transformations and are made up of rotations, translations, scalings, and shearings. A 2D affine transform is represented by the following equations:

19. AffineTransformOp - An example // Obtain the sourceBI (the source BufferedImage) AffineTransform at = new AffineTransform(); at.scale(2.0,2.0); BufferedImageOp bio; bio = new AffineTransformOp(at, AffineTransformOp.TYPE_BILINEAR); destinationBI = bio.filter(sourceBI,null); // ...

20. AffineTransformOp - Methods // rotate theta radians around the origin public void rotate(double theta); // rotate theta radians around the point x,y public void rotate(double theta, double x, double y); // scale by sx in x direction, and sy in y direction public void scale(double sx, double sy); // translate by tx in x direction, and ty in y direction public void translate(double tx, double ty); // shear using multipliers of shx and shy public void shear(double shx, double shy);

21. ConvolveOp • The java.awt.image.ConvolveOp class convolves a kernel with a source image in order to produce a destination image. • A kernel can be thought of as a two-dimensional array with an origin. During the convolution, the origin of the array is overlaid on each pixel of the source image. • This origin value is multiplied by the pixel value it is over, and all surrounding kernel array values are multiplied by the pixel values that they are over. • Finally, all these values are summed together and the resulting number replaces the pixel corresponding to the kernel centre.

22. Edge Conditions of ConvolveOp • When using convolution algorithms, edge pixels presents difficulty and hence some special instructions are needed as to how these edge pixels are handled. • There are two such instructions defined in ConvolveOp class: • EDGE_NO_OP - Pixels at the edge of the source image are copied to the corresponding pixels in the destination without modification. • EDGE_ZERO_FILL - Pixels at the edge of the destination image are set to zero. This is the default.

23. ConvolveOp • ConvolveOp(Kernel kernel) • ConvolveOp(Kernel kernel, int edgeCondition, RenderingHints hints) public Kernel(int width, int height, float[] data) • width - width of the kernel • height - height of the kernel • data - kernel data in row major order • The X origin is (width-1)/2 and the Y origin is (height-1)/2. • The RenderingHints class contains rendering hints such as interpolation type that can be used by the Graphics2D class, and classes that implement BufferedImageOp and Raster.

24. Example 2D Haar Kernels Where

25. Other Operations • RescaleOP : This class multiplies each pixel by a scaling factor before adding an offset to it. Mathematically, this can be expressed as follows: dstSample = (srcSample*scaleFactor) + offset • LookupOp : This provides a means to filter Rasters and BufferedImages using a lookup table(LUT). The LUT is simply an array in which the source pixel samples are treated as array indices. • ColorConvertOp : This class performs a pixel by pixel color conversion of the image source into the image destination. This is done by converting the pixels from the source image’s color space into the destination image’s color space.