1 / 33

Optimization and Video Processing in mMIPS: Extending LCC Compiler for Motion Estimation

This outline presents a comprehensive overview of optimizing video processing and I/O operations on the mMIPS architecture. It covers the extension of the LCC compiler focusing on motion estimation and algorithm/architecture codesign. Key concepts include spatial and temporal image processing, filter implementations like blurring and sharpening, and basic image storage methodologies. The work aims to enhance video quality by exploring pixel manipulation through algorithms, resulting in effective solutions for image processing tasks.

igor-barrera
Download Presentation

Optimization and Video Processing in mMIPS: Extending LCC Compiler for Motion Estimation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Optimization of the mMIPS Sander Stuijk

  2. Outline • Video processing • I/O operations on the mMIPS • Extending the LCC compiler • Assignment

  3. Video processing – motion estimation

  4. Video processing – algorithm/architecture codesign MBS + VIP MMI+AICP CAB MPEG 1394 Conditional access T-PI ASIP GP MSP area(mm2) 1.41 11.4 load(%) 31 37 M-PI eff area(mm2) 0.437 4.2 TriMedia VLIW power(mW) 6.0 124 MIPS bandwidth(MB/s) 135 75 picture-rate up-converter

  5. 400 pixels/line 40 pixels/line 300 lines 30 lines What is an image? • A black and white image is a matrix of luminance values • More pixels means higher image quality

  6. How do you store an image? • An image is a one dimensional pixel array width-1 0 y x width*height-1 Address: [y*width+x]

  7. How many bits do we need per pixel? Experiments: we can distinguish about 200 levels in an image We shall use 8 bit representation of luminance

  8. Video processing • Spatial domain • Image processing on a still image • Examples • Edge detection • Blurring • ... • Temporal domain • Image processing across different points in time • Examples • Motion estimation • Object recognition • ... The assignment deals with still images

  9. 3x3filter Example: filter coefficients are all “1” What does a 3x3 filter do with an image? • A 3x3 filter replaces each pixel (byte) in the file with the weighted sum of the pixel and its eight direct neighbors: • With: • And filter-coefficient Cline,pixel represented by one byte

  10. Blur filter Filter coefficient: +1 +1 +1 +1 +1 +1 +1 +1 +1

  11. C-code for blur filter for(int a=width+1; a<width*height-(width+1); a++){ result=(( 1* (int)buf_i[a-1-width] + 1* (int)buf_i[a-width] + 1* (int)buf_i[a+1-width] + 1* (int)buf_i[a-1] + 1* (int)buf_i[a] + 1* (int)buf_i[a+1] + 1* (int)buf_i[a-1 +width] + 1* (int)buf_i[a+width] + 1* (int)buf_i[a+1+width] +4 )/ 9); if(result<0) buf_o[a]=0; else if(result>255) buf_o[a]=255; else buf_o[a]=result; } clip weighted sum (pixel value) back to one byte

  12. Sharpening filter Filter coefficient: -1 -1 -1 -1 12 -1 -1 -1 -1

  13. Outline • Video processing • I/O operations on the mMIPS • Extending the LCC compiler • Assignment

  14. Input / output input image output image void main(void) { int a, result; char *buf_i = (char*)0x600000, *buf_o = (char*)0x665400; for (a=WIDTH+1; a < WIDTH*HEIGHT-(WIDTH+1);a++) { result=(( -1*(int)buf_i[a-1-WIDTH] + -1*(int)buf_i[a-WIDTH] + -1*(int)buf_i[a+1-WIDTH] + -1*(int)buf_i[a-1] + 12*(int)buf_i[a] + -1*(int)buf_i[a+1] + -1*(int)buf_i[a-1+WIDTH] + -1*(int)buf_i[a+WIDTH] + -1*(int)buf_i[a+1+WIDTH] + 128) / 4); if(result<0) buf_o[a] = 0; else if (result > 255) buf_o[a] = (char)255; else buf_o[a] = result; } }

  15. The input file with image data (name.y format) football.y bicycle.y File: {byte0,byte1,……..byten, bytewidth*height} Pixel left top Pixel right bottom Example: Two pixels above directly above each other: byten and byten+width

  16. Placing an image in the mMIPS memory • lcc -o mips_mem.bin image.c • imgproc.exe -i bicycle.y mips_mem.bin images Start of image in memory (mips_mem.bin)

  17. Extracting an image from the mMIPS memory • imgproc.exe -e mips_ram.dump bicycle.y • ImProc.exe

  18. Outline • Video processing • I/O operations on the mMIPS • Extending the LCC compiler • Assignment

  19. Adding a custom operation to LCC and the mMIPS void main(void) { int a, b, result; char *buf_i = (char*)0x600000, *buf_o = (char*)0x665400; for (a = 1; a < HEIGHT - 1; a++) { for (b = 1; b < WIDTH - 1; b++) { result=(( 5*(int)buf_i[(a - 1) * WIDTH + b - 1] + -3*(int)buf_i[(a - 1) * WIDTH + b ] + 6*(int)buf_i[(a - 1) * WIDTH + b + 1] + -7*(int)buf_i[ a * WIDTH + b - 1] + 11*(int)buf_i[ a * WIDTH + b ] + -7*(int)buf_i[ a * WIDTH + b + 1] + 6*(int)buf_i[(a + 1) * WIDTH + b - 1] + -3*(int)buf_i[(a + 1) * WIDTH + b ] + 5*(int)buf_i[(a + 1) * WIDTH + b + 1] + 128) / 13); /* Absolute value */ if(result<0) buf_o[a * WIDTH + b] = -result; else buf_o[a * WIDTH + b] = result; } } }

  20. Add a custom pattern to the C code #defineabs(a,b) ((a) - ((b) + *(int *) 0x12344321)) void main(void) { int a, b, result; char *buf_i = (char*)0x600000, *buf_o = (char*)0x665400; for (a = 1; a < HEIGHT - 1; a++) { for (b = 1; b < WIDTH - 1; b++) { result=(( 5*(int)buf_i[(a - 1) * WIDTH + b - 1] + -3*(int)buf_i[(a - 1) * WIDTH + b ] + 6*(int)buf_i[(a - 1) * WIDTH + b + 1] + -7*(int)buf_i[ a * WIDTH + b - 1] + 11*(int)buf_i[ a * WIDTH + b ] + -7*(int)buf_i[ a * WIDTH + b + 1] + 6*(int)buf_i[(a + 1) * WIDTH + b - 1] + -3*(int)buf_i[(a + 1) * WIDTH + b ] + 5*(int)buf_i[(a + 1) * WIDTH + b + 1] + 128) / 13); /* Absolute value */ result = abs(result,result); buf_o[a * WIDTH + b] = result; } } }

  21. Adding a custom operation (concept) LCC defines 4 constructs that map to custom operations in LCC: ((a) - ((b) + *(int *) 0x12344321)) ((a) + ((b) + *(int *) 0x12344321)) ((a) - ((b) - *(int *) 0x12344321)) ((a) + ((b) - *(int *) 0x12344321)) More operations (possibly with more operands) can be added. Look at the website for more information.

  22. Custom operations listed in the file lcc/src/minimips.md Add the custom pattern to the LCC compiler opcode = 0 function code = 0x31

  23. Adding a custom operation to LCC and the mMIPS void main(void) { int a, b, result; char *buf_i = (char*)0x600000, *buf_o = (char*)0x665400; for (a = 1; a < HEIGHT - 1; a++) { for (b = 1; b < WIDTH - 1; b++) { result=(( 5*(int)buf_i[(a - 1) * WIDTH + b - 1] + -3*(int)buf_i[(a - 1) * WIDTH + b ] + 6*(int)buf_i[(a - 1) * WIDTH + b + 1] + -7*(int)buf_i[ a * WIDTH + b - 1] + 11*(int)buf_i[ a * WIDTH + b ] + -7*(int)buf_i[ a * WIDTH + b + 1] + 6*(int)buf_i[(a + 1) * WIDTH + b - 1] + -3*(int)buf_i[(a + 1) * WIDTH + b ] + 5*(int)buf_i[(a + 1) * WIDTH + b + 1] + 128) / 13); /* Absolute value */ if(result<0) buf_o[a * WIDTH + b] = -result; else buf_o[a * WIDTH + b] = result; } } }

  24. Add a custom pattern to the C code #defineabs(a,b) ((a) - ((b) + *(int *) 0x12344321)) void main(void) { int a, b, result; char *buf_i = (char*)0x600000, *buf_o = (char*)0x665400; for (a = 1; a < HEIGHT - 1; a++) { for (b = 1; b < WIDTH - 1; b++) { result=(( 5*(int)buf_i[(a - 1) * WIDTH + b - 1] + -3*(int)buf_i[(a - 1) * WIDTH + b ] + 6*(int)buf_i[(a - 1) * WIDTH + b + 1] + -7*(int)buf_i[ a * WIDTH + b - 1] + 11*(int)buf_i[ a * WIDTH + b ] + -7*(int)buf_i[ a * WIDTH + b + 1] + 6*(int)buf_i[(a + 1) * WIDTH + b - 1] + -3*(int)buf_i[(a + 1) * WIDTH + b ] + 5*(int)buf_i[(a + 1) * WIDTH + b + 1] + 128) / 13); /* Absolute value */ result = abs(result,result); buf_o[a * WIDTH + b] = result; } } }

  25. Open een cygwin shell en voer de volgende commando’s uit: lcc image.c –o mips_mem.bin disas mips_mem.bin | less Zoek een assembler instructie met opcode 0 and function code 0x31 Controleer dat clipping gebruikt wordt

  26. Adding a special function to the mMIPS (hardware) aluctrl alu

  27. Outline • Video processing • I/O operations on the mMIPS • Extending the LCC compiler • Assignment

  28. Assignment Optimize the run-time of an image processing algorithm running on the mMIPS without reducing the instruction set supported by the hardware. Constraints • All programs that run on the original mMIPS must also run on your design. • Programs running on your design and the original mMIPS must produce bit-exact output Allowed • Adding special instructions to the mMIPS; • Changing the design of the mMIPS (e.g. forwarding). Not-allowed • Modification of the image processing algorithm that are not needed to use special instructions (e.g. replace multiply with shifts).

  29. Testing and implementing the design Test for functional correctness • Run the original mMIPS with the algorithm to produce a reference output. • Compare the results of your mMIPS to the reference output. • You can use the check-image utility for this purpose. • Make sure that your image is large enough to cover all possible cases. Synthesize your design • You must synthesize your design to determine the maximum clock frequency at which your mMIPS can run. • This determines part of the speed-up.

  30. Use the submit-design utility to submit your mMIPS design You can submit a new design as often as you like, but only your last design will be tested Submitting your design

  31. Important dates • Midterm meeting on March 16th in Pav b1 from 10.45 till 12.30. • Submit the first version of your modified mMIPS on March 18th before noon. • This version must at least contain: • a working forwarding unit, • a working custom clipping instruction. • A separate document (A4, 10pt font, max 2 pages) with a description of all changes made, or planned, or investigated • Provide a short description of the required changes • Provide a short motivation for the change • Explain the expected performance gain • Send document to s.stuijk@tue.nl • Instructions at www.es.ele.tue.nl/education/5JJ55-65/mmips • If you submit a design and document, then you will get feedback on your ideas!

  32. Important dates • Submit the final version of your modified mMIPS on April 1st before noon. • Instructions at www.es.ele.tue.nl/education/5JJ55-65/mmips • This is a hard deadline, no extension is possible! • Individual presentation of your mMIPS on April 4-5th • The presentation is about the changes you made to the mMIPS. • Do not talk about the book, forwarding or clipping.

  33. Every Monday from 15.45 till 17.30 in PT 9.10 Check also www.es.ele.tue.nl/education/5JJ55-65/mmips for more information, hints, etc. Support and information

More Related