Acknowledgements

1. Privacy Protected Video SurveillanceSen-ching Samson Cheung Center for Visualization & Virtual EnvironmentsDepartment of Electrical & Computer Engineering University of Kentuckyhttp://www.vis.uky.edu/mialab

2. Acknowledgements • Graduate Students • Vijay Venkatesh Mahalingam • Jian Zhao • Jithendra K. Paruchuri • Research Support • Department of Homeland Security • Oak Rridge Associated Universities

3. What is privacy? • To develop human excellence without interference [Aristotle’s Politics 350 B.C.] • Control over information about oneself [Warren and Brandeis 1890] …the right most valued by all civilized men — the right to be let alone. - U.S. Supreme Court Justice Louis Brandeis, 1890

4. Today’s privacy concerns • Electronic Voting • E-commerce • Medical Records • Financial Records • Cyber Activities

5. Tomorrow’s privacy concerns • Smart video surveillance • Biometric theft • Multimedia processing • RFID tracking

6. Privacy Protection Technology • Technologies that aim at protecting personal privacy without compromising the “legitimate” use of information. • Main PPT include the followings: • Encryption Tools • Platform for Privacy Preferences (P3P) • Automated Privacy Audit • Anonymizer • Privacy Protected Data mining Is privacy protection of multimedia any different?

7. Challenges from Multimedia • What to protect? • Identify semantic objects for protection • How to protect it? • Reliable protection without losing perceptual utility • How to control it? • Flexible control and secure authentication of privacy data

8. Talk Overview • Video Surveillance • Subject Identification • Optimal Camera Network • Video Obfuscation • Privacy Data Management • Portable AV devices • Evaluation of audio privacy protections • Secure Distributed Processing

9. Overview Subject Identification Module Obfuscation Object Identification and Tacking Secure Camera System Secure Data Hiding Surveillance Video Database Privacy Data Management System

10. Talk Overview • Video Surveillance • Subject Identification • Optimal Camera Network • Video Obfuscation • Privacy Data Management

11. Video Obfuscation In-painted Black Out Original Pixelation/ Blurring

12. Challenges of Video Inpainting

13. Dynamic Object In-painting • Basic idea: Using object template extracted form other time instant to complete a conceptually consistent sequence. • Steps: 1. Similarity based on optimal alignment 2. Motion continuity 3. Positioning of templates ? ... ... ...

14. Motion Continuity ? ?

15. Object-based Video In-painting • Better motion in-painting by better registration and task separation • Capable to in-paint partially and completely occluded objects • Improved computational performance (Matlab) Number of frames with complete occlusion Number of frames with partial occlusion

16. Public-domain Sequences

17. Complex Sequences

18. Talk Overview • Video Surveillance • Subject Identification • Optimal Camera Network • Video Obfuscation • Privacy Data Management

19. Privacy Data Management Client A Client B Client C Privacy Protection Subject A Producer Subject B Subject C Key question: How does a client know which subject to ask?

20. Three-agent architecture Step 3 U RSA(K; PKU) TOC: U RSA(K; PKU) Step 4 RSA(K; PKU) Step 2 RSA(TOC; PKm) RSA(TOC; PKm) Step 1 RSA(K;PKC) RSA(K;PKC) Step 6 Step 7 RSA(K; PKC) Step 5 Mediator Agent PKM,SKM AES(Vu; K) Subject Agent PKU,SKU Client Agent PKC,SKC AES(Vu; K)

21. Keeping sensitive information

22. Data Hiding for Privacy Data Preservation

23. Data Hiding • Data hiding/Stenography/Watermarking • Active research in the past fifteen years • Typical applications include authentication, copy detection, monitoring • Challenges in our application: • Picture-in-picture: large embedding capacity • Compatibility with existing compression scheme • Minimal visual distortion

24. Optimal Data Hiding Combined rate- distortion cost C(x) # embedded bits Block-based Rate-Distortion Calculation Psycho-visual Modeling Discrete Optimization Solve constrained optimization

25. Proposed Data Hiding Encrypted foreground video bit-stream Parity Embedding Last decoded frame Positions of the “optimal’ DCT coeff for embedding DCT R-D Optimization Perceptual Mask • DCT Domain • Frequency, contrast and • luminance masking [Watson] DCT(i,j) = watermark_bit+ 2*round(DCT(i,j)/2) Privacy protected video DCT Entropy Coding Motion Compensation H.263 H.263 J. Paruchuri & S.-C. Cheung “Rate-Distortion Optimized Data Hiding for Privacy Protection” submitted to ISCAS 2008

26. R-D framework • Target cost function: • Ri = Increase in Bandwidth of Block i • Di = Perceptual Distortion in Block i • δ = Relative Weight • Greedy embedding of P data bits in Block i: • Lagrangian optimization: determine the optimal Pi and λ to embed the target number of data bits:

27. Examples 1/2 Distortion 637 kbps 81 kbps data 119kbps No data Rate & Distortion 562 kbps 81 kbps data Rate only 370 kbps 81 kbps data

28. Examples 2/2 Distortion 743 kbps 81 kbps data 406.3kbps No data Rate & Distortion 678 kbps 81 kbps data Rate only 610 kbps 81 kbps data

29. Conclusions • Privacy Protecting Video Surveillance • Visual Tagging for subject identification • Optimized camera network for visual tagging • In-painting for video obfuscation • Privacy Data Management • R-D optimized watermarking • Current Research • Video In-painting in crowded environment • Performance Evaluation for PPT • Secure Reversible Modification • Audio Privacy Protection • Signal processing in encrypted domain