210 likes | 338 Views
A High-performance Multi-perspective Vision Studio. An Efficient System for Multi-Perspective Imaging and 3D Shape Analysis. Multi-view vision. interesting affordable challenging distributed. Multi-perspective environments. Keck Lab 64 cameras 85 frames/sec 1 min = 95GB.
E N D
A High-performanceMulti-perspective Vision Studio An Efficient System for Multi-Perspective Imaging and 3D Shape Analysis Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Multi-view vision • interesting • affordable • challenging • distributed Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Multi-perspective environments Keck Lab • 64 cameras • 85 frames/sec • 1 min = 95GB Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Volume reconstruction • multi-perspective • silhouette-based • visual cone intersection • special octree encoding Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Volume reconstruction Background subtraction Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Volume reconstruction Multi-perspective silhouette extraction - = - = - = Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Volume reconstruction Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Volume reconstruction image plane Visual cone construction Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Volume reconstruction 3D occupancy map as octree image plane Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Volume reconstruction resolution=8depth Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Data Capture Data Capture Loader Back-end services Front-end services Database Client Client Multi-perspective Vision Studio • Features • abstraction from data acquisition • multi-view sequence management • extensible application framework • based on ADR and DataCutter • Applications • Volumetric shape reconstruction • 3D density-based model fitting • Texture mapping surface meshes Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Data de-clustering based on Hilbert space-filling curve 1 2 3 4 time index 5 6 7 8 1 2 3 4 5 6 7 8 camera index Customizable Studio Server • Data elements (chunks): image<cam-ndx,time-ndx> • Loader: Hilbert curve based de-clustering algorithm • Parallel back-end: database engine • index: (x,y,z,t) -> (cam,time) • aggregation: associative&commutative • Application front-end: gateway • query: application dependent • result: AppFE node is optional Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Client GUI Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Server performance Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Constant work load performance 12000 10000 8000 seconds 6000 4000 2000 0 2 4 16 number of processors 8 8 frame group size 4 16 2 Server performance Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
A density fitting example Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
(consistency) (conservation) Density based shape modeling given a volume V, fit a density f by solving Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Hierarchical fitting Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Density based modeling results Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Mesh texture coloring Eugene Borovikov, Alan Sussman and Larry Davis, UMCP
Conclusions Multi-perspective vision studio • abstracting vision application from sensor array • portability across parallel platforms • robustness in handling large datasets • expandable functionality High performance comes from • effective data de-clustering (Hilbert curve) • frame grouping to improve workload balance • efficient voxel projection strategy Eugene Borovikov, Alan Sussman and Larry Davis, UMCP