1 / 60

Allen Hanson Chris Jaynes Mauricio Marengoni Edward Riseman Howard Schultz Frank Stolle

Allen Hanson Chris Jaynes Mauricio Marengoni Edward Riseman Howard Schultz Frank Stolle. Ascona 2001. University of Massachusetts 3D Building Reconstruction. With: Keith Hoepfner and Heddi Plumb. Talk Outline. Introduction: RADIUS and Ascender I Overview of Ascender II

abrewer
Download Presentation

Allen Hanson Chris Jaynes Mauricio Marengoni Edward Riseman Howard Schultz Frank Stolle

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Allen HansonChris JaynesMauricio MarengoniEdward RisemanHoward SchultzFrank Stolle Ascona 2001 University of Massachusetts 3D Building Reconstruction With: Keith Hoepfner and Heddi Plumb

  2. Talk Outline • Introduction: RADIUS and Ascender I • Overview of Ascender II • Geometric Reconstruction of Rooftops • Experiments • Conclusions Lab URL: http://vis-www.cs.umass.edu APGD URL: http://vis-www.cs.umass.edu/projects/ascender/ascender.html

  3. Research Goal(s) 3D Geometric Site Model Reconstruction With multiple objects, Using multiple strategies, From multiple images EO, Digital Elevation Maps IFSAR, Multi/Hyperspectral

  4. RADIUS: Ascender I System for Automatic Site Modelling Multiple Images Geometric Models • 2D polygon detection • line extraction • corner detection • perceptual grouping • epipolar matching • multi-image triangulation • geometric constraints • precise photogrammetry • extrusion to ground plane • projective texture mapping • CVIU 72(2) 1998

  5. Ascender Works Detection Rates • Extensive evaluation • Good detection but high false alarms • Good geometric accuracy Ground Truth Ascender Results

  6. Ascender Doesn’t Work • Delivered to NEL (NIMA) • Applied to classified imagery • Performance not as expected • High failure rate (buildings not detected) • High false positive rate • WHY? Imagery didn’t No Buildings Detected! conform to design constraints!

  7. Some Observations • IU Algorithms work within correct context • constrained contexts • constrained object classes • Domain knowledge provides constraints • from domain, partial results, strategies • Many strategies, only correct ones used • selective application • correct parameters • fuse results from individual strategies into complete reconstruction

  8. Key to Success A system which: applies the right strategy at the right time in the right place with the right constraints

  9. Ascender II Overview

  10. Dealing with Context: Ascender II • Goals • 3D Geometric Site Model Reconstruction • Focus: More complex building structures • Context sensitive control strategies for applying algorithms • Multiple strategies • Multiple images • Multiple sensors • Knowledge Representation, Control and Inferencing • layered Bayesian networks EO, Digital Elevation Maps IFSAR, Multi/Hyperspectral

  11. Where are the buildings? What is their structure? Observations Buildings = Location + Geometry Reconstruction = Localization + 3D Recovery 3D Recovery is a two-stage process

  12. Ascender II Overview • IU Algorithms work within correct context. • Domain knowledge provides constraints. • Choose good strategy from many alternatives. CP: Control Policy Basic Principles

  13. Ascender II Components • Visual Subsystem • Library of IU Algorithms • Geometric Database (data, models) • Display/user interface through RCDE • Knowledge Base • Control system • Belief networks in Hugin • Reasoning mechanisms over regions of discourse • Regions of Discourse • Subset of available data • Image regions, a model, partial interpretation

  14. Schema/Belief Node Possible Values H1={j,..,k} Variable of interest H1 Evidence Policy H3 H2 First level of classification hierarchy H4 Second level (object subclass) Knowledge Representation • Combination of belief networks and • visual schemas, implemented in Hugin. • Encodes: Domain Knowledge • Acquired Site Knowledge • Control Mechanism • Allows both diagnostic and • causal inference. • Evidence policy defines how • IU strategies gather relevant • evidence according to context. • Hierarchical topology allows simpler • networks which reduces propagation • time and controls complexity.

  15. Algorithms to Evidence Given an IU algorithm.... • Define the context in which it can run • currently straightforward • context only disallows an algorithm • too simple in the long run • Method for deriving certainty value • needed to update knowledge network

  16. Algorithms • Algorithms operate on regions of discourse • Ascender I • Surface Fits • Line Extraction and Count • Junction Count • Average Junction Contrast • Region Height • Region Height to Width ratio • Edge Terminals • ……………...

  17. 3D Geometric Rooftop Reconstruction

  18. Fusing DEM and Optical Data • Registered Optical and Elevation • Reconstruct Rooftops (and buildings) Optical Image (Fort Hood) Elevation Data (Fort Hood)

  19. Technical Overview Registered Optical and DEM Model Indexing Parameterized Model Library Elevation Surface and Histogram Site Model Visualization Model Fit: Optimization

  20. Surface Primitive Library 51 Models in 8 Parameterized Classes Gable

  21. Ascender + Terrest Results

  22. ExperimentsFort HoodAvenchesFort Benning

  23. Experiments • Approach • Site model from Ascender I with no filtering and loose polygon acceptance criteria => high detection rate but lots of errors (false positives) • Classify Ascender I regions into {building, parking lot, open field, unknown} • Classify buildings into {multi-level building, single-level building} • Then into {flat, peak, round, flat-peak, other} • Site model from Ascender II after classification/reconstruction

  24. 20 False Positive Buildings Degraded Ascender I ModelFort Hood No Knowledge and Loose Polygon Acceptance Criteria A 22 True Positive Buildings 1 Incorrect Model (A) (multi-level reconstructed as single level)

  25. First level of classification hierarchy Second level (object subclass) Bayes Nets for Experiments Level 1 Building Open Field Parking Lot Complex Unknown Region Evidence Policy: none Level 2 Low Medium High L Junctions Line Count Planar Fit Good Bad Multi-Level Building Width Height Single Evidence Policy: none Evidence Policy: Evidence Policy: multi-EO: triangulateHeight() DEM: medianHeight() DEM: robustFit(plane) Next Slide T Junctions Planar Fit Good Bad Yes No Evidence Policy: EO: junctions(T) <50 >50 Zero <5 >5 None Number T Junc. Contrast T Junc. Evidence Policy: Evidence Policy: EO: junction_count(T) EO: junction_contr(T)

  26. Yes No Height A > Height B Yes No Center Line Ratio Level 3 of Network Level 3 Flat Peak Round Flat-peak Other Multiple calls, one for each section Rooftop Evidence Policy: none Level 3 Good Bad Yes No Yes No Middle > Corner Planar Fit Center Line Multi- level Yes No Evidence Policy: Prism Round Flat Yes No Evidence Policy: none Shadow Format DEM: robustFit(plane) Side Line Good Bad Good Bad Planar Fit B Planar Fit A Evidence Policy: Evidence Policy: DEM: robustFit(plane) DEM: robustFit(plane)

  27. Height Visual Subsystem:Algorithms and Evidence Policies • Evidence policies associate algorithms with belief. • Function P maps request for evidence from knowledge base to algorithm. • F(x) maps algorithm result to belief value. • Example Algorithms: • Parallel Lines • T junctions • DEM-Median-Height • Region-Edge-Triangulate-Height • Planar Fit Error P{} P{} Height Stereo-Optical DEM DEM Median Height Multi-Optical Epipolar Align. Region-Edge Triangulate Hgt. USGS DTM DTM at region center F{x} x

  28. Experiments • Detection rates per Object Class (Semantic Accuracy) • Three-Dimensional Accuracy • For each DataSet • For each Object Class (buildings) • How do network priors influence performance? • Uniform (no knowledge) versus Estimated Priors • Adjustment of Priors for each dataset versus uniform • Three Datasets • Fort Benning MOUT • Fort Hood Facility • Avenches Boat Yard

  29. Priors and Operators Used Network Priors by Dataset Operators Applied by Dataset Skip F.Hood

  30. 43 Regions, 4 Misclassified, 1 Unknown 88% Accuracy Ascender II ClassificationContext Sensitive Strategies

  31. Ascender II Reconstruction 3 False Positives 23 True Positive Buildings 0 Incorrect Models (single, multi level) 3 New Functional Areas

  32. Site Reconstruction:Fort Hood

  33. Reconstruction Accuracy:Fort Hood Detection Rates by Object Class Median Accuracy Measures , Building Classes Skip Avenches

  34. ISPRS Avenches Dataset • Two Overlapping Optical Images (60% sideward) • Corresponding DEM • Calibration provided by ETH • 200x220 meter “Boatyard” • Ground Sample Dist: ~ 0.13 meters • Reference model provided by ETH

  35. Regions and Classifications:Avenches P e a k B l d g . P e a k B l d g . F l a t B l d g . F l a t B l d g . F l a t B l d g . P a r k i n g L o t F l a t B l d g . F l a t B l d g . O p e n F i e l d P e a k B l d g . P e a k B l d g . P e a k B l d g .

  36. Site Reconstruction:Avenches Skip Benning

  37. Fort Benning MOUT Site Optical DEM • Two overlapping downlooking optical images • DEM computed from image pair using Terrest • Camera resection (SRI) • Reference Model (SRI) • GSD, optical view, 0.31 meters

  38. MOUT Input Polygons Original Image Ascender I Polygons Final Polygons Polygons from SAR +

  39. MOUT Site: Classification Flat roof building detected as peak. Low walled entrance (open field) Detected as single vehicle. Missed detection of two vehicles. Object Class True False False Positive Positive Negative Building (General) 19 1 0 Peak Roof 14 1 0 Flat Roof 5 0 0 Single Car 1 1 2 Recovery of 4 attached buildings. Overall classification accuracy: 91%

  40. .939 .819 .742 .601 Plane Flat Peak Peak (15) Peak (35) Model Indexing: MOUT Region 8 Correlation Scores Corresponding Gaussian Sphere Constructed Surface Mesh

  41. MOUT Site: Accuracy Detection Rates, Full Evaluation Detection Rates, FOA Regions Only

  42. MOUT Site: Reconstruction

  43. MOUT Reconstruction Skip Kodak

  44. Hospital FineArts Examples on Three Image Chips Industrial N

  45. Building FP Hypotheses Optical Lines 2D Polygons + Building FP Hypotheses Buildings (st. lines) Trees (no st. lines) v Elevation Blobs Building Localization Trees Buildings Ground Final Building Location Hypotheses Building FP Hypotheses + Classifier: Registration FP = Footprint

  46. Geometric Recovery Building Localization: Polygons Canny Edges Grouped Lines Hypotheses Building Footprint On Range Data Polys Graph Group Focus (optional) Essentially Ascender I 2D Polygon Detection Algorithm with additional external inputs providing additional constraints. Classification

  47. Building Localization: Blobs Thresholded using bald earth model Range Data First Threshold Filtered using Long Lines and Region Size Bald Earth Model Lines from Optical

  48. Localization Results Hospital Industrial Fine Arts

  49. Final Building Models

  50. Conclusions • Vision works when properly constrained and focused • Still a lot of work to do on: • Basic Algorithms • Knowledge Representations • Representing and Using Context and Constraints • Inferencing and Causal Reasoning • System Architectures

More Related