Automated Reconstruction of Industrial Sites

1. Automated Reconstruction of Industrial Sites Frank van den Heuvel Tahir Rabbani

2. Overview • Introduction • Automation: how does it work? • Sample project off-shore platform • Accuracy • Future • Conclusions

3. The groupPhotogrammetry & Remote Sensing • “Development of efficient techniques for the acquisition of 3D information by computer-assisted analysis of image and range data“

4. The projectServices and Training through Augmented Reality (STAR) • EU fifth framework – IST programme • “Develop new Augmented Reality techniques for training, on-line documentation, maintenance and planning purposes in industrial applications” • AR-example: virtual human in video

5. The projectServices and Training through Augmented Reality (STAR) • Partners: Siemens, KULeuven, EPFL, UNIGE, Realviz • TUDelft: “Automated 3D reconstruction of industrial installations from laser and image data”

6. Automated reconstruction procedureOverview (1/3) • Segmentation • Grouping points of surface patches

7. Automated reconstruction procedureOverview (2/3) • Segmentation • Grouping points of surface patches • Object Detection • Finding planes and cylinders

8. Automated reconstruction procedureOverview (3/3) • Segmentation • Grouping points of surface patches • Object Detection • Finding planes and cylinders • Fitting • Final parameter estimation

9. Segmentation – step 1 • Estimation of surface normals using K-nearest neighbours (here K=10 points)

10. Segmentation – step 2 • Region growing using: • Connectivity (K-nearest neighbours) • Surface smoothness (angle between normals)

11. Detection – Planes • Plane detection using Hough transform • Find orientation as maximum on Gaussian sphere

12. Detection – Cylinders • Cylinder detection using Hough transform in 2 steps: • Step 1: Orientation

13. Detection – Cylinders • Cylinder detection using Hough transform in 2 steps: • Step 1: Orientation

14. u,v search space at correct Radius Detection – Cylinders • Cylinder detection using Hough transform in 2 steps: • Step 1: Orientation (2 parameters) • Step 2: Position and Radius (3 parameters)

15. Example: detection of two cylinders • Point cloud segment

16. Example: detection of two cylinders • Surface normals

17. Example: detection of two cylinders • Normals on Gaussian sphere

18. Example: detection of two cylinders • Orientation of first cylinder (next: position)

19. Example: detection of two cylinders • Remove first cylinder points from segment

20. Example: detection of two cylinders • Procedure repeated for second cylinder

21. Example: detection of two cylinders • Result: two detected cylinders

22. Fitting • Complete CSG model + constraint specification • Final least-squares parameter estimation of CSG model

23. Fitting • Final least-squares parameter estimation of CSG model • Minimise sum of squared distances • Enforce constraints

24. Results on platform modelling • Scanned by Delftech in 2003 • Subset of 17.7 million points used by TUD: • Automated detection of 2338 objects • R.M.S. of residuals 4.3 mm

25. Results on platform modelling

26. Results on platform modellingStatistics • Points: 17.7 million • Points in segments: 14.2 million (80%) • Points on objects: 9.3 million (53%) • Detected: • Planar patches: 946 • Cylinders: 1392 • Data reduction: • Object parameters 9798 • 500 Mb to 0.1 Mb

27. Cumulative histogram of residuals 100 90 80 70 60 %age of points 50 40 30 20 10 0 0 2 4 6 8 10 12 14 16 18 20 Residual (mm) Results on platform modelling Accuracy • Residual analysis: • RMS: 4.3 mm • 83% < 5 mm • 96% < 10 mm

28. Accuracy • Data precision: • Scanner: 6 mm (averaging: 3 mm) • Scanner dependent • Model precision: • Discrepancies models - real world: 0.1-10 mm ? • Limited production accuracy • Deformations • Imperfections in segmentation

29. Accuracy • Object deformation or segmentation limitations? Fitting after initial segmentation Max.residual 21 mm Fitting after rejecting large residuals Max. residual 9 mm

30. Future – automation • Reconstruction using laser data: • Segmentation, primitive detection (available) • Correspondence between primitives > registration • Model improvement: • Constraint detection (piping structure) • Recognition of complex elements in a database • Integration with digital imagery

31. Future – integration with imagery • Instrumentation developments • Scanners with integrated high-resolution digital camera • Accuracy improvement • Imagery complementary: Laser for surfaces, image for edges • Integrated fitting of models to laser and image data

32. Future – integration with imagery • Instrumentation developments • Scanners with integrated high-resolution camera • Accuracy improvement • Imagery complementary: Laser for surfaces, image for edges • Integrated fitting of models to laser and image data • Flexibility of image acquisition: Completeness • Non-geometric information (What is there?)

33. Future – integration with imagery

34. Conclusions • Bright future for automation using laser data • More research to be done: • Automated registration • Integration with digital imagery • Using domain knowledge for automated modelling: • Closer connection to the model users needed: • Domain knowledge for automation • Utilisation of research results