Frank Boochs i3mainz , Institute for Spatial Information and Surveying Technology

1. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images Frank Boochs i3mainz, Institute for Spatial Information and Surveying Technology Holzstr. 36, D-55116 Mainz, Germany boochs@geoinform.fh-mainz.de

2. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images content • motivation • conception • aspects of solution • data processing • practical aspects • examples Journée TSI "Eclairage structuré pour la vision active"

3. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images motivation area of application: image based surveying of spatial objects needs: determination of orientation values for all images to be used is based on: image rays to tie and control points Journée TSI "Eclairage structuré pour la vision active"

4. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images motivation area of application: image based surveying of spatial objects Journée TSI "Eclairage structuré pour la vision active"

5. manual identification and measurement of natural object points signals are: • templates of a certain diametre • used to define a point (centre) • not used with their spatial data (diametre, area) computer based detection of artificial signals Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images motivation collection of image rays by means of: Journée TSI "Eclairage structuré pour la vision active"

6. image noise in particular for arbitrary arranged images: left view right view • perspective distortion Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images motivation reduction of success due to: Journée TSI "Eclairage structuré pour la vision active"

7. distortion: limitation of angular differences • for optical axes of different images • for surface normal and individual optical axis Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images motivation conventional ways of compensation: • image noise: improvement of S/N ratio (sensor, light,...) Journée TSI "Eclairage structuré pour la vision active"

8. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images motivation compensation leads to increasing effort: • quality of image rays is degraded • reduced connection between images • failure to connect two images • increasing number of images • increasing number of signals in cases of reduced success consequences have to be expected: Journée TSI "Eclairage structuré pour la vision active"

9. sphere: • substitutes target plane • visible from all directions • defines one image ray body with n faces: • defines multiple target planes • visible from all directions • defines several image rays • introduces additional information (scale, local co-ordinate system, orientation in space) Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images conception idea: improvement of connectivity be means of spatial tie objects Journée TSI "Eclairage structuré pour la vision active"

10. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images conception we use of spatial tie objects with several faces advantages : • multiple surfaces useable • at least one surface with a small angle to the optical axis • allows completely new arrangements Journée TSI "Eclairage structuré pour la vision active"

11. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images conception essential requirement which has to be fulfilled: the targets have to be treated and detected as complete spatial objects • the whole body (= all visible surfaces) has to be detected • visible surfaces have to be uniquely identified • all surfaces need to have a fix and known topology Journée TSI "Eclairage structuré pour la vision active"

12. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images aspects of solution type of target chosen: cube because of its: • symmetry • equal size of faces • number of faces • angular difference of adjacent faces Journée TSI "Eclairage structuré pour la vision active"

13. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images aspects of solution discrimination of individual surfaces by means of : coloring because: • information is pixel based • sufficient number of separable colors available (red, green, blue, cyan, magenta, yellow) Journée TSI "Eclairage structuré pour la vision active"

14. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images aspects of solution discrimination of individual surfaces by means of : alternative: active ligthing because: • independent of ambient light • easier analysis of colour values • more robust Journée TSI "Eclairage structuré pour la vision active"

15. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images aspects of solution further benefit: simple geometry • simplifies modelling within different stages of analysis • allows to introduce effective constraints Journée TSI "Eclairage structuré pour la vision active"

16. target data (co-ordinates, attri-butes, codes,..) image image image analysis image image preliminary orientation values, object co-ordinates correspondence analysis adjusted orientation values, object co-ordinates triangulation Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images data processing general algorithmic structure: Journée TSI "Eclairage structuré pour la vision active"

17. edge detection • vectorisation • evaluation of color attributes vectorised edges attributed edges image (part) • verification of hypothesis • calculation of geometry • generation of cube hypothesis final cube cube candidate inner cube edges outer edge candidates Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images data processing image analysis steps: Journée TSI "Eclairage structuré pour la vision active"

18. original saturation brigthness saturation & brightness noise reduction • edge detection • calculation of geometry final cube contour filtering contour extraction segmentation Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images data processing image analysis steps (active lighting): Journée TSI "Eclairage structuré pour la vision active"

19. correspondence analysis • simplified by: • grouping of points • geometry constraints • target knowledge image triangulation based on standard algorithms: actually : CAP Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images data processing further steps: Journée TSI "Eclairage structuré pour la vision active"

20. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images practical aspects color information • simplifies detection • provides binary code • characterizes shape / relation of sides of targets • for passive coding: • may vary considerably (direction of illumination, type of light, sensor characteristic and state) • for active coding: • luminosity and homogeneity need to be controlled (overexposure, reaction of sensor) Journée TSI "Eclairage structuré pour la vision active"

21. has to be sufficient • size of faces • geometric quality of edges Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images practical aspects size of targets • has to be considered with respect to possibly strong variations within image scale Journée TSI "Eclairage structuré pour la vision active"

22. provides useful information within analysis • parallel edges • right angles at adjacent edges • constant edge length • further distances only for quality check • each cube allows to define 2 groups of 2 distances in each direction of space • distances are scattered over object space together with targets Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images practical aspects spatial characteristic of targets • introduces scale into image block Journée TSI "Eclairage structuré pour la vision active"

23. each target may define a local co-ordinate system • -> allows orientation to a local system without any further preparations within object space • already one target provides data for preliminary orientation • -> allows to estimate orientation values for images in a common system Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images practical aspects benefits from spatial shape of targets (spatial extent) Journée TSI "Eclairage structuré pour la vision active"

24. rays of each target are covering a local image area • -> provides control information for local image geometry • each target is represented by 8 rays • -> gives redundancy and allows to control quality of each point Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images practical aspects benefits from spatial shape of targets (multitude of image rays) Journée TSI "Eclairage structuré pour la vision active"

25. provides deepest connection of images possible • -> all images will be connected having sight onto a certain target • allows to optimize image arrangements simply from viewpoint of best geometry • -> constraints coming from sight onto tie points are reduced Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images practical aspects benefits from independence of viewing direction Journée TSI "Eclairage structuré pour la vision active"

26. allows complete new image arrangements • -> images with opposite viewing directions might be combined Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images practical aspects benefits from independence of viewing direction Journée TSI "Eclairage structuré pour la vision active"

27. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images practical aspects fields of application • industrial measurements • forensic tasks • facility mapping • architecture • stereoscopic visualisation (->consumer market) Journée TSI "Eclairage structuré pour la vision active"

28. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images example test: pair of images with 900 convergency 4 common targets used Journée TSI "Eclairage structuré pour la vision active"

29. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images example test: pair of images with opposite viewing directions orientation performed with minimal data (1 target) Journée TSI "Eclairage structuré pour la vision active"

30. Detection of spatial reference objects for automatic orientation of arbitrary arranged metric images thanks for your attention Journée TSI "Eclairage structuré pour la vision active"