TLS Data Processing Modules

1. TLS Data Processing Modules Armin Gruen & Zhang Li Institute of Geodesy & Photogrammetry Swiss Federal Institute of Technology Zurich, Switzerland E-Mail: agruen,zhangl@geod.baug.ethz.ch

2. Contents Introduction  Georeferencing by Aerial-Triangulation  Image Matching  DTM Generation  Ortho Image Generation  Conclusions

3. IntroductionOverview of some line-scanner digital sensor systems

4. Introduction Main Features of Airborne Digital Sensors pushbroom & three-line scanner principle. High area coverage performance (field of view and stereo angle). High resolution and accuracy (spatial & radiomatic). Stereo Imaging capability. Multispectral imaging capability. Direct digital workflow avoiding the film development and scanning. Accuracy oriented sensors, using GPS/INS integration.

5. Introduction TLS (Three Line Scanner) System A new airborne digital line-scanner developed by STARLABO Corporation of Japan.  Imaging system Focal length: 60.0mm Field of view: 61.5deg Stereo lines: 3 MS lines: 3 CCD-line pixels: 10200 Pixelsize: 0.007mm Stereo angle: 21.0deg

6. Introduction Sensor configurationJAE (Japan Aviation Electronics Industries) ACE3000TLS stabilizer. (500HZ attitude data)  a Trimble MS750 serves as Base GPS. (L1/L2 kinematic data at 5HZ)  a Trimble MS750 serves as Rover GPS.  Software Trimble Geomatics Office calculate the kinematic position.  Trimble TanzVector serve as Aircraft attitude sensor data.

7. Introduction Characteristics of TLS system TLS system can obtain three different viewing directional Seamless high-resolution (about 5-10cm in air-borne situation) images.  The raw imagery of TLS stereo channel can be directly used for stereo measurement and image matching procedure due to the employee of high quality stabilizer and no additional image rectification procedure is needed. TLS system contains the GPS/INS integrated system, this system can produce an accurate estimates for the sensor position and altitude, so much less ground control points is needed. After the precise recovery of system calibration parameters, it enables the direct georeferencing of TLS images. TLS system records digital image directly, which enable users to easily process and analyze them on the real-time basis and help minimize the processing errors. TLS can also acquire the multi-spectrum images.

8. Introduction

9. Georeferencing by Aerial-TriangulationIndirect Georeferencing: In traditional photogrammetric triangulation, the georeferencing problem is solved indirect using some well-distributed ground control points and applying geometric constraints such as colinearity equations between the image points and object points.

10. Georeferencing by Aerial-TriangulationDirect Georeferencing: The integration of INS/GPS can reach a very high absolutive accuracy. For GPS, using the differential phase observations with rover-master receiver separation below 30 km, 10 cm or even better absolute positional accuracy in airborne kinematic environments can be achieved. For high quality INS 15 arc sec can be achieved. Translational offsets between GPS, INS and camera Rotational offset between axes of INS and camera

11. Georeferencing by Aerial-TriangulationInterior Orientation Parameters of TLS Focal length f, Central points coordinate x0, y0 for each CCD arrays attached to the focal plane, Inclination angle  for each CCD arrays to the image y axis, and Lens distortion correction coefficients: a1,a3 and a5. Central Point

12. Georeferencing by Aerial-Triangulation Georeferencing of the TLS Imagery The geometry of the TLS imagery is weaker compared with the traditional film-based frame imagery. There are only 3 image lines be recorded simultaneously and share the same exterior orientations The orientations of All image lines should be recovered Piecewise polynomial model Interpolation model Airborne digital sensors have to be integrated with high accuracy INS & GPS

13. Displacement vector between GPS and camera Georeferencing by Aerial-Triangulation TLS Sensor Configuration Boresight misalignment between INS and camera

14. r0 + rres Georeferencing by Aerial-Triangulation GPS/INS/Camera Translational Displacement Correction GPS/INS displacement vector correction using the aircraft attitude data (Tanzvector data, RMS of directional error is 0.3 deg)  INS/Camera displacement vector correction using the output of INS

15. rres rotation matrix from camera frame to INS body. 0, 0, 0 is the boresight misalignment between axes of INS and Camera. r0 + rres Georeferencing by Aerial-TriangulationModel 1: Direct Georeferencing Model (DGR) X0-2,Y0-2,Z0-2 are unknowns to model the residual errors after the GPS/INS/Camera displacement correction. rotation matrix from INS body to ground coordinate frame, INS, INS, INS is the original altitude which from INS sensor. unknowns 1, 1, 1 is used to model INS draft errors.

16. Georeferencing by Aerial-TriangulationModel 1: Direct Georeferencing Model (DGR) Colinearity equations: (X,Y,Z) (x,y)

17. Georeferencing by Aerial-TriangulationModel 1: Direct Georeferencing Model (DGR) Vectors of unknowns Coefficient martices for unknown vectors Weight martices Observation vectors

18. rres rres r0 + rres Georeferencing by Aerial-TriangulationModel 2: Piecewise Polynomial Model (PPM) coordinates of perspective center denoted in the ground coordinate system, XSi0,YSi0,ZSi0;XSi1,YSi1,ZSi ;XSi2,YSi2,ZSi2 are 0,1 and 2-order unknown polynomials coefficients of ith segment to model the perspective center. rotation matrix from ground frame to camera frame

19. Georeferencing by Aerial-TriangulationModel 2: Piecewise Polynomial Model (PPM) There are 2 kinds of constraints that may be applied to each parameters at the segment boundaries. The zero order continuity constraints ensure that the value of the function computed from the polynomial in each 2 neighboring segments is equal at their boundaries, i.e. The first order continuity constraint required that the slope, or first order derivative, of the functions in 2 adjacent segments be forced to have the same value at their boundary, i.e.

20. rres rres rres r0 + rres Georeferencing by Aerial-TriangulationModel 3: Cubic Spline Interpolation Model (CSI)The Cubic Spline Interpolation model computes the orientation parameters of reference image lines at regular intervals, then calculates the parameters of intermediate image lines by cubic polynomial interpolation. These reference image lines are so-called "orientation fixes"

21. Georeferencing by Aerial-TriangulationTest Flight Area The GSI Geographical Survey Institute) testing area.  The testing area covers about 6502500m2 The image scale is 1:8000 and the footprint is about 5.6cm.  All control points are signalised.

22. Georeferencing by Aerial-TriangulationSignalised Control Points

23. Georeferencing by Aerial-TriangulationSemi-automatic Tie Point extraction Feature point extraction. The user can select a point in one image , software can automatically derive the nearest feature points in a image window sized 4141 pixels using Forstner interest operator. Imagepyramid generation and pixel level accuracy conjugate points generation. Subpixel accuracy matching by Least Squares Matching. 6 geometric parameters are used in the adjustment. Using the above semi-automatic approach, several hundards of tie points (standard deviation of 0.2-0.3 pixel) are extracted in an interactive way.

24. Georeferencing by Aerial-TriangulationSemi-automatic Tie Point extraction

25. Georeferencing by Aerial-TriangulationSemi-automatic Tie Point extraction Forward Nadir Backward

26. Georeferencing by Aerial-TriangulationSemi-automatic Tie Points extraction Forward Nadir Backward

27. Georeferencing by Aerial-TriangulationAccuracy of the aerial-triangulation  By modeling the remaining errors after GPS-INS, INS-Camera displacement vector correction as a constant offset value, about 0.10, 0.07m and 0.08m absolute accuracy in planar and heightwas achieved using DGR model.  Using PPM and CSI model, better results, about 0.08, 0.05m and 0.07m absolute accuracy in planar and heightwas achieved, especially the 20 sections PPM model, about 1 pixel accuracy was achieved.  Better accuracy results can be achieved by using more sections in PPM model and more orientation fixes in CSI model.

28. Georeferencing by Aerial-TriangulationAccuracy of the aerial-triangulation

29. Georeferencing by Aerial-TriangulationAccuracy of the aerial-triangulation

30. Georeferencing by Aerial-TriangulationAccuracy of the aerial-triangulation

31. Georeferencing by Aerial-TriangulationAccuracy of the aerial-triangulation

32. Georeferencing by Aerial-TriangulationAccuracy of the aerial-triangulation

33. Level 1 Image MatchingImage Pyramid Generation Enhancement of the texture in original image (level 0) using the Wallis filter. Image pyramid generation by reduction factor 2, including the original image, the pyramid level is fixed to 4. Level 3 upper level Level 2 lower level Level 0

34. Image MatchingFeature Points Extraction in Upper Level Image

35. Image MatchingFeature Points Extraction in Upper Level Image

36. Image MatchingSearch Area Determination in Upper Level Image Precise position/attitute data after aerial triangulation procedure. Coarse DEM/TIN data generated from the measured tie & control points. Flight Trajectory Forward Image Nadir Image Backward Image Approximate Height +dh -dh

37. Image MatchingPixel level accuracy conjugate point generation by cross- correlation method Several candidate points are selected according to their normalized correlation coefficients using epipolar line constraints  Pixel level accuracy conjugate points determined by forward intersection forward nadir backward incorrect match correct match

38. Image MatchingTranformation of the matching result from upper level pyramid image to lower level image Feature point extraction in lower level image.  Search area determined from parallax results at upper level pyramid image lower level feature upper level feature

39. Image MatchingLeast squares matching in the original image After the correlation in highest level pyramid image, blunders are deleted by forward intersection procedure.  All 6 geometric parameters (shift & shape parameters) are used in adjustment, some bundles are deleted in this stage.  As a result, 12994 point triples were extracted by using the test imagery of GSI. The standard deviation is about 0.2-0.6 pixels.

40. Image MatchingSome examples

41. Image MatchingSome examples

42. DTM GenerationEach matched point is a result of 3 imaging rays. Using the forward intersection procedure, the object coordinates of these matching points can be obtained. TIN data can be generated by a sufficient number of object points, then using two-dimensional interpolation algorithms, grid DTM data can be generated.

43. DTM Generation

44. DTM Generation

45. S p Z1 P Z2 Z0 (X,Y) Ortho-Image GenerationDirect method X=f(x,y); Y=g(x,y) y Y X x Original Image Ortho-Image

46. Resampling Nearest neighbour, Bilinear & Cubic Resampling y Y Iterative Search Algorithm x=f(X,Y); y=g(X,Y) X Orthorectified Image Original Image Ortho-Image GenerationIndirect method This is our ortho-image generation method of choice x

47. Ortho-Image GenerationBack-projection by an iterative search algorithmThe location of the corresponding scan line of a certain object point can be found with an iterative search algorithm. After starting with an initially approximate scan line in TLS image, the final corresponding scan line can be found by minimizing the perpendicular distance to the corresponding CCD line.

48. Ortho-Image GenerationBack-projection by an iterative search algorithm x-coordinate Scan line number Trajectory P (X,Y,Z) Ortho-image pixel

49. Ortho-Image GenerationExamples Original Image Orthorectified Image

50. Ortho-Image GenerationExamples Original Image Orthorectified Image