1 / 36

Optical Flow in Motion Analysis 16.2.4 presented by Mirek Cymer

May 5, 2011. Computer Vision Lecture 23: Optical Flow/Texture Challenge. 2. Overview. Optical FlowFocus of Expansion (FOE)Mutual VelocityCollision Prediction. May 5, 2011. Computer Vision

kahlilia
Download Presentation

Optical Flow in Motion Analysis 16.2.4 presented by Mirek Cymer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 1 Optical Flow in Motion Analysis (16.2.4) presented by Mirek Cymer

    2. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 2 Overview Optical Flow Focus of Expansion (FOE) Mutual Velocity Collision Prediction

    3. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 3 Optical Flow in Motion Analysis What is Optical Flow? The apparent movement between two images Can be a map of where each pixel in the first image can be found in the second The term "optic flow" refers to a visual phenomena that you experience every day. Essentially, optic flow is the apparent visual motion that you experience as you move through the world. Suppose you sitting in a car or a train, and are looking out the window. You see trees, the ground, buildings, etc., appear to move backwards. This motion is optic flow. This motion can also tell you how close you are to the different objects you see. Distant objects like clouds, and mountains move so slowly they appear still. The objects that are closer, such as buildings and trees, appear to move backwards, with the closer objects moving faster than the distant objects. Very close objects, such as grass or small signs by the road, move so fast they whiz right by you.The term "optic flow" refers to a visual phenomena that you experience every day. Essentially, optic flow is the apparent visual motion that you experience as you move through the world. Suppose you sitting in a car or a train, and are looking out the window. You see trees, the ground, buildings, etc., appear to move backwards. This motion is optic flow. This motion can also tell you how close you are to the different objects you see. Distant objects like clouds, and mountains move so slowly they appear still. The objects that are closer, such as buildings and trees, appear to move backwards, with the closer objects moving faster than the distant objects. Very close objects, such as grass or small signs by the road, move so fast they whiz right by you.

    4. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 4 Optical Flow in Motion Analysis (cont.) The optical flow vector of a moving object in a video sequence:

    5. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 5 Optical Flow in Motion Analysis (cont.) The lion is sipping at the water. Only the head is moving but camera angle was shifted:

    6. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 6 Optical Flow in Motion Analysis (cont.) The bird's head moved from being tucked under its wing:

    7. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 7 Optical Flow in Motion Analysis Optical flow Gives a description of motion and can be a valuable contribution to image interpretation even if no quantitative parameters are obtained from motion analysis. Can be used to study a large variety of motions between an observer (an eye or camera) and objects in a scene: Moving observer and static objects Static observer and moving objects Both observer and objects moving Elements of motion Translation at constant distance from the observer Translation in depth relative to the observer Rotation at constant distance about the view axis Rotation of a planar object perpendicular to the view axis

    8. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 8 Optical Flow in Motion Analysis Mathematical relationships exist between the magnitude of optic flow and where the object is in relation to you. Factors and their effect on optical flow magnitude: Speed: Higher speed results in larger optical flow Distance: Closer object gives larger optical flow Angle: Larger if to your side 90 degrees or directly above or below Less if brought forward or backward Zero if directly in front of you (straight line) There are clear mathematical relationships between the magnitude of the optic flow and where the object is in relation to you. If you double the speed which you travel, the optic flow you see will also double. If an object is brought twice as close to you, the optic flow will again double. Also the optic flow will vary depending on the angle between your direction of travel and the direction of the object you are looking at. Suppose you are travelling forward. The optic flow is the fastest when the object is to your side by 90 degrees, or directly above or below you. If the object is brought closer to the forward or backward direction, the optic flow will be less. An object directly in front of you will have no optic flow, and appear to stand still. (However, because the edges of that forward object are not directly ahead of you, these edges will appear to move, and the object will appear to get larger. There are clear mathematical relationships between the magnitude of the optic flow and where the object is in relation to you. If you double the speed which you travel, the optic flow you see will also double. If an object is brought twice as close to you, the optic flow will again double. Also the optic flow will vary depending on the angle between your direction of travel and the direction of the object you are looking at. Suppose you are travelling forward. The optic flow is the fastest when the object is to your side by 90 degrees, or directly above or below you. If the object is brought closer to the forward or backward direction, the optic flow will be less. An object directly in front of you will have no optic flow, and appear to stand still. (However, because the edges of that forward object are not directly ahead of you, these edges will appear to move, and the object will appear to get larger.

    9. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 9 Optical Flow in Motion Analysis (cont.) Motion form recognition is based on the following facts: Translation at constant distance is represented as a set of parallel motion vectors Translation in depth forms a set of vectors having a common focus of expansion (FOE) Rotation at constant distance results in a set of concentric motion vectors Rotation perpendicular to the view axis forms one or more sets of vectors starting from straight line segments

    10. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 10 Optical Flow in Motion Analysis (cont.) Moving forward down a forest path. Note the expanding optical lines showing a zooming motion:

    11. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 11 Optical Flow in Motion Analysis (cont.) Factors affecting Focus of expansion (FOE): Translation not at constant depth: Optical flow vectors are not parallel and their directions have a single FOE Translation at constant depth: FOE is at infinity Several independently moving objects: Each motion has its own FOE (shown in figure below)

    12. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 12 Optical Flow in Motion Analysis(cont.) Optical flow as seen from an aircraft (the observer): Blue arrows show the optical flow Blue circle directly at the center shows the FOE, which tells the aircraft the specific direction it is flying The figure above shows what the optic flow might look like from an aircraft flying over a rocky desert (or over Mars!). The blue arrows show the optic flow that would be seen by a camera or a passenger on the aircraft. Looking downward, there is a strong optic flow pattern due to the ground and rocks on the ground. The optic flow is fastest directly below the aircraft. It is especially fast where the tall rock protrudes from the ground. A sensor on the aircraft that responds to optic flow would be able to see this optic flow pattern and recognize the presence of the tall rock. The meaning is clear: "Look out below!!! Looking forward, there is another optic flow pattern due to the upcoming rock and anything else the aircraft might be approaching. The blue circle directly at the center shows the "focus of expansion" or FOE. The FOE tells the aircraft the specific direction it is flying. (Remember above we said that if you are travelling in a straight line, the optic flow is zero in the directly forward direction.) The aircraft sees a large optic flow to the right of the FOE, which is due to the large rock on the left-hand side of this picture. The aircraft also sees smaller optic flow patterns in the downward-front direction, due to the ground. Towards it's upper left, it sees no optic flow because this region of the visual field only has the sky. The forward optic flow pattern reveals that the aircraft will fly close by the big rock, perhaps dangerously close. If the optic flow on the aircraft's right grows larger, then the aircraft should take that as a hint to turn away. The figure above shows what the optic flow might look like from an aircraft flying over a rocky desert (or over Mars!). The blue arrows show the optic flow that would be seen by a camera or a passenger on the aircraft. Looking downward, there is a strong optic flow pattern due to the ground and rocks on the ground. The optic flow is fastest directly below the aircraft. It is especially fast where the tall rock protrudes from the ground. A sensor on the aircraft that responds to optic flow would be able to see this optic flow pattern and recognize the presence of the tall rock. The meaning is clear: "Look out below!!! Looking forward, there is another optic flow pattern due to the upcoming rock and anything else the aircraft might be approaching. The blue circle directly at the center shows the "focus of expansion" or FOE. The FOE tells the aircraft the specific direction it is flying. (Remember above we said that if you are travelling in a straight line, the optic flow is zero in the directly forward direction.) The aircraft sees a large optic flow to the right of the FOE, which is due to the large rock on the left-hand side of this picture. The aircraft also sees smaller optic flow patterns in the downward-front direction, due to the ground. Towards it's upper left, it sees no optic flow because this region of the visual field only has the sky. The forward optic flow pattern reveals that the aircraft will fly close by the big rock, perhaps dangerously close. If the optic flow on the aircraft's right grows larger, then the aircraft should take that as a hint to turn away.

    13. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 13 Mutual Velocity The mutual velocity c of an observer represented by an image point can be found in an optical flow representation. Mutual velocities in directions x, y, and z: cx = u cy = v cz = w (z gives information about depth) Image coordinates: x, y

    14. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 14 Mutual Velocity (cont.) Position at time t0 = 0: (x0, y0, z0) Position of the same point at time t can, assuming unit focal distance of the optical system and constant velocity, be determined as follows: *Note: We will be referring to this equation many times.

    15. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 15 FEO Determination FEO can also be determined from Assume motion is directed towards an observer so as t ? -8, the motion can be traced back to the originating point at infinite distance from the observer. The motion towards an observer continues along straight lines and the originating point in the image plane is: *Note: The above equation can also be used for t ? 8 and motion in the opposite direction.

    16. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 16 Distance (depth) Determination Because of the presence of a z-coordinate in: , we can use this equation to determine the current distance of a moving object from the observers position. However, in order to evaluate the distance exactly, at least one actual distance value must be known.

    17. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 17 Distance (depth) Determination (cont.) In order to calculate the distance of a point from the FOE in relation to its velocity, we use the following formula: This formula is the basis for determining distances between moving objects. The ratio of z / w specifies the time at which an object moving at a constant velocity w crosses the image plane.

    18. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 18 Distance (depth) Determination (cont.) Now by knowing the distance of any single point in the image that moves with velocity w along the z axis, we can calculate the distance from any other point in the image that is moving with the same velocity w by using the following formula: z1(t) is the known distance and z2(t) is the unknown distance.

    19. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 19 Distance (depth) Determination (cont.) Using the formulas covered up to this point, relations between real-world coordinates x, y, and image coordinates x, y can be found related to the observer position and velocity with the following formulas: *Note: Above formulas cover both moving objects and moving camera as long as the motion is long the camera optical axis.

    20. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 20 Collision Prediction Collision prediction is analysis of motion of a real world object and being able to detect its potential collisions with scene objects. Observer motion aims into the FOE of this motion. FOE coordinates: (u / w, v / w) The origin of image coordinates proceeds in the direction: s = (u / w, v / w, 1) and follows a path in real-world coordinates at each time instant defined as a straight line with the following formula:

    21. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 21 Collision Prediction (cont.) Position of an observer xobs when at its closest point of approach to some x in the real world is:

    22. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 22 Collision Prediction (cont.) The smallest distance dmin between a point x and an observer during observer motion is: Example: A circular-shaped observer with radius r will collide with objects if their smallest distance of approach dmin < r.

    23. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 23 Collision Prediction (cont.) Example of real-world practical use: Preventing rear-end vehicle collisions Drivers sensory analysis of truck ahead: The optical transformation that the visual image undergoes as the driver travels forward is the primary sensory information that drivers use to judge whether a rear-end collision is imminent. When a driver views an object such as a truck (Figure 1), it creates an image on the eye's "film," a light sensitive layer called the "retina." As the driver approaches the truck, the retinal image expands, and the edges move outward. Figure 1 shows an object's image at time T and then the same image a moment later (time T+1) as the driver nears. On the retina, the truck's edges have moved outward, creating a motion cue called "looming." The faster the closing rate, the faster the expansion, the faster the edge motion and the greater the looming. The optical transformation that the visual image undergoes as the driver travels forward is the primary sensory information that drivers use to judge whether a rear-end collision is imminent. When a driver views an object such as a truck (Figure 1), it creates an image on the eye's "film," a light sensitive layer called the "retina." As the driver approaches the truck, the retinal image expands, and the edges move outward. Figure 1 shows an object's image at time T and then the same image a moment later (time T+1) as the driver nears. On the retina, the truck's edges have moved outward, creating a motion cue called "looming." The faster the closing rate, the faster the expansion, the faster the edge motion and the greater the looming.

    24. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 24 Collision Prediction (cont.)

    25. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 25 Collision Prediction (cont.) Calculating TTC from optical and spatial variables. The graph shows the effects of closing velocity and distance. The dashed line is the mean looming threshold found in research studies. The yellow area is the best estimate for real driver looming thresholds.

    26. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 26 Optical Flow in Motion Analysis In summary, the analysis of motion, computation of FOE, depth, possible collisions, time to collision, etc., are all very practical problems.

    27. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 27 The UMB Texture Segmentation Challenge On the following slides you will see the results of each of your texture segmentation programs. I chose 5 evaluation inputs containing 3, 4, 5, 6, and 8 segments. Please vote for the program that you find to have generated the most accurate results. Give 1, 2, and 3 points to your top-ranked programs.

    28. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 28 Program #1

    29. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 29 Program #2

    30. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 30 Program #3

    31. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 31 Program #4

    32. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 32 Program #5

    33. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 33 Program #6

    34. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 34 Program #7

    35. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 35 Program #8

    36. May 5, 2011 Computer Vision Lecture 23: Optical Flow/Texture Challenge 36 The UMB Texture Segmentation Challenge and the winner is: Program #5 by Yang! Congratulations! and the second place is shared by Programs #1 (Jacky) and #3 (Roman)! Congratulations as well!

More Related