control of uav s l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Control of UAV’s PowerPoint Presentation
Download Presentation
Control of UAV’s

Loading in 2 Seconds...

play fullscreen
1 / 78

Control of UAV’s - PowerPoint PPT Presentation


  • 525 Views
  • Uploaded on

Control of UAV’s Raja Sengupta (sengupta@ce.berkeley.edu) Assistant Professor Civil and Environmental Engineering: Systems UC Berkeley Joint Work with the C3UV team Challenge of Flying Low Helicopter pilots fly low FAA requires see and avoid Find the freeway and follow it

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Control of UAV’s' - emily


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
control of uav s

Control of UAV’s

Raja Sengupta (sengupta@ce.berkeley.edu)

Assistant Professor

Civil and Environmental Engineering: Systems

UC Berkeley

Joint Work with the C3UV team

challenge of flying low
Challenge of Flying Low
  • Helicopter pilots fly low
  • FAA requires see and avoid
  • Find the freeway and follow it
used sectionals to build a manhattan model at 300 feet approx
Used Sectionals to build a Manhattan model at 300 feet (approx.)
  • Simulation testing of Control
flying low strategy
Flying Low: Strategy
  • Helicopter pilots fly low
  • Find the freeway or waterway and follow it
  • Avoid few remaining obstacles
cal freeway detection on mlb video no control
Cal Freeway Detection on MLB Video(No Control)
  • Generic corridor
  • detection by one-
  • dimensional
  • learning
  • Roads
  • Aqueducts
  • Perimeters
  • Pipelines
  • Power Lines
slide7
Generalization: Vision Based Following of Locally Linear Structures(Closed Loop on the California Aqueduct, June 2005)
tailored to
Tailored to………………..
  • For most UAV applications (>50 m), the obstacles of concern will be large objects such as towers, buildings or large trees
  • For these cases, the problem of obstacle detection is different from that of ground vehicles in environments cluttered with many obstacles.

VS

flight demonstration
Flight Demonstration
  • Experiment flown on a Sig Rascal airframe with a Piccolo avionics package and vision processing on an onboard PC104.
  • An 8.5 foot diameter balloon was used as the obstacle (distance currently calculated using GPS).
flight demonstration12
Flight Demonstration

-100

-100

0

0

100

100

200

200

300

300

400

400

0

0

Balloon

Balloon

avoidance with GPS

avoidance with GPS

-50

-50

-100

-100

-150

-150

direction of flight

direction of flight

-200

-200

y position (m)

y position (m)

-250

-250

-300

-300

-350

-350

-400

-400

autonomous control

autonomous control

started

started

-450

-450

-500

-500

x position (m)

x position (m)

objective distribute the data objects shared by the team across the members of the team
Objective: Distribute the data objects shared by the team across the members of the team

UAV

UAV

Team

publisher

Operation decomposer

Team Level

Resource allocation/scheduling

Operation monitor

Dispatcher

Formation

Navigation

UCB Rathinam 2004

scalable information management
Scalable Information Management:

Voronoi tessellation

Data objects

Euclidean Space

UAV’s

  • Geographic Data Management Network

Sengupta AINS 2003

scalable information management16
Scalable Information Management:

desired data

request

delivery

agent

Metric Space

distributed implementation in action
Distributed Implementation in Action

Movie of our

Implementation

4 agents on

4 laptops over

a wireless LAN

results tracking the california aqueduct
Results Tracking the California Aqueduct
  • The average error of the position of the vehicle from the curve was 10 meters over a length of 700 meters of the canal.
results canal following
Results – Canal Following
  • The road detection algorithm runs at 5 Hz (takes < 200 ms) or faster on the PC104 (700 MHz, Intel Pentium III).
  • No visible error was found from video sequences of over 100 frames containing the canal
flying low
Flying Low
  • Helicopter pilots fly low
  • Find the freeway and follow it
  • FAA requires see and avoid
    • obstacles and aircraft
obstacle avoidance is a constraint not a mission approach safe set theoretic
Obstacle Avoidance is a Constraint: Not a MissionApproach: Safe Set-theoretic
  • Assume capabilities of the airplane
  • Compute an unsafe set
  • When in the safe set, execute the mission
  • On the boundary execute the obstacle avoidance control
  • Assume obstacles are
    • Sparse
    • Stationary
    • Rectangular
analytical solution
Analytical Solution

Pl

  • The analytical solution can be calculated in 5 ms

Cr_ns

Cr

Oa

Ocusp

y (m)

Ona

BRS

Cl

Pr

x (m)

cal uav target capabilities obstacle avoidance
Cal UAV: Target CapabilitiesObstacle Avoidance
  • Simulation testing of Control
    • Flight through Manhattan model (300+ feet)
related work
Related Work
  • Vision-based obstacle avoidance has been studied primarily in the context of mobile ground robots.
    • Lenser ’03, Ohya ’00, Lorigo ‘97,
  • Vision based navigation of UAVs
    • Saripalli ’02, Shakernia ’02, Furst ’98 – Landing with known markings
    • Sinopoli ’01, Doherty ‘00 – Visual landmark navigation (terrain avoidance) for helicopter
    • Ettinger ’02, Pipitone ’01, Kim ’03 – Pose estimation for aircraft
  • Obstacle/Collision Avoidance for UAVs
    • Mitchell ‘01 – Aircraft avoiding known aircraft
    • Sigurd ’03 – Aircraft with magnetic sensors
    • Sastry ‘03 – Helicopters avoiding known helicopters/obstacles
    • How ’02 – MILP for Obstacle Avoidance
  • Vision based obstacle avoidance
    • Barrows ’03 – Biomimetic reactive control
related research
Related Research
  • Ground robots
    • Fixed baseline stereo – JPL, many others
    • Monocular map construction – Lenser (CMU), Kim (Berkeley)
    • Cooperative stereo - CMU
  • Optical Flow
    • Helicopter ground following – Srinivasan/Chahl (Australia)
    • Corridor following - USC helicopter
    • Micro UAV obstacle avoidance – Centeye
  • UAV depth map construction
    • Lidar – CMU Helicopter Project, Sastry (Berkeley Helicopter Project).
    • Vision + high precision IMU – Bhanu (joint with Honeywell)
  • Stereo Vision
    • GT Helicopter
requires depth typically use stereo vision
Requires DepthTypically use Stereo Vision
  • Given the image coordinates of a feature in one image
    • if one can find the image coordinates of the feature in the other image (feature matching), and
    • if one knows the rotation and translation of the two image planes then one knows the world coordinates of the feature (Ego-motion Estimation)
problem with depth estimation by stereo vision
Problem with Depth Estimation by Stereo Vision

0

Z+

Z

Z-

z

Increased accuracy requires increased camera separation

accurate depth estimation is a problem
Accurate Depth Estimation is a Problem
  • Range error due to pixel errors is .
approach
Approach
  • UAVs flying at low altitudes must autonomously avoid obstacles
  • Strategy
    • Segment the image into sky and non-sky
      • Non-sky in the middle  OBSTACLE
    • Strategy 1
      • Aim at the sky
    • Strategy 2
      • If it looms faster than a threshold and is in the middle  AVOID

Else do NOTHING

segmentation at moffet airfield
Segmentation at Moffet Airfield
  • Results for multiple regions found (only largest regions shown, dark blue represents all small regions)
vision processing
Vision Processing
  • Classification: balloon/horizon correctly found in ~ 90% of images
  • Time results: ~2Hz (120ms SVM, 200-600 ms horizon)
obstacle avoidance next steps
Obstacle Avoidance: Next Steps
  • Loom: 4 pixels/second asuming a 70deg FOV camera with 320 pixels, Speed :20 m/s
  • turn radius 100 m, processing delay of 0.5 s, safe avoidance distance of 10 m, the minimum obstacle size is about 2.5 m
vehicle target path planning problem
Vehicle Target Path Planning Problem
  • Vehicles: V = {v1,v2,...vn}, Targets:{t1,t2...tm}, Angles of approach: {θ1, θ2... θm}
  • Assign a cycle Pi for each vehicle starting and ending at vertex vi such that Ui Pi=V. A cycle Pi is a ordered sequence of vertices {ti1,...tik} for the vehicle i to visit
  • Assign paths for each vehicle that satisfy the non-holonomy constraints for the given sequence.
  • The objective is to minimize ∑Cost(Pi)
slide40
Resource Allocation or Vehicle-Target Assignment
  • Given a collection of targets that need to be serviced and a collection of vehicles, how do you assign vehicles to targets ?
  • 1-1 Vehicle Target Assignment
  • Vehicle-Target Path Planning
    • More targets than vehicles
1 1 vehicle target assignment
1-1 Vehicle Target Assignment

Objective:

Constraints:

  • Solution to the relaxed linear programming formulation is a feasible solution for the assignment problem
  • Total unimodurality
vehicle target path planning problem42
Vehicle Target Path Planning Problem
  • Traveling salesman problem if number of vehicles = 1 and no kinematic constraints
  • Asymmetry, that is dij ≠ dji but satisfies triangular inequality
  • c-Approximation algorithm: cost of the solution is atmost c times the optimal value
  • Approximation algorithms for asymmetric TSP
    • Based on the number of points visited: 0.99log(n) - Markus Blaser (2002)
    • Based on the ratio of the distances dmax/dmin - Kumar and Li (2002)
  • Single vehicle problem with non-holonomy - Bullo et al, 2005
  • Multi vehicle problem with heuristics – Zhijun et al, 2005
  • Asymmetry with kinematic constraints can be bounded if the euclidean distance between the points are reasonably apart, d ≥ 2R
  • Sensor footprints are at least of the order of the minimum turning radius.
  • Basic idea:
    • Solve the problem assuming the distances are Euclidean
    • Using the sequence for each vehicle, assign paths that satisfy kinematic constraints
motion planning
Motion Planning
  • Find the minimal distance path joining ( x1,y1 ,1) and ( x2,y2,2) subject to kinematic constraints
algorithm for vehicle target path planning problem46
Algorithm for Vehicle Target Path Planning Problem

Find the Eulerian walk for each subtree and reduce it to a TSP tour for each vehicle

2 approximation algorithm because the cost of the multigraph is ≤ 2*Cost(MST)

slide49

Algorithm for Vehicle Target Path Planning Problem

  • Theorem: The approximation algorithm has a bound of = 2ddubins/deuclidean ≈ 6
  • Further the distance between the points, better the bound is
    • At best it could be ≈ 2
  • Angles of approach for the targets were given

A

B

conclusions
Conclusions
  • Addressed the problem of resource allocation in the context of unmanned aerial vehicles
  • Algorithms for multiple vehicles satisfying kinematic and fuel constraints.
  • Tighter bounds for the algorithms
  • Future work could address vehicles with fuel constraints, targets with precedence constraints etc.
with the advent of digital computing control linked to the synchronous model of computation
With the advent of Digital Computing Control linked to the Synchronous Model of Computation

Caspi P. Embedded Control:

From Asynchrony to Synchrony

and Back, EMSOFT 2001

extend control design tools to networked environments
Extend Control Design Tools to Networked Environments?

Develop code for swarm of UAVs for collaborative control.

The code is developed on the same high-level, easy to use tools used to develop the F14 control.

The code is automatically distributed across the different UAVs and it behaves as expected.

distributing the synchronous model
Distributing the Synchronous Model
  • Both order and timing would have to be enforced across networks

Cascade Composition

Berry ’91

y=f(u)

u=g(y)

Feedback Composition: Fixpoint Semantics

the logical order can be enforced without scheduling zennaro sengupta emsoft 05
The Logical Order can be enforced Without Scheduling! – Zennaro, Sengupta EMSOFT’05
  • Implementation problem: given a map  from RA to STS* traces we want an implementation map  such that, for all STS* s and RA r the following holds:

(r = (s)  rt)  s  (t)

  • Modularity preservation: we seek a composition operator xRA with respect to which  is a monomorphism between (STS, xSTS) and (RA, xRA). We want this operator to be implementable across a network;
what is currently available
What is currently available
  • Simulink / RealTime Workshop currently does NOT support distributed implementation;
  • It has been proved that some modular synchronous systems can be compiled while preserving modularity in a distributed environment;
    • Benvenieste, Caillaud, Le Guernic “Compositionality in dataflow synchronous languages: specification and distributed code generation”, Information and Computation, col. 163, Nov 2000, Academic press
    • Non finite system representation theoretical settings;
  • Tools for the automatic distribution of Simulink programs over TTA networks:
    • P. Caspi, A. Curic, A. Maignan, C. Sofronis, S. Tripakis and P. Niebert. From Simulink to SCADE/Lustre to TTA: a layered approach for distributed embedded applications, ACM-SIGPLAN (LCTES'03), 2003
    • Cannot be ported to non-TDMA networks;
what is currently available61
What is currently available
  • Tools for the automatic distribution of synchronous systems:
    • A. Girault, C. Menier, Automatic production of Globally Asynchronous Locally Synchronous Systems, EMSOFT 2002
    • ESTERELLE / Lustre
    • Modularity is not preserved;
  • Techniques for distributing synchronous systems:
    • J.Romberg, A. Bauer. Loose synchronization of event-triggered networks for distribution of synchronous programs, EMSOFT 2004
    • Focus on timing constraints;
    • Modularity is not preserved;
what is currently available our work
What is currently available (Our work)
  • Tools for the automatic distribution of synchronous systems:
    • M. Zennaro, R. Sengupta, Distributing Synchronous Systems with Modular Structure, CDC 2004
    • M. Zennaro, R. Sengupta, Distributing Synchronous Programs Using Bounded Queues, EMSOFT 2005 (to appear)
    • M. Zennaro, R. Sengupta, Distributing Synchronous Programs Using Bounded Queues, a coordinated traffic signal application, Research Report, UCB-ITS-RR-2005-4
      • Simulink (single rate, fixed-time discrete solver)
      • Modularity is preserved;
relevant papers
Relevant Papers
  • S. Rathinam, R. Sengupta, S. Darbha, A Resource Allocation Algorithm for Multi-vehicle Systems with Non-Holonomic Contraints, Submitted to the IEEE Transactions of Automation Science, June 2005.
  • S. Rathinam, Z. Kim, A. Soghaikan, R. Sengupta, Vision Based Following of Locally Linear Structures, Submitted to the 44th IEEE conference on Decision and Control 2005, Spain.
  • M. Zennaro, R. Sengupta. Distributing Synchronous Programs Using Bounded Queues. To appear in the Proceedings of EMSOFT 2005, Jersey City, USA.
  • McGee T., Sengupta R., Hedrick J.K. Obstacle Detection for Small Autonomous Aircraft Using Sky Segmentation. In proceedings of the International Conference on Robotics and Automation, April 2005, Barcelona, Spain.
  • Frew E., Sengupta R. Obstacle Avoidance with Sensor Uncertainty for Small Unmanned Aircraft. In proceedings 43rd IEEE Conference on Decision and Control, December 2004, Paradise Island, Bahamas.
  • Rathinam S., Sengupta R. UAV Navigation in an Unknown Environment, 43rd IEEE Conference on Decision and Control, December 2004, Paradise Island, Bahamas.
  • Zennaro M., Sengupta R. Distributing Synchronous Systems with Modular Structure. In Proc. of the 43rd IEEE Conference on Decision and Control, December 2004, Paradise Island, Bahamas.
  • Frew E., Spry S., Mcgee T., Xiao X., Sengupta R., Hedrick J. K. Flight Demonstrations of Self-Directed Collaborative Navigation of Small Unmanned Aircraft. To appear at AIAA 3rd Unmanned Unlimited Technical Conference, Workshop, & Exhibit, Chicago, IL, September 2004.
  • Rathinam S., Zennaro M., Mak T., Sengupta R. An Architecture for UAV Team Control. 5th IFAC Symposium on Intelligent Autonomous Vehicles, Lisbon, Portugal, July 2004
relevant papers65
Relevant Papers
  • Frew E., Kim Z., Howell A., McGee T., Rathinam S.,Xiao X, Zennaro M., Jackson S., Morimoto M., Hedrick J. K., Sengupta R.. Stereo-Vision-Based Control of a Small Autonomous Aircraft Following a Road. Second Annual Swarming Conference, Crystal City, MD, June 2004.
  • Frew E., McGee T., Kim, Z., Xiao X., Jackson S., Morimoto, M., Rathinam R., Zennaro M. , Padial J., Sengupta R. Vision Based Road-Following Using a Small Autonomous Aircraft. IEEE Aerospace Conference, Big Sky, Montana, March 2004.
  • Rathinam S., Sengupta R. A Safe Flight Algorithm for Unmanned Aerial Vehicles. IEEE Aerospace Conference, Big Sky, Montana, March 2004.
  • Mahajan A., Ko J., Mak T., Sengupta R. GDMN: An Information Management Network for Distributed Systems. 2nd IEEE Conference on Autonomous Intelligent Networked Systems, Menlo Park, CA, June 2003.
  • Lee J., Huang R., Vaughn A., Xiao X., Hedrick J.K., Zennaro M., Sengupta R. Strategies of Path-Planning for a UAV to Track a Ground Vehicle. 2nd IEEE Conference on Autonomous Intelligent Networked Systems, Menlo Park, CA, June 2003.
  • Mahajan A., Ko J., Sengupta R. Distributed Probabilistic Map Service. Proc. of the 41st IEEE Conference on Decision and Control, December 2002.
  • Ko J., Mahajan A., Sengupta R. A Network-Centric UAV Organization for Search and Pursuit Operations. Proc. of the 2002 IEEE Aerospace Conference, March 9-16, 2002.
  • Zennaro M., Ko J., Sengupta R., Tripakis S. A Service Network Architecture for a Multi-Vehicle Search Mission. Proc. of the 40th IEEE Conference on Decision and Control, December 4-7, 2001.
scalable information management target map and risk map
Scalable Information Management:Target Map and Risk Map

Risk Map

Example: Target Map

  • Target distribution map
    • P(A, N, t); probability of N targets of type t in area A
  • Target distribution update
    • Fuses measurements from different kinds of sensors (SAR and EO)
    • Bayesian update
  • Risk map computation
    • Integral of threat model with respect to the measure P(A, N, t)
    • Generates the value function for navigation

UCB Rathinam 2003

movie of implementation
Movie of Implementation

Total data is this map

  • 4 laptops over wireless
  • One publisher per laptop
  • Start with one publisher
  • Three others come up
  • Some die
  • Data redistributes as publishers join and leave
movie of implementation69
Movie of Implementation

Total data made of many

data objects

  • 4 laptops over wireless
  • One publisher per laptop
  • Start with one publisher
  • Three others come up
  • Some die
  • Data redistributes as publishers join and leave
movie of implementation70
Movie of Implementation

Voronoi tessellation

  • 4 laptops over wireless
  • One publisher per laptop
  • Start with one publisher
  • Three others come up
  • Some die
  • Data redistributes as publishers join and leave
movie of implementation71
Movie of Implementation
  • 4 laptops over wireless
  • One publisher per laptop
  • Start with one publisher
  • Three others come up
  • Some die
  • Data redistributes as publishers join and leave
movie of implementation72
Movie of Implementation
  • 4 laptops over wireless
  • One publisher per laptop
  • Start with one publisher
  • Three others come up
  • Some die
  • Data redistributes as publishers join and leave
data consistency in the publisher inconsistent copies are detected whp
Data Consistency in the Publisher:Inconsistent copies are detected whp

Wrong location copy 1

Data Location

Wrong location copy 2

geographic data management network survivable information for uav swarms
Geographic Data Management Network:Survivable Information for UAV Swarms
  • The server backbone dynamically tracks the client agent organization
  • Servers move in and out while the information survives
tracking the agent organization dynamic gdmn backbone control
Tracking the Agent Organization:Dynamic GDMN backbone Control
  • Design a distributed control algorithm for the servers to partition the data and the clients to minimize the total bit-meters (Kumar etal.) of work done in the system and balance the load on the servers.
  • Let the load generated in each client be bi. If the locations of the points are denoted by pi and the location of the servers are denoted by cj, then the total cost is:

 bi ( min dist(pi, cj) )

 i  j

  • The control algorithm updates server positions to reduce this cost
simulation
Simulation
  • This example involves 100 clients and 6 servers
control algorithm
Control algorithm
  • In each sampling interval, each server
    • Measures the positions and the traffic generated by its clients
      • GDML routing protocols make the client set the Voronoi cell
    • Calculates the weighted centroid of all the clients it serves
    • Moves towards its weighted centroid
  • Works well if the servers travel faster than the clients
  • The algorithm is based on the k-means algorithm (MacQueen ,1967 )