slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Peter van Lith PowerPoint Presentation
Download Presentation
Peter van Lith

Loading in 2 Seconds...

play fullscreen
1 / 95

Peter van Lith - PowerPoint PPT Presentation


  • 277 Views
  • Uploaded on

Peter van Lith Embedded Intelligence Embedded Intelligence The Technology What are Embedded Processors and Why Processor Families Application Areas Facilities I/O Systems Trends Embedding Intelligence Typical problems Applications RoboCup, Aibo and Camera’s Overview

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Peter van Lith' - johana


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Peter van Lith

Embedded Intelligence

embedded intelligence
Embedded Intelligence
  • The Technology
    • What are Embedded Processors and Why
    • Processor Families
    • Application Areas
    • Facilities
    • I/O Systems
    • Trends
  • Embedding Intelligence
    • Typical problems
    • Applications
    • RoboCup, Aibo and Camera’s
overview
Overview

What is different about embedded processors

  • What are Embedded Processors
  • Processor Families
  • Application Areas
  • Facilities
  • I/O Systems
  • Trends
introduction
Introduction
  • What is it
    • Small computers
    • Controllers
  • Why Embedded processors
    • Size, weight, Power consumption, Simplicity / Reliability
    • Cost
    • Real-time
  • How to embed intelligence
    • Usually with PC / Notebook
      • Consumes much energy / space
      • Also use of PLC’s
    • Distributed model functions more flexibly
    • Use of hardware agents
embedded intelligence5
Embedded Intelligence

What it is NOT

  • The kind of intelligence in
    • Intelligent Terminals vs
    • Dumb Terminals
  • Applied AI in smaller processors
applications
Applications

Number of processors considerably larger than PC’s, medium and mainframe computers together

  • Stand Alone
    • PDA, Games, Remote Control
  • Peripheral Controllers
    • Keyboards, TV, Video, Radio, Machines
  • Complex
    • Cars, Process Control
  • Networked
    • Telephone, Web Interface, Printers, Coffee machines
    • Ad-Hoc - Sensor Nets, Swarms, Dust
  • Micro
    • Smart Cards, Smart Tags
  • Autonomous
    • AGV’s, Robots, Aibo, RoboCup, Rescue, RoboSail
embedded families
Embedded Families
  • PC Based
    • Interpreters
    • C/C++
  • OS Based
    • Symbian, WinCE, WinXP Embedded, Linux
    • DragonBall, Arm, StrongArm
    • PC 104
    • J2ME – cldc, cdc
  • Stand Alone
    • 8051, HC11, ST6, H8, Pic
  • Interpreters
    • Basic Stamp
    • Java Stamp, TStik (Tini)
  • JCX
    • Lego Brick (H8)
    • JCX Interfaces
  • Java processors
    • JStik, JStamp
applications8
Applications
  • Historical growth
    • Mainframes - N : 1
    • PCs - 1 : 1
    • Embedded - 1 : N
  • The software problem
    • Hardware used to be reliable
      • Simple
      • Clear interfaces
    • Growing software unreliability
    • Testing of interfaces
facilities
Facilities
  • Protection
  • Power Management
  • Memory
  • Architecture
  • Bus Architectures / Interfacing
  • Level converters
  • In Circuit
  • Speed
facilities10
Facilities

Protection

  • Power Management,
  • Brownout
  • Watchdogs
  • Code Protection

Power Management

  • Speed control
  • Sleep mode
  • Function switchoff

Memory

  • EEProm, Flash
  • On Chip RAM, Off Chip RAM/Flash
  • Bank switching
  • Memory Mapped I/O
facilities11
Facilities

Architecture

  • RISC, CISC, Harvard, von Neumann
  • Small Footprint
  • Multiply/Divide
  • Array Functions
  • DSP Functions
  • Level Comparator
  • AD/DA converters

Bus Architectures / Interfacing

  • Chip Select
  • Data and Address Bus
  • High Speed and Low Speed busses
  • Peripheral interfaces
facilities12
Facilities

Level converters

  • TTL 5v
  • CMOS 3.3 v, 1.3 v
  • Differences in logic levels

In Circuit

  • Programming (ICP)
  • Debugging (ICD)
  • Emulation (ICE)

Speed

  • Crystal divide
  • Phase Lock Loop
  • Idle Loop
  • Clock speed variation
  • Overclocking
interfaces
Interfaces

Inter Procesor

  • SPI, I2C, One Wire, Parallel, Serial

Testing

  • Boundary Scan, JTAG, ICD, ICE

Inter-System

  • Serial, TCP/IP, USB
  • Wireless
    • RF, IrDa, WiFi, BlueTooth, ZigBee
    • Sensor Networks

Inter-Chip

  • SPI, I2C, On-Chip networking
i o systems
I/O Systems
  • I/O types
  • Switches
  • Sensors
  • Displays
  • Sound
  • Actuators
i o systems inputs
I/O Systems - Inputs

Switches

  • Keyboards
  • Touch Screens
  • Contact switches
  • Magnetic switches
i o systems sensors
I/O Systems - Sensors
  • Optical Encoders
  • Magnetic Sensors
    • Hall Effect, Magneto Resistive
  • Optical Sensors
    • IR, PIR, LDR
  • Image Sensors
    • Arrays, Camera chips
  • Sound Sensors
    • Microphones, Piezo sensors
    • Ultrasonic sensors
  • Gas Sensors
    • CO, CO2
  • Other
    • Temp, Wind, Humidity, Strain gauges
i o systems output
I/O Systems - Output

Displays

  • LEDs
  • 7 Segment Displays
  • LCD Displays
  • COG Displays

Sound

  • Beepers
  • Speaker Systems
  • Speech chips
  • Speech recognition chips
i o systems actuators
I/O Systems - Actuators

Motors

  • Motor Controllers
  • Stepper Motors
  • DC Motors

Power Drivers

Valves, Pneumatic, Hydrolic

Relays, Solenoids

power management
Power Management

Power systems

    • Mains, Battery, Solar
  • Battery Systems
    • Lead Acid, NiCad, NiMh, LiOn, Polymer, Fuel Cell
  • Regulators
    • Step Down, Step Up, Switching
trends
Trends
  • Embedded Logic
    • Asics
    • CPLD’s
    • Hybrids
  • Configurable Logic
    • FPGA’s
    • VHDL, JHDL
  • On-Chip networks
new hardware logic
New Hardware Logic

ASICs – Application Specific Integrated Circuits

    • Development of IC is very expensive (> 1 mi$)
    • Takes a long development cycle
    • Development done with VHDL (Very hi speed ic Hardware Definition Language)
  • GAL – Gate Array Logic
  • CPLD – Custom Programmable Logic Devices
  • FPGA – Field Programmable Gate Arrays
  • RAW - Random Access Wiring project at MIT
slide22
FPGA
  • FPGA consists of Gate Arrays, which has:
  • DLL - Delayed Locked Loop for clock timing
  • RAM – Block RAM
  • IOB – IO Blocks
  • CLB – Configurable Logic Block
slide23
CLB
  • Configurable Logic Block uses:
  • LUT – Lookup Table to define functions
  • MUX - Multiplexers to connect input pins to Gate Arrays
logic block
Logic Block
  • Logic is defined just as with discrete hardware components
  • Process the same as creation of PCB and IC
  • Many tested IP components available
  • Compiled into VHDL to generate layout
  • JHDL version to define logic in Java
  • Compilers to transform C into VHDL
embedded intelligence25
Embedded Intelligence

Conventional

  • State Machines
    • Heartbeat
  • PID controllers
  • Sensors and actuators

Advanced

  • Fuzzy Logic
  • Subsumption
  • Sensor fusion
  • Organic growth
  • Neural Nets

High level programming

  • Patterns
  • Virtual Machines
subsumption
Subsumption
  • Development of complex systems often requires several generations
  • Each generation requires a redesign
  • Restructuring causes incompatibility

Rodney Brooks (MIT) developed an architecture that allows evolutionary development

  • Once a components works it is not changed any more
  • New functionality is added on top of existing structure
subsumption architecture

Avoid

Walk

Leg Up

Leg Down

Vertical

Position

Leg Forward

Vertical

Balance

Horizontal

Position

Subsumption architecture
emergent properties
Emergent properties

A new combination of existing properties that prove useful for something unexpected

  • School of fish / Swarms
  • Cockroaches
  • Birds / feathers

Something for nothing

Side effects that have a life or their own

embedded intelligence29
Embedded Intelligence

Control Issues

  • Localization, odometry
  • Grounding
  • Path planning
  • Goals and beliefs
  • Conflicts
  • Noise

Desirable interfaces

  • Camera Systems
    • Image Analysis
  • Sensor processing
    • Sampling
    • Sensor fusion
    • Sensor networks
understanding

Understanding

Understanding

Braitenberg’s Vehicles

Most people give an explanation of the behavior that is more complex than the internal structure deserves

Angry

Afraid

Admiration

Curious

the need for intelligence
The need for intelligence
  • A robot needs to make its own decisions
  • It needs to recognize and understand its environment
  • It needs to be able to communicate
    • With people
    • With other robots
  • It needs to move around in its environment
  • It needs to adapt to its environment
current possibilities
Current possibilities
  • What can we do today
    • Only limited applications
    • Even simple interaction is problematic
  • Embodiment is important
    • Honda Robot
    • Sony Aibo and Pino
    • NEC PaPeRo
    • IS Robotics My Real Baby
examples of applications
Examples of applications
  • Asimo, primarily for demonstrations
    • Walks statically
    • Requires prepared environment
    • 140 cm
  • Sony bipeds QRIO
    • Static and pseudo-dynamic walk
    • Many sensors
    • 60 cm
embedded intelligence35
Embedded Intelligence
  • The Technology
    • What are Embedded Processors and Why
    • Processor Families
    • Application Areas
    • Facilities
    • I/O Systems
    • Trends
  • Embedding Intelligence
    • Typical problems
    • Applications
    • RoboCup, Aibo and Camera’s
overview36
Overview

How to use in intelligent systems

  • Typical problems
  • Robot types
    • Autonomous Robots
    • Interactive Robots
    • Cooperating Robots
    • Walking Robots
    • Humanoids
  • The Projects
    • Aibo and RoboCup
    • Intelligent Camera
autonomous robots
Autonomous Robots

Two approaches:

  • Top Down (1960 – 1986)
    • Using reasoning
    • Goal directed behavior
    • Model building and planning
  • Bottom Up (1986 – now)
    • Using reactive behaviors
    • No models
the need for intelligence38
The need for intelligence
  • A robot needs to make its own decisions
  • It needs to recognize and understand its environment
  • It needs to be able to communicate
    • With people
    • With other robots
  • It needs to move around in its environment
  • It needs to adapt to its environment
the first robots
The first robots

Shakey (1966)

  • Received commands like SHRDLU
  • Needed to reason about every command
  • Used large external computer
  • Took a long time to execute every command
  • Used Reasoning and Planning
embedded intelligence typical problems
Embedded Intelligence - Typical problems

Problem areas are too simple

  • Optimal configuration solves the problem
    • ScareCrow
    • Sensorman
    • Deep Blue
    • PID controllers

AI component is always very small

  • Infrastructure problems are major
  • Construction and sensor processing
  • Fixed behaviour
  • Very difficult to integrate
the problem must be demanding
The problem must be demanding

ScareCrow (David Miller JPL)

  • Won the 1992 AAAI robot competition
  • Had only relays and some contact switches
  • Won by speed and brute force
the sensorman
The Sensorman
  • Infra Red sensors
  • Video camera
  • Stereo video camera
  • Stereo video color camera
  • Infra Red sensors

So: The simpler the sensor, the better

sense reason act

Sense

Reason

Act

Sense – Reason - Act
  • Sense
    • Get input from the world
  • Reason
    • World modelling
    • Reasoning, Decisions
  • Act
    • Output to the world
    • Interaction with the world
behavior
Behavior
  • ProActive Behavior
    • Planning
    • Goals
    • Beliefs
  • ReActive Behavior
    • Curiosity / Fright-or-Flight
    • Obstacle avoidance
acting
Acting
  • Strategies
    • Reasoning
    • Neural Nets
    • Genetic Algorithms
    • Decision Trees, Augmented State Machines
    • Negotiation
big issues
Big Issues
  • The Frame Problem
    • Get the object from the

room

  • Noticing
  • Remembering
controversy
Controversy
  • Analog vs Digital
    • The travelling salesman
    • Hardware networks
  • Physical vs Virtual
    • Stored program computer is a virtual network
  • Parallel vs Serial
    • Massively Parallel Machines
    • Still slower than network
robot insects
Robot insects
  • Construction of artificial organisms in hardware
  • Evolutionary development. Layered architecture. No redesigns.
  • Minimal control. No central processing and as much as possible reactive systems
  • Use emergent principles where possible
robot insects49
Robot Insects

Crawling is an emergent property

the need for intelligence50
The need for intelligence
  • A robot needs to make its own decisions
  • It needs to recognize and understand its environment
  • It needs to be able to communicate
    • With people
    • With other robots
  • It needs to move around in its environment
  • It needs to adapt to its environment
interactive robots
Interactive Robots
  • Interact with people
    • React to observer
    • Safe actions
  • Use and adaptive system to repeat in a surprising way
zzappo
Zzappo
  • Follow with his eyes
  • Lip synchronous speech
  • ST6 processor
  • Interactive attraction
  • Simple speech recognition
an interactive robot
An interactive robot

Zzappo!

  • Rat
  • the Box
  • Reacts to people
  • Recognizes words
  • Attract attention
  • Designed by door Mark de Jonge
flexible robot arm
Flexible robot arm
  • Usage of mcCibben air muscles
  • Very low stiffness
  • Not dangerous for people
  • Developed by

Leg Lab of TUD

zzappo55
Zzappo
  • Simple speech recognition
  • ELIZA type of interaction
  • Use of a people-safe arm
  • Lip Synchronous mouth movement
the need for intelligence56
The need for intelligence
  • A robot needs to make its own decisions
  • It needs to recognize and understand its environment
  • It needs to be able to communicate
    • With people
    • With other robots
  • It needs to move around in its environment
  • It needs to adapt to its environment
autonomous robots57
Autonomous Robots
  • RoboSail
    • Syllogic – a robotic sailboat for single handed transatlantic racing
  • Roomba
    • Autonomous vacuum cleaner built by MIT
robosail
RoboSail
  • React to boat sensor information
    • Many sensors
    • Conflicting and unreliable
  • Learn from the sailor
  • Allow limited autonomous behavior
    • Only helm control is allowed
    • Sailor prefers only use at night
  • Advice on current performance
robosail59
RoboSail
  • Intelligent Rudder Control Unit
  • Start in 1999
  • Electronic Compass
  • Embedded Web Server
  • CAN Interface
de taakverdeling

Skipper

Navigator

Watchman

Helmsman

De taakverdeling
  • Skipper
    • Processes weather maps etc.
  • Navigator
    • Watches sails and routing
  • Watchman
    • Optimizes boat behavior
  • Helmsman
    • Steers the ship
intelligent rudder control unit
Intelligent Rudder Control Unit
  • Developed in cooperation with NikHef
  • Max load 250 Amp
electronic compass
Wat kind of sensors

Various manufactureres

Different approaches

Earth’s magnetic field - KNMI

Processing data

Reading sensors

Calculating positiin in 3 dimensions

Calibration – TU Delft Physics

Motion Sensors

Various approaches

Electronic Gimballing

Examples by Philips and Honeywell

Used for:

RoboSail

RoboCup

Dionisys

Electronic Compass
web server
Connection with other equipment

TCP/IP and UTP cabling

Used for:

RoboSail

People counting system Efteling

Web Server
the need for intelligence66
The need for intelligence
  • A robot needs to make its own decisions
  • It needs to recognize and understand its environment
  • It needs to be able to communicate
    • With people
    • With other robots
  • It needs to move around in its environment
  • It needs to adapt to its environment
walking robots
Walking Robots
  • Wheels are too easy
  • Wheels are not natural
  • Wheels do not work well in rough terrain

So: Robots must have legs

dappie
Dappie
  • 4 x HC11 processors
  • Autonomous
  • Reactrs to sound, movement and obstacles
  • Detects table edge

1

2

3

4

dappie a robot dog
Dappie, a robot dog
  • Autonomous robot
  • Walks on 4 legs
  • 4 computers
  • 10 servo motors
  • 13 sensors
  • Sees, hears, feels and gets hungry
  • Beeps, winks and reacts with head
architecture walking
Architecture walking

System

Direction

Left

Right

Side

Speed

Sequencer

Leg

Stride

Step

Motor

Knee

Leg

dappie s senses
Dappie’s Senses
  • Active IR eyes, measure distance, direction and speed
  • Ears, recognize command and direction
  • Edge detect, recognizes table edge
  • Remote Control
  • Battery level
  • Stall sensors measure motor current
communication
Communication
  • RS232 with PC to program
  • SPI for internal communications
  • Exchange messages between processors
  • Continuos mutual control
  • Messages may get lost
architecture communication
Architecture communication

Eyes

Ears

Head

Sensors

Speech

IPC

Body

Sensors

Motors

RC/Batt

Stall

Motors

repertoire dappie
Repertoire Dappie
  • Stand up / lay down
  • Forward, backward, corners
  • 7 Speeds
  • Sees obstacles and avoids them
  • Follows movement with head
  • Listens and answers with beep
  • Reacts to commands
  • Follows sound
the need for intelligence75
The need for intelligence
  • A robot needs to make its own decisions
  • It needs to recognize and understand its environment
  • It needs to be able to communicate
    • With people
    • With other robots
  • It needs to move around in its environment
  • It needs to adapt to its environment
humanoids
Humanoids
  • Walking robots that look and act like people
  • Behave adaptably
  • Are more easy to relate to
  • Interface better to our environment
  • Perform useful tasks
a walking robot
A walking robot
  • HTS Alkmaar / TU Eindhoven
  • HC11 processor
  • Contact Richard van der Linde (TU Delft)
  • TUD Biped project
walking robots79
Walking robots

Walking on 6 legs requires:

  • Relatively little balancing
  • A lot of coordination
  • Walking on 2 legs requires:
  • Much more balancing
  • Much less coordination

Mark Raibert therefore started with a single legged robot

walking robots80
Walking robots

First on a stick

Then stand alone

walking robots81
Walking robots

Then robots on 2 and 4 legs, that could:

  • Run (± 20 km/u)
  • Jump
  • Make a Flip

Further research into:

  • Crawling
  • Climbing
  • Rough terrain
max walking robot
Autonomous walking robot, TU Delft

Embedded Java Processor

Two Pic-based UPB processors

McGibben pneumatic muscles

STW project

Martijn Wisse

Participation with Efteling

MAX Walking Robot
the need for intelligence86
The need for intelligence
  • A robot needs to make its own decisions
  • It needs to recognize and understand its environment
  • It needs to be able to communicate
    • With people
    • With other robots
  • It needs to move around in its environment
  • It needs to adapt to its environment
the projects
The projects
  • Aibo
    • Autonomous robot
    • 4 legged RoboCup league
  • Intelligent Camera
    • Embedded vision project
    • Java based programmable camera
sony s robots
Sony’s robots
  • First experiments with Robocup
  • Aibo first generation
    • Several production problems
    • Limited production
  • Aibo second generation
    • Much more responsive
    • Other processor family
    • Unlimited production
    • Delivery problems
  • AI components
    • Only associative learning
    • Good speech recognition
    • Limited image recognition
aibo ers 7
Aibo ERS-7
  • Aibo ERS-7 third generation
    • Faster
    • 6 x faster and more memory
    • Better camera
  • Camera processing is very hard
problems
Problems
  • Kits are too limited or too difficult
    • The Lego effect
    • Practicum is too short to gain sufficient experience
  • Development of a kit with:
    • Many possibility
    • Easy to program
    • Access to all system functions
flexible robot components
Flexible Robot Components
  • Small, simple stackable components
  • Highly interconnected
  • Hardware agents
  • Programmable
    • Via simulator
    • Via generated code
    • Via written code in Java or Assembler
intelligent systems
Intelligent Systems
  • PIC based processor board
    • Servo controllers / Motor driver
    • Web Server
    • Compass / CAN controller
    • Display and IR receiver
  • Embedded Java
    • aJile hardware Java implementation
    • muVium PIC Java compiler
  • Java based Camera
upbtest
UPBtest
  • Testing environment for robot control applications
  • Shows interactive interfaces
  • Online testing and debugging
  • Code generation

Components:

  • Motion Sensor
  • Position Sensor
  • Camera Unit
  • Actuators
the need for intelligence94
The need for intelligence
  • A robot needs to make its own decisions
  • It needs to recognize and understand its environment
  • It needs to be able to communicate
    • With people
    • With other robots
  • It needs to move around in its environment
  • It needs to adapt to its environment
summary
Summary
  • 90% of the development effort goes into building and testing the infrastructure
  • Embedded intelligence is important but still is only a small portion of the total
  • A lot of work is still to be performed
  • Brute Force is still the most succesfull AI strategy
  • Some essential ingredients are still missing