slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SLIDES SET NUMBER 1. PowerPoint Presentation
Download Presentation
SLIDES SET NUMBER 1.

Loading in 2 Seconds...

play fullscreen
1 / 110

SLIDES SET NUMBER 1. - PowerPoint PPT Presentation


  • 111 Views
  • Uploaded on

SLIDES SET NUMBER 1. Class 478/578: General 1. My name is Marek Perkowski You can call my Marek, or Dr. Perkowski or whatever you like. This class is fun, at least for me. I hope that you will have fun also. We build practical robots – embedded systems

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SLIDES SET NUMBER 1.' - joann


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
class 478 578 general 1
Class 478/578: General 1
  • My name is Marek Perkowski
  • You can call my Marek, or Dr. Perkowski or whatever you like.
  • This class is fun, at least for me.
  • I hope that you will have fun also.
  • We build practical robots – embedded systems
  • Class is graded based on practical achievements, a little bit similar to Capstone Project.
  • You can find all information on my webpage, find me through Google.
class 478 578 general 2
Class 478/578: General 2
  • If you are a graduate student your project is more difficult, otherwise the same.
  • Two homeworks and Project
  • No exam.
  • Student presentations (related to homeworks or projects)
  • I expect high quality of reports (many graduate students had publications based on these reports)
  • Robots connected to Internet (demo and explanation next Thursday).
class 478 578 grading
Class 478/578: grading
  • Homework 1 – 10 % (evolutionary algorithms and foraging)
  • Homework 2 – 10 % (any subset of your project)
  • Presentation – 10 %
  • Project – 70 %
  • Groups – 1 to 5 students, group leader.
  • In final report, each student has a separate part to demonstrate his/her work.
  • Each student presents a separate presentation of his work.
class 478 578 book
Class 478/578: book
  • Braunl.
    • You can find slides to this book on internet
    • Book was ordered early but it must be reprinted “on demand”.
    • If you have no book, do not worry. All is in my slides.
    • Somebody told me that PDF of all text is also on internet
  • Slides of my class on my webpage – look for “Embedded Robotics” on my main webpage.
  • To find my webpage do search on Google “Marek Perkowski”
class 478 578 your background
Class 478/578: yourbackground
  • Programming
    • Matlab
    • C
    • C++
    • Java
  • Some basic digital design and interfacing experience (only in some projects)
  • Some basic math, Boolean Algebra, probability.
  • Digital Signal Processing, Image Processing (for some projects, will be covered in debth in ECE 479 next quarter)
class 478 578 your background review
Class 478/578: yourbackground review
  • Boolean functions, gates and circuits
  • Finite State Machines
  • Probabilistic State Machines
  • Grammars
  • Linked Lists
  • Arduino
class 478 578 your background information
Class 478/578: yourbackground information

Please give me today the following information:

  • Your first name, last name and contact (email, phone)
  • Do you want to be on my Facebook – send me message on Facebook.
  • Programming classes you have taken.
  • Programming projects you have done.
  • Robot projects you have done. Please write more.
  • Any hardware projects you have done, like fixing a radio or a computer, building a FPGA controller etc.
  • Your background (hardware, software, art, physics, math, biology, etc)
  • Are you a graduate or undergraduate student.
  • For each of three areas: theory, programming and practical robot building, write percentages of your project’s grade (I am not sure I will be able to take this into account in every case)
  • Do you prefer to work alone or in a team for this class?
class 478 578 your background information1
Class 478/578: yourbackground information

Please give me today the following information:

  • Your first name, last name and contact (email, phone)
  • Do you want to be on my Facebook – send me message on Facebook.
  • Programming classes you have taken.
  • Programming projects you have done.
  • Robot projects you have done. Please write more.
  • Any hardware projects you have done, like fixing a radio or a computer, building a FPGA controller etc.
  • Your background (hardware, software, art, physics, math, biology, etc)
  • Are you a graduate or undergraduate student.
  • For each of three areas: theory, programming and practical robot building, write percentages of your project’s grade (I am not sure I will be able to take this into account in every case)
  • Do you prefer to work alone or in a team for this class?
class 478 578 projects and lab
Class 478/578: Projects and Lab
  • Meeting with Chris Clark
  • Meeting with class TA
  • Webpages with previous projects
  • Interfacing to internet
  • Lab keys (cards)
class 478 578 projects for this year
Class 478/578: Projects for this year
  • Dancing hexapods
  • Foraging hexapods
  • Robot Theatre
  • Sustainable Robot for advertising
  • Robot Guide for PSU
  • Robots controlled by iPhones, Ipads, etc.
  • Advanced theories for robotics (only for individual graduate students)
slide13
Textbook:
  • T. Bräunl Embedded Robotics, Springer 2003
plan of class
Plan of class
  • Week 1:
    • Servo programming
    • Evolutionary algorithms
  • Week 2:
    • Humanoid Robots
    • Models of robotics
      • Mapping, grammars, automata, probabilistic, Braitenberg Vehicles, natural language, logic based learning.
1 1 definition
1.1 Definition
  • Definition for: embedded system
  • A combination of hardware and software which together form a component of a larger machine.
  • An example of an embedded system is a microprocessor that controls an automobile engine.
  • An embedded system is designed to run on its own without human intervention, and may be required to respond to events in real time.
  • Source: www.computeruser.com/resources/dictionary
application areas
Application Areas

• TV

• stereo

• remote control

• phone / mobile phone

• refrigerator

• microwave

• washing machine

• electric tooth brush

• oven / rice or bread cooker

• watch

• alarm clock

• electronic musical instruments

• electronic toys (stuffed animals,handheld toys, pinballs, etc.)

• medical home equipment (e.g. blood

pressure, thermometer)

• …

• [PDAs?? More like standard computer system]

Consumer Products

application areas1
Application Areas
  • Medical Systems
    • pace maker, patient monitoring systems, injection systems, intensive care units, …
  • Office Equipment
    • printer, copier, fax, …
  • Tools
    • multimeter, oscilloscope, line tester, GPS, …
  • Banking
    • ATMs, statement printers, …
  • Transportation
    • (Planes/Trains/[Automobiles] and Boats)
  • radar, traffic lights, signalling systems, …
application areas2
Application Areas
  • Automobiles
    • engine management, trip computer, cruise control, immobilizer, car alarm,
    • airbag, ABS, ESP, …
  • Building Systems
    • elevator, heater, air conditioning, lighting, key card entries, locks, alarm systems, …
  • Agriculture
    • feeding systems, milking systems, …
  • Space
    • satellite systems, …
application areas3
Application Areas
  • Facts:
    • 1997: The average U.S. household has over 10 embedded computers (source: www.it.dtu.dk/~jan)
  • 1998: 90% Embedded Systems vs. 10% Computers
    • (source: Frautschi, www.caliberlearning.com)
  • 2001: The Volvo S80 has 18 embedded controllers and 2 busses (source: Volvo)
reactive robot is the simplest behavioral robot
Reactive Robot is the simplest behavioral robot

Brain

is a

mapping

sensors

effectors

This is the simplest robot that satisfies the definition of a robot

reactive robot in environment
Reactive Robot in environment

ENVIRONMENT is a feedback

brain

sensors

effectors

This is the simplest robot that satisfies the definition of a robot

braitenberg vehicles homework 1 idea
Braitenberg Vehicles: Homework 1 idea
  • Can you think about other robot behaviors?
  • Can you develop software for robots with other mechanics/kinematics but the same emergent principles?
  • Design circuits for switchable behaviors: like sound that switches from shy to aggressive robot.
emotional robot has a simple form of memory or state
Emotional Robot has a simple form of memory or state

Brain

is a

Finite

State

Machine

sensors

effectors

This is the simplest robot that satisfies the definition of a robot

behavior as an interpretation of a string
Behavior as an interpretation of a string
  • Newton, Einstein and Bohr.
  • Hello Professor
  • Hello Sir
  • Turn Left . Turn right.

behavior

behavior as an interpretation of a tree
Behavior as an interpretation of a tree
  • Newton, Einstein and Bohr.
  • Hello Professor
  • Hello Sir
  • Turn Left . Turn right.

behavior

Grammar. Derivation. Alphabets.

robot head construction 1999
Robot Head Construction, 1999

High school summer camps, hobby roboticists, undergraduates

Furby head with new control

Jonas

We built and animated various kinds of humanoid heads with from 4 to 20 DOF, looking for comical and entertaining values.

slide40
Mister Butcher

Latex skin from Hollywood

4 degree of freedom neck

robot head construction 2000
Robot Head Construction, 2000

Skeleton

Alien

We use inexpensive servos from Hitec and Futaba, plastic, playwood and aluminum.

The robots are either PC-interfaced, use simple micro-controllers such as Basic Stamp, or are radio controlled from a PC or by the user.

slide42

Technical Construction, 2001 Details

Marvin the Crazy Robot

Adam

slide43

2001

Virginia Woolf

heads equipped with microphones, USB cameras, sonars and CDS light sensors

slide44

2002

BUG (Big Ugly Robot)

Max

Image processing and pattern recognition uses software developed at PSU, CMU and Intel (public domain software available on WWW).

Software is in Visual C++, Visual Basic, Lisp and Prolog.

slide46

2002, Japan

Professor Perky

Professor Perky with automated speech recognition (ASR) and text-to-speech (TTS) capabilities

  • We compared several commercial speech systems from Microsoft, Sensory and Fonix.
  • Based on experiences in highly noisy environments and with a variety of speakers, we selected Fonix for both ASR and TTS for Professor Perky and Maria robots.
  • We use microphone array from Andrea Electronics.

1 dollar latex skin from China

construction details of maria

location of head servos

Construction details of Maria

skull

  • location of controlling rods
  • location of remote servos

Custom designed skin

currently the hands are not moveable
Currently the hands are not moveable.

We have a separate hand design project.

software hardware architecture
Software/Hardware Architecture
  • Network- 10 processors, ultimately 100 processors.
  • Robotics Processors. ACS 16
  • Speech cards on Intel grant
  • More cameras
  • Tracking in all robots.
  • Robotic languages – Alice and Cyc-like technologies.
face detection localizes the person and is the first step for feature and face recognition
Face detection localizes the person and is the first step for feature and face recognition.

Acquiring information about the human: face detection and recognition, speech recognition and sensors.

slide55
Use of Multiple-Valued (five-valued) variablesSmile, Mouth_Open and Eye_Brow_Raise for facial feature and face recognition.
hahoe kaist robot theatre korea summer 2004
HAHOE KAIST ROBOT THEATRE, KOREA, SUMMER 2004

Czy znacie dobra sztuke dla teatru robotow?

Sonbi, the Confucian Scholar

Paekchong, the bad butcher

we are familiar with latex and polyester technologies for faces
We are familiar with latex and polyester technologies for faces

Martin Lukac and Jeff Allen wait for your help, whether you want to program, design behaviors, add muscles, improve vision, etc.

slide64

A simplified diagram of software explaining the principle of using machine learning based on constructive induction to create new interaction modes of a human and a robot.

probabilistic state machines to describe emotions
Probabilistic State Machines to describe emotions

“you are beautiful”

/ ”Thanks for a compliment”

P=1

Happy state

“you are blonde!”

/ ”I am not an idiot”

P=0.3

“you are blonde!”

/ Do you suggest I am an idiot?”

Unhappy state

P=0.7

Ironic state

slide67

Facial Behaviors of Maria

Do I look like younger than twenty three?

Maria asks:

Response:

  • “no”
  • “no”
  • “yes”

0.7

0.3

Maria smiles

Maria frowns

probabilistic grammars for performances
Probabilistic Grammars for performances

Speak ”Professor Perky”, blinks eyes twice

P=0.1

Speak ”Professor Perky”

Where?

P=0.3

Who?

P=0.5

P=0.5

P=0.5

Speak “in some location”, smiles broadly

Speak “In the classroom”, shakes head

Speak ”Doctor Lee”

What?

P=0.1

P=0.1

P=0.1

Speak “Was singing and dancing”

P=0.1

Speak “Was drinking wine”

….

human controlled modes of dialog interaction
Human-controlled modes of dialog/interaction

Human teaches

“Thanks, I have a lesson”

“Hello Maria”

“Lesson finished”

Robot

performs

Robot asks

“Question”

“Stop performance”

“Questioning finished”

“Command finished”

“Thanks, I have a question”

“Thanks, I have a command”

Human asks

Human commands

robot receptionist initiated conversation
Robot-Receptionist Initiated Conversation

Human

Robot

What can I do for you?

Robot asks

This represents operation mode

robot receptionist initiated conversation1
Robot-Receptionist Initiated Conversation

Human

Robot

What can I do for you?

I would like to order a table for two

Robot asks

robot receptionist initiated conversation2
Robot-Receptionist Initiated Conversation

Human

Robot

Smoking or non-smoking?

Robot asks

robot receptionist initiated conversation3
Robot-Receptionist Initiated Conversation

Human

Robot

Smoking or non-smoking?

I do not understand

Robot asks

robot receptionist initiated conversation4
Robot-Receptionist Initiated Conversation

Human

Robot

Do you want a table in a smoking or non-smoking section of the restaurant?

Non-smoking section is near the terrace.

Robot asks

robot receptionist initiated conversation5
Robot-Receptionist Initiated Conversation

Human

Robot

Do you want a table in a smoking or non-smoking section of the restaurant?

Non-smoking section is near the terrace.

A table near the terrace, please

Robot asks

human initiated conversation
Human-Initiated Conversation

Human

Robot

Hello Maria

initialization

Robot asks

human initiated conversation1
Human-Initiated Conversation

Human

Robot

Hello Maria

What can I do for you?

Robot asks

human asking
Human-Asking

Human

Robot

Question

Question

Robot asks

Human asks

human asking1
Human-Asking

Human

Robot

Yes, you ask a question.

Question

Human asks

human asking2
Human-Asking

Human

Robot

Yes, you ask a question.

What book wrote Lee?

Human asks

human asking3
Human-Asking

Human

Robot

I have no sure information.

What book wrote Lee?

Human asks

human asking4
Human-Asking

Human

Robot

I have no sure information.

Try to guess.

Human asks

human asking5
Human-Asking

Human

Robot

Lee wrote book “Flowers”.

Try to guess.

Human asks

human asking6
Human-Asking

Human

Robot

Lee wrote book “Flowers”.

This is not true.

Human asks

human teaching

Human ends

questioning

Human-Teaching

Human

Robot

Questioning finished

Human asks

Human teaches

“Questioning finished”

Robot asks

Thanks, I have a lesson

human teaching1

Robot enters

asking mode

Human-Teaching

Human

Robot

What can I do for you?

Questioning finished

Human asks

Human teaches

“Questioning finished”

Robot asks

Thanks, I have a lesson

human teaching2

Human starts

teaching

Human-Teaching

Human

Robot

What can I do for you?

Thanks, I have a lesson

Human asks

Human teaches

“Questioning finished”

Robot asks

Thanks, I have a lesson

human teaching3
Human-Teaching

Human

Robot

Thanks, I have a lesson

Yes

Human teaches

human teaching4
Human-Teaching

Human

Robot

I give you question-answer pattern

Yes

Human teaches

human teaching5
Human-Teaching

Human

Robot

Question pattern:

What book Smith wrote?

Yes

Human teaches

slide92

Human-Teaching

Human

Robot

Answer pattern:

Smith wrote book “Automata Theory”

Yes

Human teaches

human teaching6
Human-Teaching

Human

Robot

Checking question:

What book wrote Smith?

Yes

Human teaches

human teaching7
Human-Teaching

Human

Robot

Checking question:

What book wrote Smith?

Smith wrote book “Automata Theory”

Human teaches

human teaching8
Human-Teaching

Human

Robot

I give you question-answer pattern

Yes

Human teaches

human teaching9
Human-Teaching

Human

Robot

Question pattern:

Where is room of Lee?

Yes

Human teaches

human teaching10
Human-Teaching

Human

Robot

Answer pattern:

Lee is in room 332

Yes

Human teaches

human checking what robot learned
Human-Checking what robot learned

Human

Robot

Lesson finished

Human teaches

“Lesson finished”

Question

Robot asks

Human asks

human checking what robot learned1
Human-Checking what robot learned

Human

Robot

Lesson finished

What can I do for you?

Human teaches

“Lesson finished”

Question

Robot asks

Human asks

human checking what robot learned2
Human-Checking what robot learned

Human

Robot

Question

What can I do for you?

Human teaches

“Lesson finished”

Question

Robot asks

Human asks

human asking7
Human-Asking

Human

Robot

Yes, you ask a question.

Question

Human teaches

“Lesson finished”

Question

Robot asks

Human asks

human asking8
Human-Asking

Human

Robot

Yes, you ask a question.

What book wrote Lee?

Human asks

human asking9
Human-Asking

Human

Robot

I have no sure information.

What book wrote Lee?

Human asks

human asking10
Human-Asking

Human

Robot

I have no sure information.

Try to guess.

Human asks

human asking11
Human-Asking

Human

Robot

Lee wrote book “Automata Theory”

Try to guess.

Observe that robot found similarity between Smith and Lee and generalized (incorrectly)

Human asks

behavior dialog and learning
Behavior, Dialog and Learning
  • The dialog/behavior has the following components:
    • (1) Eliza-like natural language dialogs based on pattern matching and limited parsing.
      • Commercial products like Memoni, Dog.Com, Heart, Alice, and Doctor all use this technology, very successfully – for instance Alice program won the 2001 Turing competition.
    • This is a “conversational” part of the robot brain, based on pattern-matching, parsing and black-board principles.
    • It is also a kind of “operating system” of the robot, which supervises other subroutines.
behavior dialog and learning1
Behavior, Dialog and Learning
  • (2) Subroutines with logical data base and natural language parsing (CHAT).
    • This is the logical part of the brain used to find connections between places, timings and all kind of logical and relational reasonings, such as answering questions about Japanese geography.
behavior dialog and learning2
Behavior, Dialog and Learning
  • (3) Use of generalization and analogy in dialog on many levels.
    • Random and intentional linking of spoken language, sound effects and facial gestures.
    • Use of Constructive Induction approach to help generalization, analogy reasoning and probabilistic generations in verbal and non-verbal dialog, like learning when to smile or turn the head off the partner.
behavior dialog and learning3
Behavior, Dialog and Learning
  • (4) Model of the robot, model of the user, scenario of the situation, history of the dialog, all used in the conversation.
  • (5) Use of word spotting in speech recognition rather than single word or continuous speech recognition.
  • (6) Continuous speech recognition (Microsoft)
  • (7) Avoidance of “I do not know”, “I do not understand” answers from the robot.
    • Our robot will have always something to say, in the worst case, over-generalized, with not valid analogies or even nonsensical and random.
questions to students
Questions to students
  • Present a concept of a robot with architecture based on combinational logic mapping. Design a function from gates.
  • Present a concept of a robot with architecture based on deterministic Finite State Machine. Show a graph or table of the machine. You can also draw a flowchart.
  • Present a concept of a robot with architecture based on probabilistic Finite State Machine. Show a graph or table of the machine.
  • Present a software internet robot for natural language conversation, similar to receptionist robot from this set of slides. The robot should discuss Intelligent Robotics Laboratory, its research, faculty and students. What are the “states of robot”? What are the key-words to transit from state to state, draw a diagram.
  • Analyze four different Braitenberg Vehicles based on a robot with kinematics of a standard car. Two of them can be similar to Shy and Aggressive robots from class.
  • Analyze four different “Braitenberg-like robots” that have a head and one hand. Two of them can be similar to Shy and Aggressive robots from class

This is not a homework, just to test your knowledge. You do not have to give it to me but you may if you want.