embodied learning of qualitative models n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Embodied Learning of Qualitative Models PowerPoint Presentation
Download Presentation
Embodied Learning of Qualitative Models

Loading in 2 Seconds...

play fullscreen
1 / 18

Embodied Learning of Qualitative Models - PowerPoint PPT Presentation


  • 123 Views
  • Uploaded on

Embodied Learning of Qualitative Models. Jure Žabkar. joint work with xpero partners. Exploration and Curiosity in Robot Learning and Inference , DAGSTUHL, March 2011. problem. “ How should a robot choose its actions and experiences so as to maximize the effectiveness of its learning ?”.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Embodied Learning of Qualitative Models' - abia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
embodied learning of qualitative models

Embodied Learning of Qualitative Models

Jure Žabkar

joint work with xpero partners

Exploration and Curiosity in Robot Learning and Inference, DAGSTUHL, March 2011

problem
problem

“How shoulda robotchooseitsactions

and experiences so as to maximizethe

effectivenessof its learning?”

goals
goals
  • to learn comprehensible models
  • no extrinsic reward
  • intrinsic reward: improved prediction model about the environment
our way
our way
  • learning from scratch(no explicit background knowledge, but given a learning algorithm)
  • real robots, real-time learning
learning loop
learning loop
  • observe the environment (collect data)
  • learn a model
  • use the model to predict the effect of each action
  • choose the best action (w.r.t. active learning strategy)
  • observe the environment and check whether the predictions match new observations
starting scenario
starting scenario

Q: how does the area of the ball (as observed by the robot)change w.r.t. robot's actions?

area := #pixels of the red blob in the image from robot's camera

actions: sL, sR(the distance of the L/R wheel)

area area s l s r
area = area(sL,sR)

task: find the appropriate model

equation discovery?

we tried several algorithms, no success

motivation
motivation

why learning qualitative relations?

people most often

reason qualitatively

AI: robots should mimic

human intelligence

the area problem qualitatively
the area problem, qualitatively

if action=forward then the area increases until it becomes constant(blob occupies the whole image)

if orientation<0 and action=left (increasing the absolute value of the angle)then the area decreases until it becomes constant(zero)

...

qualitative rules
qualitative rules

prediction model gets

much more accurate,

but the predictions are

not that precise.

methods
methods
  • active learning + planning
  • learning methods:

PadéŽabkar, Možina, Bratko, Demšar Learning Qualitative Models from Numerical Data, AIJ, 2011

STRUDELKošmerlj, Bratko, Žabkar Embodied Concept Discoverythrough Qualitative Action Models, IJUFKS, 2011

QubeŽabkar et al Preference Learning from Qualitative Partial Derivatives, ECML Preference Learning Workshop, 2010

Hyper (with predicate invention mechanism)Leban, Žabkar, Bratko An experiment in robot discovery with ILPProc. ILP 2008

    • tested on simulated (billiards) and real data (medical application, robotics)
ceteris paribus
ceteris paribus

"all other things being equal"

  • e.g. partial differentiation
  • observe a qualitative relation between two selected features, other features held constant
  • qualitative relations of 3 types:
    • x increases  f(x) increases (Padé)
    • preference relation: x y  f(x) f(y)
    • structural: on(A,B,t1), on(A,C,t2)
slide13

qualitative models

data

Padé, Qube, STRUDEL

qualitative changes

machine learning,

statistics

qualitative models

slide14

qualitative models

data

Padé, Qube, STRUDEL

qualitative changes

machine learning,

statistics

qualitative models

slide15

qualitative models

data

Padé, Qube, STRUDEL

qualitative changes

machine learning,

statistics

qualitative models

l earning with structured data
learning with structured data
  • ILP with predicate invention too complex for real-time learning
  • we use ILP to learn smaller subtasks – structural qualitative changes
the concept movable
the concept "movable"

the discovered condition which distinguishes different effects of actions:

p1(Obj):-

at(T1, Obj, Pos1),

at(T2, Obj, Pos2),

neq_pos(Pos1, Pos2).

move(T, Obj):-

p1(Obj),

f1(T, Obj).

move(T, Obj):-

not p1(Obj),

f2(T, Obj).

the discovered effects of actions:

  • f1(T1, Obj):-

at(T1, Obj, Pos1),

at(T2, Obj, Pos2),

Pos1 \== Pos2,

{T2 = T1+1}.

  • f2(T, Obj):-

not f1(T, Obj).

p1 is true if the objectwas observed at two different positions