Bacs review meeting
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

BACS Review Meeting PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on
  • Presentation posted in: General

BACS Review Meeting. FCT-UC Jorge Dias 17 th – 19 th March 2008 Collège de France, Paris. FCT-UC Key Role within BACS for M13 – M24. Contributions in WP2: Bayesian models for sensor fusion (in collaboration with College de France). WP5.4:

Download Presentation

BACS Review Meeting

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Bacs review meeting

BACSReview Meeting

FCT-UC

Jorge Dias

17th – 19th March 2008

Collège de France, Paris


Fct uc key role within bacs for m13 m24

FCT-UC Key Role within BACS for M13 – M24

  • Contributions in

    • WP2: Bayesian models for sensor fusion (in collaboration with College de France).

    • WP5.4:

      • Computational Laban Movement Analysis (LMA) using the Bayesian framework for Task 5.4.2 (Vision-based detection and reconstruction of human actions ( MPS))

      • Task 5.4.3: Multi-modal sensor integration including ego-motion

    • WP6: D6.2: Robotic Implementation of Gaze Control and Image Stabilization


Fct uc overview effort infrastructure

FCT-UC Overview: Effort & Infrastructure

  • List of people involved in BACS (changes M1-12 to M13-24)

    • Paid by BACS

      • Joerg RettPhD Student1,32

      • Filipe FerreiraPhD Student6,6

      • José PradoPhD Student7,79

      • Amilcar PedrosaTechnical Support 3,04

      • Hugo FariaTechinal Support2,66

      • Luis SantosPhD Student1,1

      • Alberto NevesPhD Student4,95

      • Cátia PinhoPhD Student4,4

      • Hadi AliakbarpourPhD Student2,64

      • Total 34,5

    • Own Resources

      • Jorge DiasProfessor2,4

      • Jorge LoboProfessor2,4

      • J Filipe FerreiraPhD student2,4

      • Luis MirizolaPhD student2,4

      • Diego FariaPhD student2,4

      • Total 12

  • Infrastructure involved in BACS (M13 – M24)

    • IMPEP

    • Nicole platform for gesture recognition


Wp 5 task 5 4 2 goal

WP 5, Task 5.4.2 Goal

Computational Laban Movement Analysis (LMA) using the Bayesian framework

Problem:

The research field of computational Human Movement Analysis is lacking a general underlying modeling language.

How to map the features into symbols?

How to model human behavior?

Solution:

A semantic descriptor allowing to recognize a sequence of symbols taken from an alphabet consisting of motion-entities.

Benefit:

Establish a set of labels for observable human behavior.

The possibility to build large databases with labeled training data.


Wp 5 task 5 4 2 goal1

WP 5, Task 5.4.2 Goal

Computational Laban Movement Analysis (LMA) using the Bayesian framework

Laban Movement Analysis:

Model for human behaviour

Bayesian Model:

Probabilistic model to analyse human interaction


Wp 5 task 5 4 3 goal

WP 5, Task 5.4.3 Goal

Multi-modal sensor integration including ego-motion

Biomimetic Artificial Multimodal Perception Systems

A moving observer observes a non-static 3D scene, possibly containing several moving objects:

  • How does the observer perceive:

    • his own motion (egomotion)

    • the 3D structure of all objects in the scene

    • the 3D trajectory and velocity of moving objects (independent motion)?


Wp 5 task 5 4 3 goal1

WP 5, Task 5.4.3 Goal

Multi-modal sensor integration including ego-motion

  • We mainly expect to contribute in developing novel perceptual computational models which:

    • are based on the fusion of perceptual modalities of vision, audition, haptic and inertial sensing;

    • mimic as closely as possible biological multimodal perceptual fusion processes;

    • perform perceptual fusion within a Bayesian framework.

  • and in the process to:

    • implement unimodal perceptual modules for Bayesian cue integration within each modality.

    • implement and assemble modules with several existing state-of-the-art computational models of visual, auditory, haptic and vestibular perception.


  • Wp 6 d6 2 goal

    WP 6, (D6.2) Goal

    • Gaze control and image stabilization:

      • rely on fusing inertial and visual sensing modalities

        • human and biological system also combine the two sensing modalities for the same goal.

      • contribution of psychophysical studies

        • Bayesian models have been successfully used to explain psychophysical experimental findings

      • a robotic implementation using Bayesian inference.


    Wp 5 task 5 4 2 achievements

    WP 5, Task 5.4.2 Achievements

    Computational Laban Movement Analysis (LMA) using the Bayesian framework

    Processes for online classification

    LMA labels

    Classified behavior

    LLF

    3-D Points

    Low Level Feature Computation

    Bayesian Inference

    Bayesian Inference

    Tracking


    Wp 5 task 5 4 2 results examples and demos

    WP 5, Task 5.4.2 Results – Examples and Demos

    Move-ment

    Tracked positions

    Low-level features

    Laban descriptors

    Behavior hypothesis

    Space

    Speed Gain

    Effort.Time

    Effort.Space

    Curvature


    Wp 5 task 5 4 2 future plans

    WP 5, Task 5.4.2 Future Plans

    • D5.17FCT-UC/Probayes:

      • Publication on ‘Computational Laban Movement Analysis based on Vision and 3-D Position Estimation’ T31

    • D5.18FCT-UC/Probayes:

      • Publication on ‘Bayesian Model for Computational Laban Movement Analysis’ T33

    • D5.20FCT-UC/Probayes:

      • Publication on ‘Computational Laban Movement Analysis using Multi-Camera Systems’ T36


    Wp 5 task 5 4 3 achievements

    WP 5, Task 5.4.3 Achievements

    • Experimental Setup: The Integrated Multimodal Perception Experimental Platform (IMPEP) current version operational and new one under construction

    • Bayesian volumetric map (BVM), egocentric log-spherical, for multimodal perception of 3D structure and motion

    • An egocentric, log-spherical spatial memory map has been devised as a framework for multimodal sensor fusion, named the Bayesian Volumetric Map (BVM)

    • This map stores the independent probabilistic states of occupancy OC and velocity VC for each cell C in a volumetric grid with log-spherical configuration


    Wp 5 task 5 4 3 results examples and demos

    WP 5,Task 5.4.3 Results – Examples and Demos

    • The Integrated Multimodal Perception Experimental Platform (IMPEP) .

    new version under construction

    current version

    (PoP FP6-IST-2004-027268)

    • Artificial multimodal active perception systemwith gaze control capabilities for image stabilization and perceptual attention with:

      • a stereovision setup;

      • a binaural setup;

      • a motorised head platform, with inertial sensors emulating the vestibular system.


    Wp 5 task 5 4 3 results examples and demos1

    WP 5,Task 5.4.3 Results – Examples and Demos

    IMPEP/BVM Framework Overview


    Wp 5 task 5 4 3 results examples and demos2

    WP 5,Task 5.4.3 Results – Examples and Demos

    Using the BVM Framework for Entropy-Based Active Exploration


    Wp 5 task 5 4 3 future plans

    WP 5,Task 5.4.3 Future Plans

    • D5.15FCT-UC:

      • Integrated multimodal perception experimental platform demo V1.0 T27

    • D5.16FCT-UC/INRIA/CNRS-Gren./Probayes:

      • Publications on Bayesian models of multimodal perception of 3D structure and motion T30

    • D5.21FCT-UC :

      • Integrated Multimodal Perception Integrated Platform demo v2.0 T36

    • D5.25FCT-UC/ CNRS-LPPA:

      • Publication on ‘Bayesian visuo-inertial gaze control’ T42


    Wp 6 d6 2 achievements

    WP 6, (D6.2) Achievements

    Probabilistic Block Matching Optical Flow

    • Robotic implementation of gaze control and image stabilization

    • Simple probabilistic optical flow algorithm:

      • population code-type data structure storing two-dimensional pdfs on the image velocity space Du, Dv as an output.

      • Primarily based on Zelek's adaptation of the block matching (correlation) algorithm.

    • Bayesian program for processing of inertial data:

      • Bayesian model of the human vestibular system [Laurens and Droulez(2006)]

      • adapted to the use of inertial sensors

      • estimate the current angular position and angular velocity

      • mimicking human vestibular perception


    Wp 6 d6 2 results examples and demos

    WP 6, (D6.2) Results – Examples and Demos

    • Text

    Observed yaw and roll, pan and tilt motor control, and remaining observed optical flow.

    We can see a strong correlation between the pan and yaw, and tilt and roll signals, since the pan&tilt is compensating the observed motion at a low sample rate.

    The small remaining optical flow observed shows that the controller is working to some extent, although it is not fully reliable since abrupt motions are not observable.


    Wp 6 d6 2 future plans

    WP 6, (D6.2) Future Plans

    Our work will focus on the multimodal sensor integration within WP5, and future work will address:

    • Implement the image stabilization alrorithm with the new robotic system.

    • Adding the magnetic data to our Bayesian implementation, providing a more robust attitude estimation.

    • Focus on gaze control and attention models

    • Contact partners within BACS (CNRS-LPPA):

      • Following previous contacts, confront partners with our models and implementation and discuss possible parallel trials of robotic and psychophysical experiments

      • Going beyond the initial implementation reported in D6.2, and propose joint work and subsequent publication D5.22 T24-T42 with FCT-UC/ CNRS-LPPA:

      • D5.25: Publication on ‘Bayesian visuo-inertial gaze control’ T-42


    Wp 5 summary 1

    WP 5Summary 1

    • Major Achievements during M13-M24

      • Computational Laban Movement Analysis based on Vision and 3-D Position Estimation

      • Experimental Setup: The Integrated Multimodal Perception Experimental Platform (IMPEP) current version operational and new one under construction

      • Bayesian volumetric map (BVM), egocentric log-spherical, for multimodal perception of 3D structure and motion

    • List of Deliverables

      • D5.7 (MPS) Report on computational human pose recovery in clutter M18

      • D5.8 (FCT-UC) State of the art on (Artificial) 3D Structure and Motion Multimodal Perception M15

      • D6.2 (FCT-UC) Robotic Implementation of Gaze Control and Image Stabilization M18


    Wp 5 summary 11

    WP 5Summary 1

    • Conference

      • Rett, J., Dias, J.: Human-robot interface with anticipatory characteristics based on Laban Movement Analysis and Bayesian models. In: Proceedings of the IEEE 10th International Conference on Rehabilitation Robotics (ICORR). (2007)

      • Rett, J., Dias, J.: Human Robot Interaction based on Bayesian Analysis of Human Movements. In: Proceedings of EPIA 07, Lecture Notes in AI, Springer Verlag, Berlin. (2007)

      • Luiz G. B. Mirisola, Jorge Lobo, and Jorge Dias. 3D map registration using vision/laser and inertial sensing. In European Conference on Mobile Robots (ECMR2007), Freiburg, Germany, Sep. 2007.

      • J. Dias, Carlos Simplicio, Diego R. Faria - 3D Photo-realistic talking head for human-robot interaction - Proceedings of the 3rd Internacional Conference on Advanced Research in Virtual and Rapid Prototyping, Leiria, Portugal, 24 to 29 September, 2007

      • F. Ferreira, V. Santos and Jorge Dias, Robust Place Recognition Within Multi-sensor View Sequences Using Bernoulli Mixture Models, The 6th IFAC Symposium on Intelligent Autonomous Vehicles, IAV 2007, Toulouse, France

      • Rett, J. and Dias, J. “Computational Laban Movement Analysis using probability calculus.” In the Proceedings of Workshop on Robotics and Mathematics, RoboMat 2007.

      • Rett, J. and Dias, J.:”Bayesian models for Laban Movement Analysis used in Human Machine Interaction.” Proceedings of ICRA 2007 Workshop on "Concept Learning for Embodied Agents“.

      • Ferreira, F. , Davim, L. , Rocha, R., Santos, V. and Dias, J.:”Using Local Features To Classify Objects Having Printable Codes”. Proceedings of the International Conference 5th Workshop on European Scientific and Industrial Collaboration on promoting Advanced Technologies in Manufacturing, WESIC

    • Journal

      • Jorge Lobo and Jorge Dias, "Relative Pose Calibration Between Visual and Inertial Sensors", International Journal of Robotics Research, Special Issue 2nd Workshop on Integration of Vision and Inertial Sensors, vol.26, n.6, June 2007, pages 561-575.

      • Peter Corke, Jorge Lobo and Jorge Dias, "An introduction to inertial and visual sensing", International Journal of Robotics Research, Special Issue 2nd Workshop on Integration of Vision and Inertial Sensors, vol.26, n.6, June 2007, pages 519-535.

    • Thesis

      • Jorge Lobo, "Integraton of Vision and Inertial Sensing", PhD Thesis, Supervisor: Jorge Dias,defended July 2007.


    Wp 5 summary 2

    WP 5Summary 2

    • Major collaborations within BACS

      • RDG2: collaboration between FCT-UC, IDIAP, ETH Zürich and MPS Tübingen was initiated, with the purpose of developing tools for audiovisual VR world production, so as to create stimuli both for human perception studies (WP6.4) and for simulations related to this Task, so as to test and demo the artificial multimodal perception system. This ongoing effort has already produced a co-authored State-of-the-Art Report: “State of the Art on 3D Audiovisual APIs/SDKs for Stimulus Generation and Presentation”.

      • RDG3, a collaboration effort between the FCT-UC, INRIA Rhône-Alpes, Probayes and CNRS-Grenoble concerning the subject “Bayesian Models for Multimodal Perception of 3D Structure and Motion” involving a student exchange from M20 to M22 was undertaken, having been successfully completed and for which several deliverables are being finalised and expected to be ready by M24, namely:

        • A final Technical Report, describing the work done during the exchange period and delineating future collaboration between these partners.

        • 2 joint-publications, to be submitted for ICVS 2008 and CogSys 2008…

    • Major collaborations outside BACS

      • FCT-UC collaborates with the Perception on Purpose project (PoP FP6-IST-2004-027268, of which FCT-UC is also a partner) sharing common physical resources and know-how, namely in the construction of the new robotic vision head

    • Summary of future plans

      • Computational Laban Movement Analysis using Multi-Camera Systems

      • Bayesian models of multimodal perception of 3D structure and motion

      • Going beyond the initial implementation reported in D6.2, and propose joint work and subsequent publication D5.22 T24-T42 with FCT-UC/ CNRS-LPPA:


  • Login