University of Milano-Bicocca, Doctoral School of Science
1 / 26

Advisor: Prof. Francesco Tisato Tutor: Prof. Carla SimoneSeptember 23th 2009 - PowerPoint PPT Presentation

  • Uploaded on

University of Milano-Bicocca, Doctoral School of Science Ph.D. Program in Informatics, Cycle XXIV Department of Informatics, Systems and Communication

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Advisor: Prof. Francesco Tisato Tutor: Prof. Carla SimoneSeptember 23th 2009' - jana

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

University of Milano-Bicocca, Doctoral School of Science

Ph.D. Program in Informatics, Cycle XXIV

Department of Informatics, Systems and Communication

Diego BerniniPh.D. research status and thesis hypothesisSoftware Architectures forResponsive Environments:the case of Interactive Art & Design

Advisor: Prof. Francesco Tisato

Tutor: Prof. Carla Simone

September 23th 2009

Outline l.jpg

  • Reference scenario

  • Research topic, status and thesis hypothesis

  • State of the art

  • Plan of the activities

  • Miscellanea

D. Bernini, Ph.D. research status

Software architecture l.jpg
Software Architecture

  • Having a sound software architecture is a key aspect of ICT systems

  • In different classes of ICT systems relevant high-level concepts are not turned into proper elements in their software architecture

    • need of proper architectural abstractions

    • examples: time and space in surveillance systems and Geographic Information Systems (Micucci et al., 2006) (Micucci et al., 2009)

  • This leads to several issues

    • intermix of conceptual, design aspects and implementation details

    • strong dependency on specific technologies

D. Bernini, Ph.D. research status

Responsive environments l.jpg
Responsive environments

  • Real physical spaces equipped with ICT technologies capable of sensing and responding accordingly to entities that inhabit them (Silva and Vasconselos 2007) (Pinhanez and Bobick 2003)

  • Different aims

    • From more concrete …

      • Automation of offices, shops, houses (Elrod et al., 1993)

    • …to more aesthetics

      • Interactive Art & Design (Bullivant 2007) (Bullivant 2006)

  • Various related research areas, different characterizations

    • From Software Engineering, Ubiquitous (Pervasive) Computing

    • to Human-Computer Interaction, Ambient Intelligence

D. Bernini, Ph.D. research status

In interactive art l.jpg Interactive Art

  • ICT-supported interactive art installations

    • Art Installations: artworks where space is central

      • “art that appropriates space to its own artistic ends” (Bestor 1996)

    • Interactive Art installations: the viewer become an active player

    • ICT-supported: some interactions are achieved by ICT

      • For itself: they are integrant part of the installation

      • For the exhibition: valorization of traditional artworks/installations

        • Interactive museums and galleries

(Studio Azzurro,

(Rafael Lozano Hemmer,

D. Bernini, Ph.D. research status

In interactive design l.jpg
.. in Interactive Design

  • Various classes of interactive small-scale architectural & design interventions (Castle 2007) (Bullivant 2007) (Bullivant 2006)

    • Interactivity: the environment enhance the user experience

    • Interactive Architecture & Design, urban-open & closed spaces

(Interactive tunnel,

Vito Acconci Studio and CSAI,

(Bandini et al 2008))

(Digital Pavillion,

ONL [Oosterhuis_Lénárd] design office

D. Bernini, Ph.D. research status

High level concepts examples l.jpg
High-level concepts: examples

  • Time

    • real-time responsiveness: the interactions may require strong time constraints

    • temporal input analysis and time-driven output

  • Space

    • locating objects in multiple spaces (e.g. sensor network space vs. environment topology)

  • Multi-modality

    • Multiple heterogeneous sensing and responding

      • Sensing:

        • Strict physical environmental properties

          • Pressure, temperature, humidity, lighting condition, noise levels

        • Entity properties

          • Presence, identification, motion, contact, physiological

      • Responding: audio and visual effects, commanding of specific devices

D. Bernini, Ph.D. research status

Current practice l.jpg
Current practice

Various development tools, but

no explicit treatment of the previous high level concepts!

low level of abstraction

Hardware computing devices: “low level” programming

E.g. Arduino, input, output & computing device, C programming

oriented to specific technologies

Different visual programming language & tools, but too specific

E.g. Cycling’74 Max, oriented to audio and image processing

… consequently, they hardly address issues like:

Management of complexity: do not add complexity to the inherent complexity of the desiderata interactions

Integration: how to integrate new and heterogeneous technologies

Scalability: from few homogenous sensors to a lot of heterogeneous sensors

September 23th 2009

D. Bernini, Ph.D. research status


Research topic l.jpg
Research topic

Filling the gap between low-level, technology oriented aspects and high-level concepts of the previous response environments

What are suitable architectural abstractions for the development of responsive environments in Interactive Art & Design? (Trifonova 2008) (Edmonds et al., 2005) (Machin 2002)

September 23th 2009

D. Bernini, Ph.D. research status


What i have done l.jpg
What I Have Done

Characterization of a thesis hypothesis

Investigation of the state of the art

Identification of case studies and collaborations (in progress)

GAS (“Grandi Attrezzature Scientifiche”) project (UniMiB-DISCo): location aware, multimodal applications

my overall collaboration: architectural abstractions for the integration of heterogeneous components

they allow the establishment of communication flows based on space and time

my current specific task: design and implementation of a supporting platform (in progress)

September 23th 2009

D. Bernini, Ph.D. research status


Thesis hypothesis what l.jpg
Thesis hypothesis (what)

  • Devising effective architectural abstractions for responsive environments in Interactive Art and Design that:

    • Goal 1: abstract from the specific technologies (hardware-input/output and computing devices- and software- specific programming languages & components-)

    • Goal 2: go beyond ad-hoc approaches, fulfillment of real-time and multimodal requirements in an effective, robust and scalable way

    • Goal 3: are usable by artists & designers, not only technologists

    • Goal 4: are applicable to other purposes, e.g. Video-surveillance, Domotics

  • Design and development of experimental applications based on the proposed abstractions

    • One for Interactive Art & Design, another for a different context

D. Bernini, Ph.D. research status

Thesis hypothesis how and who l.jpg
Thesis hypothesis (how and who)

  • Core approach: Software Architectures (Prof. Tisato) enhanced with other disciplinary perspectives

    • Ambient Intelligence

    • Interaction Design/Human-Computer Interaction

    • Collaborative work and Knowledge Management

    • Prof. Tisato, Prof. Archetti, Prof. Bandini, Prof. De Michelis, Prof. Schettini, Prof. Simone...

  • Identified case studies, collaborations and abroad period

    • A specific workgroup among CSAI center, DISCo-ITIS and DISCo-SAL labs

    • Studio Azzurro (ref. Prof. De Michelis)

    • GAS project

    • Abroad period: acquiring knowledge & skills in interactivity vs. Art & Design vs. ICT

      • e.g. The Mixed Reality Lab, University of Nottingham (ref. Prof. De Michelis)

D. Bernini, Ph.D. research status

State of art a reference schema l.jpg






Physical environment



State of Art: a reference schema

Different conceptual

models are adopted:

from Data flow models

to Agents

ICT System organization:

from (purely) reactive,

stateful/reflexive, to emergent behaviors

  • It does NOT imply a centralized design organization and/or deployment

Responsive environment

Output flow

Input flow

ICT System

Computing & Communication

D. Bernini, Ph.D. research status

State of the art enabling technologies l.jpg
State of the Art: enabling technologies

  • Input: sensors, vision & tracking systems, tangible & wearable devices

    • Values of strict physical environmental properties and entity properties, a survey in (Cook and Das 2007)

    • Common technologies:

      • Wireless sensor networks, a survey in (Yick et al., 2008)

      • Cameras, image detection and tracking systems

        • Object tracking, human motion capture ad analysis, surveys in (Yilmaz et al., 2006) (Moeslund et al., 2006)

        • Face recognition (Colombo et al., 2007) and other systems: DISCo-Imaging and Vision Lab

      • Tangible interfaces and wearable devices (Harrison and Hudson 2008)

  • Output: lights, projectors, displays, speakers, sound synthesizers and other specific devices

    • Some common protocols: DMX (lights), MIDI (sound), Power Line Communication (PLC) and X10 (general electronic devices)

D. Bernini, Ph.D. research status

State of the art tools programming languages l.jpg
State of the Art: tools & programming languages

  • Arduino

    • “an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software”


  • Cycling ’74 Max

    • Visual programming language,

    • System as computational graph of objects (patches) representing input/output devices, processing and data containers

      • Objects: self-contained program, they communicate by messages

        • Data flow oriented model, high extensible

        • MSP package: audio synthesis

        • Jitter package: video processing

        • Support for execution in local networks

  • Others: Pure Data (PD), vvvv, Quartz Composer

    • Core-model similar to MAX, but different focus

D. Bernini, Ph.D. research status

State of the art architectural abstractions l.jpg
State of the Art: architectural abstractions

  • Software Components

    • (Heineman and Council 2001), (Shaw and Garlan 1996), component styles (Garlan 1996) (Maier and Rechtin 2000)

    • Specific component typologies: components with ports and connectors (Caflisch et al., 2005), (Senthil et al., 2009), targeted to responsive environments: context components (Jacquet et al., 2005), e-Gadgets (Markopoulos et al., 2004)

  • Time and space awareness

    • time (De Paoli and Tisato 1995), (Micucci et al., 2006), space (Micucci et al., 2009), GAS project

  • Emergent conceptual and architectural models

    • Multi-Agent models, Cellular Automata (CA) models, suitable for emergent and lifelike behaviors

    • (Bandini et al., 2008), (Silva and Vasconcelos 2007), (Sparacino et al., 2001)

D. Bernini, Ph.D. research status

State of the art conceptual frameworks l.jpg
State of the art: conceptual frameworks

Time and space

time, space and audience vs. interactivity (Benford et al., 2009)


multi-modality categories (Bongers and Veer 2007)

typologies of sensing (Benford et al., 2005)

human movement representation and responsive environments (Loke and Robertson 2009)

September 23th 2009

D. Bernini, Ph.D. research status


Plan of activities l.jpg
Plan of Activities

  • Phase 1 (investigation) - 12 months

    • Investigation of the state of the art (completed)

    • Identification of case studies & collaborations (currently in progress)

  • Phase 2 (research) – 12 months

    • Devising and development of the architectural abstractions, by generalizing the analysis of the identified case studies

  • Phase 3 (experiment) – 8 months

    • Design and development of experimental applications based on the proposed abstractions

  • Phase 4 (research) - 4 months

    • Refinement of the architectural abstractions on the basis of the previous experimental applications

D. Bernini, Ph.D. research status

Wishful thinking l.jpg
Wishful thinking

  • How artists/designers would like to conceive the interactivity, the audience/user participation, independently from available technologies?

    • From collaborations with artists/designers!

    • This is NOT A GOAL of the thesis

D. Bernini, Ph.D. research status

Target conferences and journals l.jpg
Target Conferences and Journals

  • Conferences (and related Symposia)

    • IEEE/IFIP Conference on Software Architecture (WICSA) and European Conference on Software Architecture (ECSA)

    • ACM Special Interest Group on Graphics and Interactive Techniques (SIGGRAPH) conferences

      • ACM’s Creativity & Cognition conferences

      • ACM’s International conference on Digital Interactive Media in Entertainment and Arts

    • ACM Special Interest Group for Computer–Human Interaction (SIGCHI) conferences

    • ACM International Conference on Multimedia

    • International Conference on Tangible and Embedded Interaction (TEI)

  • Journals

    • ACM Transactions on Computer Systems (TOCS)

    • Pervasive and Mobile Computing, Elsevier (PMC)

    • ACM Transactions on Computer-Human Interaction (TOCHI)

    • ACM Transactions on Multimedia, Applications, and Computing (TOMCCAP)

D. Bernini, Ph.D. research status

Courses l.jpg

  • Attended courses:

    • “Knowledge Management: conoscenza, condivisione, tecnologia”, Prof. Simone

    • “Interaction Design”, Prof. De Michelis

    • Others

      • “Laboratorio di Comunicazione” (from QUASI_SI Ph.D. Program)

      • “Comunicazione della Scienza” (Doctoral School of Science)

      • “Gestione dei progetti di ricerca” (Doctoral School of Science)

      • English course (DISCo), ongoing

  • Next courses:

    • 2/3 internal courses

      • “Aspetti epistemologici dell’informatica”, DISCo, Prof. Bandini

      • Enabling technologies: courses of Prof. Bisiani & Schettini?

    • 2/1 external courses/Ph.D. schools

D. Bernini, Ph.D. research status

Other collaborations l.jpg
Other collaborations

F.I.R.B. - Integrated Systems for Emergency project

two papers related to my Master’s degree thesis

Bernini, D., Toscani, D., and Frigerio, M. A software architecture for the deployment of executable transformation models. Proceedings of the 2009 International Conference on Wireless Communications and Mobile Computing: Connecting the World Wirelessly, ACM (2009), 47-51.

Toscani, D., Frigerio, M., and Bernini, D. Dynamic update of data analysis models in emergency systems. Proceedings of the 2009 International Conference on Wireless Communications and Mobile Computing: Connecting the World Wirelessly, ACM (2009), 37-41.

workshop participations and talks

Leipzig, 21-24 June 2009, International Workshop on Advanced Topics in Mobile Computing for Emergency Management: Communication and Computing Platforms (MCEM 2009), presentation of (Bernini et al., 2009)

Rome, 3-4 April 2009, InSyEme internal workshop, talk “From Application Requirements to Application Architecture”

September 23th 2009

D. Bernini, Ph.D. research status


References 1 l.jpg
References (1)

  • Overview of Responsive Environments, Interactive Art & Design

    • Bullivant, L. Responsive Environments: architecture, art and design. Victoria and Albert Museum, 2006.

    • Bullivant, L. 4dSocial: Interactive Design Environments. John Wiley & Sons, 2007.

    • Castle, H. Editorial. Architectural Design 77, 4 (2007), 4-5.

    • Elrod, S., Hall, G., Costanza, R., Dixon, M., and Rivières, J.D. Responsive office environments. Commun. ACM 36, 7 (1993), 84-85.

    • Edmonds, E.A., Weakley, A., Candy, L., Fell, M., Knott, R., and Pauletto, S. The studio as laboratory: combining creative practice and digital technology research. Int. J. Hum.-Comput. Stud. 63, 4-5 (2005), 452-481.

    • Machin, C.H.C. Digital Artworks: Bridging the Technology Gap. Proceedings of the 20th UK conference on Eurographics, IEEE Computer Society (2002), 16.

    • Trifonova, A., Jaccheri, L., and Bergaust, K. Software engineering issues in interactive installation art. International Journal of Arts and Technology 1, 1 (2008), 43-65.

D. Bernini, Ph.D. research status

References 2 l.jpg
References (2)

  • Enabling technologies, tools & programming languages

    • Bestor, C. MAX as an overall control mechanism for multidiscipline installation art. Computers & Mathematics with Applications 32, 1 (1996), 11-16.

    • Cook, D.J. and Das, S.K. How smart are our environments? An updated look at the state of the art. Pervasive Mob. Comput. 3, 2 (2007), 53-73.

    • Colombo, A., Cusano, C., and Schettini, R. Face^3 a 2D+3D Robust Face Recognition System. Proceedings of the 14th International Conference on Image Analysis and Processing, IEEE Computer Society (2007), 393-398.

    • Harrison, C. and Hudson, S.E. Scratch input: creating large, inexpensive, unpowered and mobile finger input surfaces. Proceedings of the 21st annual ACM symposium on User interface software and technology, ACM (2008), 205-208.

    • Pinhanez, C.S. and Bobick, A.F. Interval scripts: a programming paradigm for interactive environments and agents. Personal Ubiquitous Comput. 7, 1 (2003), 1-21.

    • Moeslund, T.B., Hilton, A., and Krüger, V. A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104, 2 (2006), 90-126.

    • Yick, J., Mukherjee, B., and Ghosal, D. Wireless sensor network survey. Comput. Netw. 52, 12 (2008), 2292-2330.

    • Yilmaz, A., Javed, O., and Shah, M. Object tracking: A survey. ACM Comput. Surv. 38, 4 (2006), 13.

D. Bernini, Ph.D. research status

References 3 l.jpg
References (3)

Software architectures and architectural abstractions

Caflisch, L., Savigni, A., Schettini, R., and Tisato, F. A software architecture for real-time, embedded monitoring systems. Advanced Video and Signal Based Surveillance, 2005. AVSS 2005. IEEE Conference on, (2005), 540-545.

De Paoli, F. and Tisato, F. Architectural Abstractions for Real-Time Software. Proceedings of the Second Asia Pacific Software Engineering Conference, IEEE Computer Society (1995), 199.

Heineman, G.T. e Councill, W.T., cur. Component-based software engineering: putting the pieces together. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2001.

Garlan, D. and Shaw, M. An Introduction to Software Architecture. Carnegie Mellon University, 1994.

Jacquet, C., Bourda, Y., and Bellik, Y. An architecture for ambient computing. Intelligent Environments, 2005. The IEE International Workshop on (Ref. No. 2005/11059), (2005), 47-54.

Markopoulos, P., Mavrommati, I., and Kameas, A. End-User Configuration of Ambient Intelligence Environments: Feasibility from a User Perspective. In Ambient Intelligence. 2004, 243-254.

Maier, M.W. and Rechtin, E. The art of systems architecting (2nd ed.). CRC Press, Inc., 2000.

Micucci, D., Oldani, M. and Tisato, F. Time-Aware Multi Agent Systems. In Proocedings of Multiagent Systems and Software Architecture (MASSA). D. Weyns and T. Holvoet (Eds.), 2006, 71-78.

Micucci, D., Tisato, F., and Adorni, M. Engineering spatial concepts. Knowl. Eng. Rev. 24, 1 (2009), 77-93.

Senthil, R., Kushwaha, D.S., and Misra, A.K. An improved component model for component based software engineering. SIGSOFT Softw. Eng. Notes 32, 4 (2007), 9.

Shaw, M. e Garlan, D. Software architecture: perspectives on an emerging discipline. Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 1996.

September 23th 2009

D. Bernini, Ph.D. research status


References 4 l.jpg
References (4)

Emergent conceptual and architectural models

Bandini, S., Bonomi, A., Vizzari, G., et al. A CA-Based Approach to Self-Organized Adaptive Environments: The Case of an Illumination Facility. Proceedings of the 2008 Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems Workshops - Volume 00, IEEE Computer Society (2008), 1-6.

Silva, F.S.C.D. e Vasconcelos, W.W. MANAGING RESPONSIVE ENVIRONMENTS WITH SOFTWARE AGENTS. Appl. Artif. Intell. 21, 4-5 (2007), 469-488.

Sparacino, F., Davenport, G., and Pentland, A. Media in performance: interactive spaces for dance, theater, circus, and museum exhibits. IBM Syst. J. 39, 3-4 (2000), 510, 479.

Conceptual frameworks

Benford, S., Giannachi, G., Koleva, B., and Rodden, T. From interaction to trajectories: designing coherent journeys through user experiences. Proceedings of the 27th international conference on Human factors in computing systems, ACM (2009), 709-718.

Benford, S., Schnädelbach, H., Koleva, B., et al. Expected, sensed, and desired: A framework for designing sensing-based interaction. ACM Trans. Comput.-Hum. Interact. 12, 1 (2005), 3-30.

Bongers, B. and Veer, G.C. Towards a Multimodal Interaction Space: categorisation and applications. Personal Ubiquitous Comput. 11, 8 (2007), 609-619.

Loke, L. and Robertson, T. Design representations of moving bodies for interactive, motion-sensing spaces. Int. J. Hum.-Comput. Stud. 67, 4 (2009), 394-410.

September 23th 2009

D. Bernini, Ph.D. research status