Crossmodal attention & public-private displays

1. Crossmodal attention & public-private displays Patrick Olivier, Stephen Gilroy, Han Cao Daniel Jackson and Christian Kray Informatics Research Institute Newcastle University, UK

2. Talk summary • public-private divide and ambient displays • spatially context and location-based services • crossmodal cognition and displays • crossmodal ambient navigation (CROSSFLOW) • extensions to display boards (CROSSBOARD) • closing discussion

3. Public-private divide • location-based services furnish information of different degrees of privacy • handheld personal displays are the standard technological response to privacy issues • personal displays: • place high demands on the user • undermine our situational awareness • need new display paradigms to address this…

4. Ambient displays (design) • subclass of peripheral displays • low cognitive load • match display & environment (calm) • sufficient information design • consistent and intuitive mapping • visibility of state • aesthetic and pleasing design • …but they are not personalised…

5. Information display & spatial context • standard configuration (location-based display) • personal display • positional sensing • spatially multiplexed (specific location, all the time) • alternative configuration(location-based display) • global (spatially) display • no positional sensing • temporally multiplexed (specific time, all locations) • user recieves a personal index to relevant information (time)

6. Crossmodal cognition • multimodal cognition widely exploited • “capacities and effects involved in process of matching information received through multiple perceptual modalities” (e.g. McGurk effect) • humans spontaneously (and pre-attentively) integrate spatial cues across modalities • GOAL: utilise crossmodal signals to cue perception of temporally multiplexed information in a public display

7. Crossmodal ambient navigation temporalmultiplexing crossmodalcue crossmodal display ambient display

9. Preliminary user study • Goal: • compare CROSSFLOW & a map on navigation task • investigate impact of CROSSFLOW on a primary task • explore the degree of ambience (cognitive load) • Design • dual-task • primary task: test a set of arithmetic questions • secondary task: find 5 of 15 targets using CROSSFLOW/map • subjects: 9 participants, 4 females and 5 males • small experimental area: 10 x 6.5 meters • three phases for each subject

11. Observations • the mean time: 28% quicker • the mean accuracy when using CROSSFLOW: 17% higher • self-reports of cognitive load lower for CROSSFLOW

12. Discussion (on observations) • crossmodal navigation led to improved performance on both the primary (arithmetic question answering) and secondary (navigation) • directions appear vaguer in a small area with dense destinations than in a larger area with larger targets • need evaluation of navigation “in the wild” • questions: • are there other forms of temporal multiplexing to exploit? • are there other situated displays to apply this to?

13. Crossmodal access to dense displays

14. Conclusions (on design space) • modality design space • modality selection & configuration • aggregation (e.g. visual continuity, dynamic vs static) • public-private divide • infrastructure (broadcast crossmodal cue schedules) • action visibility / legibility (for users and bystanders) • scalability • temporal multiplexing inherently low resolution • decompose the environment into “regions” (some tracking) • evaluation and other configurations

15. Thanks for listening!