Uncertainty Computation,Visualization, and Validation

1. Uncertainty Computation,Visualization, and Validation Suresh K. Lodha Computer Science University of California, Santa Cruz lodha@cse.ucsc.edu (831)-459-3773

2. Personnel • Suresh K. Lodha • PhD, CS, Rice University, 1992 • Scientific & information visualization, uncertainty quantification & visualization, multi-modal visualization, computer-aided geometric design • Uncertainty research supported by National Science Foundation & Department of Energy ASCI Program (LLNL, LANL, SNL)

3. Overview • Uncertainty representation & computation • Data/information fusion • Quality-of-service issues • Uncertainty visualization • Uncertainty validation

4. Uncertainty Visualization Pipeline

5. Sources of Uncertainty • Sensor and human limitations • Noise, clutter, jamming etc. • Modeling assumptions • Algorithm limitations • Data compression • Communication errors • Visualization-induced errors

6. Uncertainty Representation • Uncertainty formalisms used by the fusion community • Probability • Dempster-Shafer evidence theory • Fuzzy sets and possibility theory • Uncertainty representation in visualization research • Confidence intervals • Estimation error • Uncertainty range

7. Uncertainty Computation(Previous Work) • Data/Information Fusion • Knowledge-based systems • Random sets (Goodman, Nguyen, Mahler) • Visualization • NIST/ NCGIA `91 (Beard et al.) • BattleSpace `98 (Durbin et al.) • Visualization Software `96 (Globus, Uselton) • Scientific Visualization `96 -- (Lodha, Pang, Wittenbrink)

8. “Any battlefield necessarily deals with uncertainty, and it is necessary to determine ways to represent and encode the confidence level that exists for each piece of battlefield data.” – Durbin et al ’98 (NRL)

9. Designing Error Metrics • True vs. measured/observed/anticipated • Observed vs. simulated • High resolution vs. low resolution • Continuous vs. discrete • Individual source vs. multiple sources • Static vs. dynamic • Time-independent vs. time-critical • Error-free vs. error-prone communication

10. Examples of Error Metrics • Local metrics -- distance metric -- curvature metric -- sampling-number or depth metric (distribution of error) • Global metrics -- Topology metric

11. Uncertainty Metrics : Isosurfaces

12. Uncertainty Metrics : Fluid Flow Topology Original 332 cp 55% 65%

13. Research Issues • Representations and data structures for uncertainty measures • Design and integration of error metrics • Uncertainty-aware and uncertainty-reducing data processing (algorithms and models) • Common consistent uncertainty representation over a distributed mobile network ?

14. Uncertainty Visualization • How to convey uncertainty to human users? • Uncluttered display • Intuitive metaphors for mapping • Data characteristics • Multi-modality • Do NOT hide processes that produce problems for the human users? • Visualize the abstraction (e.g uncertainty pipeline, graphical models)

15. Uncertainty Visualization • Display devices /environment • screen space (monitor, PDA, workbench,..) • mobility • Modality • vision • audio • haptics

16. Uncertainty Visualization • Data types/ characteristics • scalar/vector/tensor • discrete/continuous • static/dynamic • Levels of fusion • data-level (raw/abstract) • image-level (physical phenomena) • feature-level (compressed view) • decision-level (super-compressed)

17. Uncertainty Visualization (continued) • Techniques • glyphs • deformation • transparency • texture • linking • superimposing/backgrounding • augmented reality • modality

18. Unc Viz: Example 3Fluid Flow Visualization

19. Unc Viz: Example 5 Geometric Uncertainty

20. Uncertainty Visualization: Example 4

21. Research Issues • Uncertainty mapping and metaphors for different modalities, data types and fusion tasks • Display support for a variety of uncertainty metrics/formalisms • Interactive display for uncertainty-source -> task analysis • Integration and analysis of uncertainty for decision-making?

22. Uncertainty Validation • Does addition of uncertainty information help human users in making decisions? • Can humans integrate qualitative and quantitative (or heterogeneous information) when there is uncertainty? • Task definitions • Careful design of experiments • Usability studies • Statistical analysis

23. Uncertainty Validation • Task definitions • primary level tasks (raw estimation) • secondary level tasks (correlation or simple spatio-temporal relationships) • higher level tasks • Examples • feature existence (binary decision) • feature recognition (finite multiple choices) • target aiming (zone-centered decision within a specified space-time region)

24. Validation Strategies • Formative vs. summative studies • With or without uncertainty mapping • Representative sampling of tasks, data and uncertainty mappings • Constrained, interactive or free-form environment • Within-subjects/between-subjects and tabular designs

25. Uncertainty Validation (Previous Work) • Validation of user interfaces (CHI `90s) • Validation of multi-modal mappings (Melara, Marks, Massaro (UCSC)) • Validation of uncertainty mappings

26. Uncertainty Validation: Example 1 (with M. Hansen) • Protein structural alignment (intuitive metaphors)

27. Uncertainty Validation: Example 1 (continued) • Protein structural alignment -- accuracy of discrimination

28. Uncertainty Validation: Example 2 (GIS) Rainbow Saturated

29. Uncertainty Validation:Tasks(averaging, comparisons)

30. Uncertainty Validation: (with Wittenbrink & Pang) • Vector uncertainty glyph evaluation

31. Research Issues • Construction of user evaluation environments • Conduct user evaluation studies for efficiency and accuracy • Data analysis and statistical testing • Feedback loop to improve performance • Integrated decision tool combining uncertainty approaches in visualization and command and control?

32. Concluding Remarks I • Provide human users with uncertainty information • Representation and computation of uncertainty • Uncertainty-aware and uncertainty-reducing algorithms and models • Uncertainty visualization • Visualization of uncertainty pipeline or hidden processes or abstract models • (continued)

33. Concluding Remarks II • Effective and clutter-free visualization of uncertainty along with the data/information • Sensitive to data characteristics/ fusion level/ tasks/ display environments (intuitive and cognitively accurate metaphors) • Multi-modality • Usability studies

34. Collaboration • Uncertainty representation/ fusion (UCSC, Syracuse  GTech, UCB,USC) • Uncertainty visualization (UCSC, Gtech  UCB, USC) • Multi-modal interaction (UCSC  USC, GTech) • Other MURIS/ DoD?