semantics for innovation in visualization and multimedia smarter information science l.
Skip this Video
Download Presentation
“ Semantics ” for Innovation in Visualization and Multimedia: Smarter Information Science

Loading in 2 Seconds...

play fullscreen
1 / 34

“ Semantics ” for Innovation in Visualization and Multimedia: Smarter Information Science - PowerPoint PPT Presentation

  • Uploaded on

“ Semantics ” for Innovation in Visualization and Multimedia: Smarter Information Science. ICSTI Workshop February 8, 2011, Redmond WA. Peter Fox (RPI) Tetherless World Constellation Please buckle your seatbelt. Working premise and the burden

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about '“ Semantics ” for Innovation in Visualization and Multimedia: Smarter Information Science' - Philip

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
semantics for innovation in visualization and multimedia smarter information science

“Semantics” for Innovation in Visualization and Multimedia: Smarter Information Science

ICSTI Workshop

February 8, 2011, Redmond WA

Peter Fox (RPI)

Tetherless World Constellation

please buckle your seatbelt
Please buckle your seatbelt
  • Working premise and the burden
  • Opportunity – new means
    • Linked open data (LOD)
    • Open-source software (Field)
  • Science conduct
  • Semiotics of portrayal
    • Includes semantics
  • Representing e.g.
    • Uncertainty, quality, bias
  • Speculation

Tetherless World Constellation

working premise
Working premise

Scientists – actually ANYONE - should be able to access and use a global, distributed knowledge base of scientific data that:

  • appears to be integrated
  • appears to be locally available

But… data and information is obtained by multiple means (instruments, models, analysis) using various (often opaque) protocols, in differing vocabularies, using (sometimes unstated) assumptions, with inconsistent (or non-existent) meta-data. It may be inconsistent, incomplete, evolving, and distributed AND created in a form that facilitates generation, not use (except by accident)

And … significant levels of semantic heterogeneity, large-scale data, complex data types, legacy systems, inflexible and unsustainable implementation technology…


changing the equation
Changing the equation
  • “Changing the Equation for Scientific Data Visualization” – Fox and Hendler (Feb 11, 2011) Science (Perspectives), in press (embargoed, sorry)
  • Three important points
    • Unlocked data (and it’s big, really, really…)
    • Visualization for the masses throughout the ‘life-cycle’ but scale-free (!)
    • Smarter data, smarter visualization
data has lots of audiences
.. Data has Lots of Audiences

More Strategic

Less Strategic

From “Why EPO?”, a NASA internal

report on science education, 2005

Science too!

shift the burden from the user to the provider for viz too
Shift the Burden from the Userto the Provider – for Viz. too!

Fox Informatics and Semantics, © 2008

l inked open data
Linked open data
  • Simply put: data is in RDF and has a URI and/ or it’s behind a query-able ‘triple-store’ interface





new means artists to the rescue
New means – artists to the rescue
  • Digital artists, they needed good creative visual tools, art at the speed of creative thought, feeling, intuition, mental representation and they love programming
  • And, RPI has EMPAC – Experimental Media and Performing Arts Center
what we are doing
What we are doing
  • Field meets Linux!
  • Linked data meets Field!
    • Feed the current LOD graphics into Field for manipulation
    • Then….
      • Unscrew the Google graphics
      • Unscrew the JSON feed
      • Query / consume raw RDF
      • Visualizing at the speed of thought/ typing..
      • From the laptop to scale
  • So this is where the semantics re-enter, especially for portrayal
l inked open data14
Linked open data
  • Field consumes JSON (webify it)





l inked open visualization
Linked open visualization
  • Field queries ‘triple stores’ (semantic webify it)




l inked open visualization16
Linked open visualization
  • Field queries ‘triple stores’ (semantic webify it)





so what about abduction
So what about abduction?
  • No, not the criminal meaning…
  • Is a method of logical inference introduced by Peirce which comes prior to induction and deduction for which the colloquial name is to have a "hunch".
  • Abductive reasoning
    • starts when an inquirer considers of a set of seemingly unrelated facts,
    • armed with an intuition that they are somehow connected
    • oh, wait, good job for visualization!!!
  • Leverage open world, semantics, too… on the web
information theory
Information theory
  • Semiotics, also called semiotic studies or semiology, is the study of sign processes (semiosis), or signification and communication, signs and symbols, into three branches:
    • Syntactics: Relation of signs to each other in formal structures
    • Semantics: Relation between signs and the things to which they refer; their denotata
    • Pragmatics: Relation of signs to their impacts on those who use them
semiotics of portrayal
Semiotics of portrayal
  • But we are talking about a digital world increasingly more than an analog one
  • Beyond the separation of content from presentation
    • We have means for content (context and structure) semantics but pragmatics?
  • Portrayal (not just ‘maps’ or ‘graphs’)
    • How – representation of content, context and structure, capture visualization provenance
    • Graphs, points, lines polygons, titles, axes, color, shade, dimensions, … and their relation to each other!
for science viz
For science viz.
  • We leave the untold things untold – big (really, really big) problem(s) – like:
    • Uncertainty
    • Quality
    • Bias
    • Need evidence
  • An example?
modis terra aqua vs airs cloud top pressure
MODIS Terra & Aqua vs. AIRS Cloud Top Pressure

AIRS vs. MODIS Terra


Correlation maps for Jan 1 – 16, 2008

Impact:Throw your hands up in the air and just walk away, silently…

MODIS Aqua vs. MODIS Terra


Known Issues: The difference of EQCT and Day Time Node, modulated by data-day definition, caused the included overpass time difference, which makes the artifact difference. See sample images:


Included Overpass time Difference

MODIS Terra vs. MODIS Aqua AOD Correlation

abductive information system
Abductive Information System?
  • What would this look like in application tools? How to explore ‘hunches’ (hints)?
  • If you consent that induction is fundamentally part of how an information system is developed, then how to allow for abduction before induction may be possible?
  • Open world, integrative
  • Design factors? Architecture factors? Library factors? Cognitive factors?
  • But back to big data and the need to turn the visualization ‘walls’ into exhibits, 4-dimensions – installations – i.e. not immersion but experience
    • Synesthesia – why only one sense?
    • Rapid
  • At scale – why?
  • Stereo – why?
  • Linked to the live data – minimal curation!
  • Goal: restore abductive reasoning to the conduct of science for specialists and non-specialists
  • And this has to be informatics-based not some ad-hoc techies making stuff up…
  • Collaboration – wanna play?
so long and
So long and …

Need to be here

Curation stages

20080602 Fox VSTO et al.


Mind the Gap!

  • There is/ was still a gap between science and the underlying infrastructure and technology that is available
  • Informatics - information science includes the science of (data and) information, the practice of information processing, and the engineering of information systems. Informatics studies the structure, behavior, and interactions of natural and artificial systems that store, process and communicate (data and) information. It also develops its own conceptual and theoretical foundations. Since computers, individuals and organizations all process information, informatics has computational, cognitive and social aspects, including study of the social impact of information technologies. Wikipedia.
  • Cyberinfrastructure is the new research environment(s) that support advanced data acquisition, data storage, data management, data integration, data mining, data visualization and other computing and information processing services over the Internet.
modern informatics enables a new scale free framework approach
Modern informatics enables a new scale-free** framework approach
  • Use cases
    • requirements
  • Stakeholders
  • Distributed authority
  • Access control
  • Ontologies
  • Maintaining Identity
tetherless world constellation tw rpi edu
Tetherless World
  • Future Web
  • Web Science
  • Policy
  • Social



  • Xinformatics
  • Data Science
  • Semantic eScience
  • Data Frameworks



  • Semantic Foundations
  • Knowledge Provenance
  • Ontology Engineering Environments
  • Inference, Trust

Multiple depts/schools/programs ~ 35 (Post-doc, Staff, Grad, Ugrad)