Cuypers: Smart Style on the Semantic Web

Cuypers: Smart Style on the Semantic Web Jacco van Ossenbruggen Lynda Hardman CWI Amsterdam

Talk overview • Requirements for 2nd& 3rdgeneration multimedia • why multimedia is different from text • Smart style • a document engineer meets a graphic designer • Overview of CWI’s Cuypers engine • testbed for multimedia transformations • Future directions • towards 3rd generation multimedia

The Web in three generations • Hand-coded (HTML) Web content • easy access through uniform interface • huge authoring and maintenance effort • hard to deal with dynamically changing content • Automated on-the fly content generation • based on templates filled with database content • later extended with XML document transformations • Automated processing of content • The Semantic Web • explicit meta-data instead of “screen scraping” • agreed upon semantics (RDFS, DAML+OIL)

Multimedia on the Web • Real multimedia Web content is still rare • Mostly bells & whistles to enhance HTML text … • … or mono-media AV-streams • Virtually all presentations are hand-authored • proprietary formats that are hard to generate • limited support for dynamic content and multichanneling • most Web technology is text/page-oriented … • … with SMIL as one of the few exceptions • Conclusion:Multimedia has hardly caughtup with the 1stgeneration Web!

2nd generation multimedia • Adapts to end-user’s platform capabilities • multichanneling: PC, PDA, mobile, voice-only, … • Adapts to the network resources available • bandwidth and other QoS parameters • Personalization • language, abilities, level of expertise, ...

But multimedia is not text... • Different document and presentation abstractions • hard to separate style from structure • Formatting is not based on text flow • no pages or scrollbars, no line-breaking or hyphenation • templates often do not work well either • Feedback from the formatting back-end required • need to check whether proposed layout is feasible • layout of media items is less flexible than text-based layout • Transformations are hard in a functional language • need to try out designs and backtrack when necessary

Smart style (our proposed solution) • The creation of a web-based infrastructure that is • an extension of the current document engineering perspective • taking into account the graphic design perspective

Document Engineering Perspective • Content is selected & structured • Mappings are defined to a new presentation structure • Styles (such as color and font) can be applied • The transformation process is linear and assumes that • Content/document structure, • presentation structure and, • style are independent of each other.

Transformation Sheet (XSLT) Stylesheet (CSS) Rendering (Web-device) HTML WML Post Script Document structure Presentation structure Styled presentation Finalform Document Engineering Perspective

Graphic Design Perspective • Basically: presentation structure, content and style depend on one another. • In multimedia presentations, spatio-temporal layout gives meaning to the presentation(in contrast to the "linear" nature of text-flow).

Grouping deter-mines layout style Presentation structure Style Layout aestheticsdetermines grouping Top-level order/groupingaffects local arrangements Overall style determineslocal style of media items Grouping deter- mines selection of media items Style determined byselected media items Grouping depends on semantic relations among media items Style determinesselection of media items Content Internal design dependencies

Example

External Forces on Design Process • Content provider • Mission (make profit, promote image) • Limited resources (cost) • Preferences (company colors) • End-user • Goals, needs • Delivery context limitations (time, environment) • Preferences (images vs text, audio vs visual) • Designer • Design experience • Resource limitations

Requirements for Smart Style Integrate Semantic Web technology with current Web engineering technology: • Communicating delivery contexts • Supporting metadata for content and description of its function • Encapsulating explicit design knowledge Processing delivery context, metadata and design knowledge within the document engineering pipeline.

Cuypers transformation engine • Cuypers testbed is based on • media independent presentation abstractions • transformation rules with built-in backtracking andconstraint solving Semanticstructure Discoursestructure Presentationstructure Constraintlayer Final form presentation Multimedia IR system HTTP Client

Cuypers transformation engine • Cuypers testbed is based on • media independent presentation abstractions • transformation rules with built-in backtracking andconstraint solving … but all knowledge is • implicit and hidden in the rules • lost in the generated Web presentation • not reusable for other Web sites

UserProfile DesignModel DiscourseModel DomainModel DeviceProfile Cuypers Knowledge Sources User Query MMDB MetadataDB Cuypers Engine

Cuypers& the Semantic Web • Domain model • Reuse existing domain ontologies, RDFS/OWL • Device profile • CC/PP • Design model • ? (local rules, global constraints, templates?) • Discourse model • ? (RST, discourse ontologies, narration models) • User model • ? (AHAM/AHA!/MAO)

Multimedia is mostly first generation hand-authored presentations common Web-tools are too text-centric Cuypers system realises second generation on-the-fly multimedia generation media-centric transformations Third generation is around the corner generating annotated multimedia reusing knowledge available on the Semantic Web Conclusions

Cuypers: Smart Style on the Semantic Web