Business unit empowerment through declarative systems
This presentation is the property of its rightful owner.
Sponsored Links
1 / 58

Business Unit Empowerment Through Declarative Systems PowerPoint PPT Presentation


  • 36 Views
  • Uploaded on
  • Presentation posted in: General

M. D. Metadata Solutions. Business Unit Empowerment Through Declarative Systems. Why IBM DB2 PureXML™ Rocks! Dan McCreary Dan McCreary & Associates September 11, 2007. DISCLAIMER.

Download Presentation

Business Unit Empowerment Through Declarative Systems

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Business unit empowerment through declarative systems

M

D

Metadata Solutions

Business Unit Empowerment Through Declarative Systems

Why IBM DB2 PureXML™ Rocks!

Dan McCreary

Dan McCreary & Associates

September 11, 2007


Disclaimer

DISCLAIMER

  • The following views are my own and do not necessarily reflect the views of the Minnesota Department of Revenue or the State of Minnesota


Presentation abstract

Presentation Abstract

Many non-programmers feel that complex Java and .Net middle-tier systemprevents business units from being able to quickly build and modify simpleweb applications (aka CRUD applications).  DB2 version 9 has powerfulsupport for quickly storing and searching complex XML documents.  Thisfeature combined with storing XML in a web browser using XForms makesmiddle-tier code unnecessary, putting non-programmers in control ofapplications.This presentation will cover a case study that allows a business unit to"draw pictures" to build simple web applications.  These pictures (XMLSchema diagrams) are transformed directly into XForms and stored in DB2version 9.  Rapid searching is provided using XQuery.  Very little (ifany) procedural code is involved in this process.  The presentationfinishes with a set of 10 recommendations that empower business units tobuild and maintain their own applications and leverage the power of acentral DB2 database.This presentation is appropriate for business and data strategists as wellas DBAs and application architects.


Agenda

Agenda

  • How did we get here?

    • Why trees not tables?

  • Demos

    • NIEM, XML Schemas, XForms

  • Costs of translation from XML to OO to tables

  • XQuery in DB2

  • DB2 and your Enterprise Technology Strategy


Incoming

web

Incoming!

Has this web

thing gone away yet?

XML


More new xml data is being generated than relational data

More New XML Data is Being GeneratedThan Relational Data

  • Growing at 2X rate of total database market [IDC]

  • XML is now pervasive in many of organizations

  • Almost every sector has XML based standards"XML needs its own entry here to reflect its transformation of information … XML isenabling the revolution of Web services and driving a database revolution."-- eWeek "25 More Top Technologies", September 18, 2006


A happy partnership

A Happy Partnership

XForms

DB2

PureXML


Strategic consequences

Strategic Consequences

  • Empowerment of business units to build and maintain their own applications with much less involvement of IT staff

  • Dramatically reduced need for “middleware” IT staff using traditional procedural languages like Java and C#

  • Evolution of DB2 to a web service data providor and core component of ESB


Did you know

Did you Know…

  • That with XForms you can build rich-client web applications without writing a single line of JavaScript?

  • That IBM DeveloperWorks has almost 300 articles that discuss XForms?

  • That DB2 version 9 "PureXML" has fantastic support for native XML data types and supports W3C XQuery standards?


How did we get here

How did we get here?

  • From punch cards…

    to tables…

    to trees…

    to cubes and graphs…


In the beginning

In the Beginning…

There were puchcards…

…and there were 80 columns…

and the widths were fixed…

and batches of cards were called "decks“…

and they were loaded into tables in “core” memory…


And codd and date said

And Codd and Date said…

  • Let there be relationships…

Person

Role

Project

person_id

person_id

project_id

first_name

project_id

project_nm

last_name

role_nm

description

birth_date

time_pct

status

and we had the relational databases…

and we had SQL…

and we had joins…

and we had ER diagrams…

and we had PowerBuilder…

and all seemed right with the world…until


Sir berners lee said

Sir Berners-Lee Said…

  • Let there be HTML…

<html>

<head>

<title>Dan’s Homepage</title>

</head>

<body>

<h1>Welcome!</title>

<p>Welcome to my web site!</p>

<a href="about.htm">About Me</a>

</body>

</html>

and we had URIs…

and we had HTTP…

and we had we links and resources…

and we had trees (not tables)…

and we had search…until


Customers said

Customers Said..

  • We wanted to access our tables…on the web…in HTML!

html

?

head

body

h1

p

a

and there were objects…

and there was Java and JSP … and there was ASP.Net

and there were models, views, controllers…

and there was conversion from tables to objects to trees

and trees to objects to tables…

and there were rooms and rooms of programmers…


And there was chaos

And there Was Chaos…

HTML Forms

html

Objects

JSP/ASP

head

body

Hibernate

Serialization

h1

p

a

100+ Frameworks

OR Mapping

Struts

JavaScript

XML

JDBC

SQL

.Net

AJAX

Sessions

Stored Procedures

And the number of languages you need to learn and the headcount of the IT departmentwent up…and up

RDBMS


The translation pain chain

The Translation “Pain Chain”

  • From web forms to objects…to SQL inserts…to selects…to objects and back to web forms

    • Many format translations…

Name:

Street:

City:

Zip:

Web Forms

Objects

RDBMS


And the w3c said

Person

FirstName

LastName

Roles

Projects

Project

And the W3C Said…

  • Let their be simplicity…what if we…allowed you to XML in the browser…and database

html

Database

head

body

model

input

label


A single line of sql

A Single Line of SQL

INSERT into MY_TABLE values(XMLPARSE(document '

<MyDocument>

</MyDocument>

'))

The equivalent SQL for the eCRV project would have required “shredding”the XML document into over 25 tables (i.e. 25 separate INSERT statements).


Enter the metadata registry

Enter the Metadata Registry…

  • A metadata registry is a central location in the enterprise that the semantics or meaning of data elements is captured and maintained

<DataElement>

<Namespace>http://purl.org/dc/elements/1.1</Namespace>

<DataElementName>ResourceTitleText</DataElementName>

<Label>Title</Label>

<Definition>The name given to the resource,

usually by the Creator or Publisher.</Definition>

</DataElement>

Begin with the end in mind…semantic precision.


Demos

Demos

  • NIEM Subset Generator http://niem.gtri.gatech.edu/niemtools/ssgt

  • Certificate of Real Estate Value

    • http://proptax.mdor.state.mn.us:8088/exist/rest//db/crv/forms/submit/index.xhtml

  • Contacts Demo

    • http://proptax.mdor.state.mn.us:8088/exist/rest/db/contacts/index.xhtml


Xml schema xmlspy diagram

XML Schema (XMLSpy Diagram)

Solid linemeans required

Data Element

Sequence

of items

Dashed linemeans optional

Cardinality

Definition

Graphical notation that most non-programmers can learn in 20 minutes.


Xforms user interface generated

XForms User Interface (Generated)

Use XSL to transform XML Schema directly into Xforms

Use the metadata registry to extract element-specific rules

Separate semantics from constraints


Metadata driven apps data flow

Metadata Driven Apps Data Flow

NIEM

MDR

wantlist.xml

Local

MDRs

xforms-global.css

field-widths.css

NIEM Sub

Schemas

.xsd

instance.xml

patterns.xml

read-only.xml

screen-labels.xml

import

My-Form.xhtml

import

DocumentConstraint

Schema

.xsd

import

niem2xforms.xsl

This is a simplified version of the actual workflow.


How about reporting on xml data

How About Reporting on XML data?

  • OK… it is easy to get XML into DB2..

    …but how about reporting? How do I do “joins” on XML?

    …How do I update XML documents?

    …Can I mix SQL and XQuery?


Db2 viper architecture

DB2 Viper Architecture

  • Many benefits of mix and match SQL/Xquery

  • Use the right tool for the right job


Xquery

XQuery

  • W3C Recommendation 23 January 2007

  • Created by experts from SQL industry

    • IBM, Microsoft, Oracle

    • IBM played a leadership role

  • XQuery is to XML as the SELECT statement is to SQL


Xquery structure

XQuery Structure

  • FLWOR expression (pronounced “flower”)

    • For – select the sequence of nodes in the tree

    • Let – temporary variables and lists

    • Where – restrict results

    • Order – order results

    • Return – return the data embedded in a tree


Sample xquery

Sample XQuery

for $d in db2-fn:xmlcolumn(“depts.depts")//deptno

let $e := db2-fn:xmlcolumn(“emps.emps")//employee[deptno = $d]

wherecount($e) >= 10

order by avg($e/salary) descending

return

<big-dept>

{ $d,

<headcount>{count($e)}</headcount>,

<avgsal>{avg($e/salary)}</avgsal> }

</big-dept>

Returns a list of all big departments (over 10 employees),

their headcount and average salary directly in XML format


Xslt and xquery

XSLT and XQuery

  • Functionally similar languages developed by the W3C

  • Working together to ensure a common approach where appropriate.

  • They share the same

    • data model

    • type system(XML Schema data types)

    • function library

    • Both us XPath 2.0 as a sublanguage


What i like about xquery

What I like About XQuery

  • XPath expressions are very short but can still easy to read (c.f. Perl)

  • Consistent with XSL, XForms, XML Schema and Schematron (XML rules)

  • You can extend XQuery using stored procedures

  • Recursion simplifies many tasks


But can i

But Can I…

  • Do joins between XML documents…yes

  • Extend XQuery…yes

  • Create “functions” with XQuery…yes

  • Integrate Java functions…yes


Joins in xquery

Joins in XQuery

declare variable $firstName external;<videosfeaturing="{$firstName}">{for $v in db2-fn:xmlcolumn(‘videos’)//video,      $a in db2-fn:xmlcolumn(‘actors’)//actors/actorwhere ends-with($a, $firstName)and $v/actorRef = [email protected] by $v/yearreturn      <videoyear="{$v/year}">         {$v/title}      </video>}</videos>

List of videos in one file

List of actors in another

Return a list of all videos

That have an actor with a

given first name.


Xquery update

XQuery Update

  • Adds XQuery expressions to:

    • Insert

    • Update

    • Delete

      XML documents

  • Still in W3C Working Draft

    • Last call for Comments is Oct. 2007

    • Should be full recommendation by 2008

  • Available in DB2 9.1 as a stored procedure

  • Additional support in DB2 9.5


Stored procedures for update

Stored Procedures For Update

CREATE PROCEDURE ECRVADMIN.INCREMENT_COUNTY_CRVID (IN COUNTY VARCHAR(255), OUT SUCESS INT)

LANGUAGE SQL

BEGIN SET ARG1 = CONCAT(

'<updates><update action="compute" col="1" path="//County[Code=&quot;' COUNTY &quot;]/NextID/text()">)

CPMCALL DB2XMLFUNCTIONS.XMLUPDATE( ARG1, ‘

select COUNTY from ECRVADMIN.COUNTY

where id=0', 'update ECRVADMIN.COUNTY set COUNTY=? where id=0',?,?);

CALL DB2XMLFUNCTIONS.XMLUPDATE(

'<updates> <update action="compute" col="1" path="//County[Code=&quot; &quot;]/NextID/text()"> ?+1 </update> </updates>',

'select COUNTY from ECRVADMIN.COUNTY where id=0', 'update ECRVADMIN.COUNTY set COUNTY=? where id=0',?,?);

CALL DB2XMLFUNCTIONS.XMLUPDATE(

'<updates> <update action="compute" col="1" path="//County[Code=&quot;COUNTY&quot;]/NextID/text()"> ?+1 </update> </updates>',

'select COUNTY from ECRVADMIN.COUNTY where id=0',

'update ECRVADMIN.COUNTY set COUNTY=? where id=0',?,?);

END


Sidebar the semantic web

Sidebar: The Semantic Web

Subject

Predicate

Object

Node

  • Note that the semantic web infrastructure is based on graphs, not trees

  • Each node in the graph can be a URL

  • External web sites can be used to create “joins” based on a URL

Node

Node

Node

Node

Node


Strategic impact

Strategic Impact

  • Consider this:

    • 50% of Applications are simple CRUD Applications

    • Create, Read, Update, Delete

  • Complex business logic can be stored in an XML-based rules engine such as Schematron

  • How will this transform an organization?

  • How will it allow business units to build and maintain IT applications?

  • How will it allow the business to be responsive to custom needs?


Change where the line is drawn

Graphical Requirements and Specifications

SME/BA

IT Staff

Change Where the Line is Drawn

  • Shorten the “distance” between the business unit and the IT staff

Requirements

Requirements

BAs

SME

Developers

vs.


Many small vs one large

XForms

XQuery

XPath

XSLT

XML Schema

Requirements

Schematron

Many Small vs. One Large

It is easier to build a GUI on a small, precise language than a single large language


Data stewardship governance

Data Stewardship/Governance

  • Importance of the business units role in defining data elements semantics “on-the-wire”

  • How to write and maintain enterprise definitions

    • Concise

    • Precise

    • Distinct

    • Non-circular

    • Unencumbered


Security and performance

Security and Performance

  • Security and access control based on URL patterns

  • A middle tier still needed to cache web pages that have similar URLs

  • Deep REST considerations (don’t regnerate a pages if you don’t have to)

  • Stateless web server farms can still be used


Modules and quality assurance

Modules and Quality Assurance

  • XQuery “Modules” allow encapsulation of functions to manipulate complex XML

  • Need better unit testing tools for XQuery (similar to JUnit)

  • Need support for regression testing frameworks (similar to CruiseControl)


Parker projection

Procedural code (Java, JavaScript, VB, C#, C++)

Declarative code (XHTML, CSS, XSLT, XQuery, XForms)

Parker Projection

RelativeCode Base

100%

Time

Source: Jason Parker, Minnesota Department of Revenue, November 2006


10 business empowerment steps

10 Business Empowerment Steps

  • Make data stewardship a business unit imperative

  • Start by putting a simple glossary of terms and their definitions in a database

  • Build a simple metadata registry based on a simple taxonomy

  • Build a metadata "shopping cart"

  • Build exchange documents using XML Schemas that import your metadata shopping lists

  • Transform XML Schemas into XForms

  • Put your XML data in DB2 XML data types (no shredding!)

  • Use XQuery for reports directly into XML and HTML

  • Express level-two business rules by selecting XPath expressions

  • Build XML-centric ESB with DB2 databases as sources


Next steps

Next Steps

  • Proof of Technology (POT) – hands on labs

  • Get the evaluation version…start with some small XForms and XQueries

  • Create a pilot project – goal – small middle tier

    • Pick a "friendly" business unit

    • Find a medium sized problem

  • Train staff with XPath-centric tools

  • Roll out to the enterprise in stages


Developerworks article

Developerworks Article

http://www.ibm.com/developerworks/library/x-xformsniem


Xforms wikibook

XForms Wikibook

http://en.wikibooks.org/wiki/Xforms

Over 75 working XForms example programs


Xquery book

XQuery Book

  • XQuery

  • Priscilla Walmsley

  • Tutorial and Reference

  • O’Reilly - 2007

  • Everything except XQuery update


Devx article

Devx Article

  • Semantics and the Evolution of Specialized Languages

    http://www.devx.com/semantic/Article/34805


The ibm workplace xforms video

The IBM Workplace XForms Video

http://www.ibm.com/developerworks/library/x-xformspointclick.html

  • One-click generation of XML instance data

  • Drag-and-drop of complex type elements to create groups of UI controls that are already bound to data

  • Drag-and-drop of UI controls from the design palette

  • Drag-and-drop association of data to UI controls created from the design palette

  • Wizard-based generation of data-driven hiding/showing of groups of UI controls

  • Wizard-based generation of tables, including add row and delete row buttons

  • Wizard-based generation of intelligent row calculations and column summations


More resources

MoreResources

  • Keywords: XForms, XQuery, PureXML

  • This presentation:

    • http://www.danmccreary.com/presentations/db2ug


Useful links

Useful Links

  • DB2 9 Trial

    • http://www.ibm.com/software/data/db2/v9

  • DB2 9 Express-C - Free limited cpu version

    • http://www.ibm.com/software/data/db2/udb/db2express/download.html

  • IBM DeveloperWorks

    • http://www.ibm.com/developerworks/xml

    • http://www.ibm.com/developerworks/db2

  • IBM DB2 9 PureXML Wiki

    • http://www.ibm.com/developerworks/wikis/display/db2xml/Home


Thank you

Thank You!

Dan McCreary

Dan McCreary & Associates

Enterprise Data Architecture and Strategy Consulting

[email protected]

http://www.danmccreary.com

Cell: (612) 986-1552

Home: (952) 931-9198


  • Login