Enterprise data architecture
1 / 36

Enterprise (Data) Architecture - PowerPoint PPT Presentation

  • Uploaded on

Enterprise (Data) Architecture. Christian Nentwich Model Two Zero. Caveat Emptor. This is an introductory overview, a detailed treatment would take at least a whole term! It’s fiendlishly complex – nobody has a satisfactory answer. About Model Two Zero.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Enterprise (Data) Architecture' - bianca

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Enterprise data architecture

Enterprise (Data) Architecture

Christian Nentwich

Model Two Zero

Caveat emptor
Caveat Emptor

This is an introductory overview, a detailed treatment would take at least a whole term!

It’s fiendlishly complex – nobody has a satisfactory answer

About model two zero
About Model Two Zero

  • Software company based near “Silicon Roundabout” (Old Street, London)

  • Mission: help large businesses establish the next generation of enterprise data architectures

  • Innovators in:

    • Executable specifications

    • Code generation from (restricted) natural language

    • High-volume document matching

About me
About Me

  • BSc and PhD in Computer Science from University College London

  • Never held down a “proper job”...

  • Founded two startups – Model Two Zero is number 2!

  • Long involvement in financial services IT

    • Strategic data architecture consultancy for many big banks, fund managers, clearing houses

    • Pushing forward industry-wide data standards


  • Large companies build or buy new systems all the time

  • Nobody likes monolithic systems, but components need to be assembled

Batch load or real-time query

Customer Records

Invoicing System

Batch load

Real-time message

Order Management System


  • Companies also buy other companies

  • Consolidation requirements

Company B

Accounting System

Company A

Accounting System


Accounting System

No planning chaos
No planning  Chaos

Format diversity
Format Diversity

  • Relational databases

  • APIs

    • Java

    • .Net

    • Mainframe

  • File formats

    • Comma-separator

    • Fixed width

    • XML

    • COBOL mainframe data files

    • Tag/Value files

    • Proprietary binary

  • Message formats

    • XML

    • JSON

    • Proprietary queueing systems

  • Taking a consistent approach is a challenge


  • Domain complexity

  • Architecture diversity

  • Volume of data

  • Complex, often out of date business processes

  • Key person dependencies

  • Lost business knowledge

  • Time pressure

Esb key idea
ESB – Key Idea

  • From this:

Esb key idea1
ESB – Key Idea

  • To this:

Common Bus

Zoomed in view
Zoomed in View

Common Bus

Standardised Data Format


Proprietary Data Format

Proprietary System


  • Bilateral connections require O(n2) integration projects for n systems

  • ESB with standard formats and adapters requires 2n = O(n) integration projects

  • This calculation does not always work out in practice!

  • Why?

Key disciplines
Key Disciplines

  • Data Modelling

    • The data is more important than any execution technology!

    • Need to define a “standard language” for interchanging data on the bus

    • Very frequently XML and XML Schema

    • Sometimes JSON, but schema definition is problematic

  • Governance

    • Curation of the standard

    • Versioning (one of the hardest problems in data architecture)

    • Regression testing and rollout management

Quick comparison
Quick Comparison

  • UML

    • Fits lots of information on one page

    • Understood by relatively technically unsophisticated users

    • A huge standard, contains more than is necessary for the task

    • Has no “wire format”

  • XML Schema

    • Technically complex

    • Comes with a wire format – XML

    • Liked by developers

    • Fairly big standard, contains more than is necessary for the task


  • Central team

  • Distributed teams (open source style)









  • Versioning of standard data formats needs to deal with two important scenarios:

    • Backward compatibility: a new version of a connecting system is able to read/write an old version of the standard

    • Forward compatibility: an old version of a connecting system is able to read/write a newer version of the standard

  • Careful not to confuse them – most people do

Versioning example
Versioning Example


System A








<postcode>ABC CDE</postcode>




System B

Versioning example1
Versioning Example


System A









<postcode>ABC CDE</postcode>




System B


  • Adding optional elements

    • Leaves senders unaffected

    • Breaks receiver forward compatibility if receivers cannot deal with unknown elements

  • Adding mandatory elements

    • Breaks senders

    • Breaks receivers

  • Removing mandatory elements

    • Breaks senders

    • Breaks receivers

  • Removing optional elements

    • Breaks senders

    • Leaves receivers unaffected

Service oriented architecture
Service-Oriented Architecture

  • Establish a landscape of coarse grain business services

  • Establish service wrappers around applications

  • Build “value add” applications by composing / orchestrating services

Reservation Service

Customer Management


Flight booking


Service oriented architecture1
Service-Oriented Architecture

WSDL Wrapper

REST Wrapper

Reservation Service

Customer Management


Flight booking



Event driven architecture
Event-Driven Architecture

  • Build an architecture build on business eventsrather than data

  • Components of the architecture subscribe for updates and listen for events they are interested in (sources and sinks model)

From this:







To this:







Event modelling
Event Modelling














Event driven architecture1
Event-Driven Architecture

  • Discipline: send only data required for an event

  • Model things that actually happen in the business

  • Let participants determine how to react to events

    • No central planning

    • Scalability benefits

Semantic integration
Semantic Integration

  • Instead of “syntactic” manipulation of data, specify the full meaning of data in each system

  • Relate data items to an overarching “ontology”

  • Infer integration automatically

  • Immature discipline – many issues to solve

Trade Date




Epilogue what we do
Epilogue: What we do

Instead of writing informal specifi-

cations and then code, create

executable specifications.

What we do executable specifications
What we do – Executable Specifications


publicvoid validate(Transaction transaction) {

if (transaction.getDebit() != null && transaction.getCredit() != null) {

if (transaction.getDebit().getValue() == transaction.getCredit()

.getValue() && getChangeInAmount() != null) {


"Must not specify changeInAmount if a debit is equal to a credit"));





Context: Transaction

Validation Rule"Our Sample Rule"

Ifa debit ispresentanda credit ispresentthen


Report'Must not specify changeInAmount if a debit is equal to a credit'


context: Transaction

inv: (self->debit = self->credit) implies (self->changeInAmount->empty())


<rule context="ns:transaction">

<assert test="ns:debit != ns:credit or not(ns:changeInAmount)">

Must not specify changeInAmount if a debit is equal to a credit



Natural rule language
Natural Rule Language

  • An open language for expressing:

    • Validation rules (constraints)

    • Action rules (e.g. Enrichment rules)

    • Transformation / mapping

  • Aimed at the core problem areas in integration

  • Goals

    • Read like an English sentence wherever possible

    • Require no customisation to get going

    • Offer symbolic and textual alternatives

  • Specification: http://nrl.sourceforge.net

Nrl parser
NRL Parser

  • The NRL parser is free and open source, also on sourceforge

  • Designed for processing / code generation

1. Text in NRL concrete syntax

2. Parser-generated AST



3. Code generators (from AST)



Possible projects
Possible Projects

  • Propose solutions to the model versioning / XML schema versioning problem

    • Classify permissible changes in detail

    • Specify a formal logic for proving the impact of changes, and establishing policies

    • Review W3C approaches to the problem and contrast

Technologies to review
Technologies to Review

  • Data standards / formats

    • XML / XML Schema

    • JSON

  • Semantic standards

    • RDF

    • OWL

    • SBVR

    • NRL

  • Open source frameworks

    • Spring integration

    • Mule ESB

  • Messaging

    • AMQP

    • SOAP / XML RPC

    • JMS

  • Useful

    • REST

    • XPath

    • Xquery

    • Schematron

    • JAXB

    • JIBX

    • SAX / DOM

    • Python format modules