Databases in Internet Applications: Case Studies Anil Nori CTO AserA Inc. Palo Alto USA anori@asera.com - PowerPoint PPT Presentation

Databases in
Download
1 / 58

Databases in Internet Applications: Case Studies Anil Nori CTO AserA Inc. Palo Alto USA anori@asera.com. Acknowledgements. Sources for some of the material Oracle Corporation CNN Custome News Excite Cisco. Database Technology Timeline. Simple Data Management.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Databases in Internet Applications: Case Studies Anil Nori CTO AserA Inc. Palo Alto USA anori@asera.com

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Acknowledgements

Databases in

Internet Applications:

Case Studies

Anil Nori

CTO

AserA Inc.

Palo Alto

USA

anori@asera.com


Acknowledgements

Acknowledgements

Sources for some of the material

  • Oracle Corporation

  • CNN Custome News

  • Excite

  • Cisco


Database technology timeline

Database Technology Timeline

Simple Data

Management

Global Enterprise Management

Early 80s

Late 80s

Early - Mid 90s

Late 90s - 21st C

EarlyRelational

Client-server

Relational

Enterprise -capable

Relational

Internet

Computing

Pre-

relational

Packaged & Vertical Applications

Data

Warehouse &

Hi-end OLTP

Simple

OLTP

Active

Database

Middleware (messaging, queues, events)

Java, CORBA, Web interfaces

Scaleable OLTP, parallel query, partitioning, cluster support, row-level locking, high availability

Simple transactions, on-line

backup & recovery

Support for all types of data, extensibility, objects

Stored procedures, triggers


Current state of dbmss

Current State of DBMSs

  • OLTP applications

    • Large amounts of data

    • Simple data, simple queries and updates

      • Update statement from debit/credit transaction:UPDATE accountsSET abalance = abalance + :deltaWHERE aid = :aid;

    • Typically update intensive

    • Large number of concurrent users (transactions)

  • Data warehousing applications

    • Large amounts of data

    • Simple data but complex querying

    • Typically read intensive

    • Large number of users


Current state of dbmss1

Current State of DBMSs

  • These applications require:

    • Large users/transactions

    • High performance

    • High availability (7x24 operations)

    • Scalability

    • High levels of security

    • Administrative support

    • Good utilities


Internet applications challenges

Transaction Processing

Larger User Populations

Trained

Self-Service

Network Systems

Gigabytes

Terabytes

Independent

Integrated

Systems Management

Usage

Batch

Immediate

Simple

Intelligent

Operations Hours

Importance

Local

Global

Business-Critical

Useful

Internet Applications: Challenges

Data Warehousing

Users

Analysts

Every Employee

Size


Acknowledgements

E-commerce/Apps

Information Management

APIs

Type

Proprietary

Open

Tabular

Heterogeneous

Applications

Delivery

Standalone

Integrated

Generic

Personalized

Access

Read/write

Lots of read-only

Content

Direct

Search

Internet Applications: Challenges

Site Operation

Management

Low TCO, Mission Critical

Availability

Occasional

24X7


Internet challenges

Internet Challenges

  • Availability

    • Need near 100% availability

    • Must be easy to manage

    • Replication, hot standby, foolproof system?

  • Scalability

    • Number of users is orders of magnitude higher

  • Security

    • Global users

    • Managing millions of users

    • Encryption

    • Performance

  • Internet user expectations

    • Speed vs correctness

      • (e.g. Search engines vs blade/cartridge/extender

    • Availability vs correctness


Internet application architecture today

Internet Application Architecture: Today

Client Tier

authoring

Browser

Browser

tools etc.

HTTP

HTTP

Physical Middle Tier

WEB/APP Server

Data Integration,

Storage, Query,

Management

Middle Tier

Application

Application messages

Remote messages

Gateways

Data Sources

Other

OLE/DB

ORDBMS

Data

Data source

Sources


Case studies

Case Studies

  • CNN Custom News

  • Excite

  • Cisco Internet Applications


Cnn custom news

CNN Custom News

  • On-line news service

  • Allows users to customize news in a personalized manner

  • Offers variety of news items (e.g. national, international, business etc.)


Custom news application architecture

Application

Server

Application

Server

Application

Server

Oracle

DBMS

Oracle

DBMS

Custom News Application Architecture

Client Tier

Browser

Browser

HTTP

Hardware Load Balancing

Physical

Middle Tier

WEB Server

WEB Server

WEB Server

...

Database

Tier

OPS


Cnn custom news1

CNN Custom News

  • Backend:

    • SUN SOLARIS enterprise servers

    • Oracle Parallel Server 7.3.4

  • Middle-Tier (9 Machines)

    • Web Servers

    • Oracle Application Servers

    • PL/SQL Cartridges

  • Load Balancing

    • Harware based

    • DNS router

    • Round -robin


Oracle application server

Cartridge

Cartridge

Cartridge

Oracle Application Server

Adapter

CORBA Backend


Cnn custom news2

CNN Custom News

  • Data feeds into the database

  • Keeps text in the database

  • Images in files

  • Images accessed in the middle-tier

  • PL/SQL Cartridge


Pl sql cartridge

PL/SQL Cartridge

PL/SQL Cartridge

Connection pooling

Session Caching

Parameter Marshalling

Validation

Result Processing

OAS

Oracle

DBMS

PL/SQL


Pl sql

PL/SQL

  • Server-side

  • Used to generate HTML

  • Suited for database logic


Searching

Searching

  • Uses Oracle ConText cartridge

  • Content-based searching

  • Uses bitmap indexes


Cnn custom news observations

CNN Custom News: Observations

  • Database-centric

  • Uses PL/SQL based scripting

  • Application Server for scalability


Excite

Excite

  • Personalized online service that gives Web users everything they want, all in one place

  • Builds tools that manage vast amounts of information available on the internet

  • Provides variety of user services (apps):

    • News

    • Money and Investing -- stock quotes

    • Message boards and Chat

    • Mail

    • Communities

    • Classifieds

    • Jobs


Excite1

Excite

  • Supports suite of applications

  • Each application uses three-tier architecture

  • Federated approach

    • Many databases

    • Databases specific to applications

  • Application logic in the middle-tier as multi-threaded embedded C programs (pro*c programs)


Excite an application architecture

Middle Tier

Application

Middle Tier

Application

Middle Tier

Application

Oracle

DBMS

Excite: An Application Architecture

Client Tier

Browser

Browser

HTTP

HTTP

Physical

Middle Tier

WEB Server

WEB Server

Database

Tier


Excite pfp application

Excite - PFP Application

  • Personalized front page application

  • Application is deployed as 50 middle-tier daemon processes

  • The middle-tier application daemons perform:

    • Application logic in C

    • Connection pooling

      • Each daemons keeps about 40 connections to the database (about 2000 total connections to the database)

    • Load balancing


Excite pfp database configuration

Excite - PFP Database Configuration

  • Oracle8 on SUN solaris server

    • 2 SUN 6500s -- 28 way SMP

  • PFP database is split into multiple databases for load balancing and scalability

  • Scalar data stored in the database in relational tables

  • About 20 tables for storing user profiles; 100 tables for content


Excite pfp database configuration1

Excite - PFP Database Configuration

  • Multi-media content (e.g. Stock quotes or news item) stored in memory mapped files for fast access. File references stored in the database

  • Lot of the content is read-only; need not be backed up; can be reconstructed from the original sources


Excite scalability

Excite - Scalability

  • By partitioning the application across multiple databases

  • Each application partition supported by multiple middle-tier daemon processes

  • Multiple web servers to reduce traffic congestion


Excite availability

Excite - Availability

  • Using replication and hot standby

  • Uses oracle8 hot standby feature

  • Uses asynchronous replication. Data replicated at 10 sec latency

  • Almost every database is replicated for failover

  • Replication preferred over hot standby. Hot standby cannot be used for normal usage


Excite other applications

Excite - Other Applications

  • Most of the Excite applications have similar three-tier architecture


Excite observations

Excite - Observations

  • Some content (specially, for communities applications) could be stored in the database. Management benefits attractive. If content stored in the database, access performance is very critical

  • Need fast replication

  • Currently not using middle-tier caching. Caching could be quite useful but coherency is an issue


Cisco

Cisco

  • Successfully implemented applications for the internet

  • Internet commerce

    • Order placement

    • Checking order status

    • On-line, guided product configuration

    • Price quotes

  • Employee self-service

    • Provides all employee services electronically

    • Employee directories

    • Employee benefits

    • Expense reports


Cisco1

Cisco

  • Supply chain management

    • Networked suppliers, resellers and customers

    • Enables business partners to manage and operate major portions of its supply chain

    • Entire supply chain works off one central demand forecast

  • Customer care

    • Exchange of technical information

    • Software upgrades (90% of software upgrades via internet)

    • On-line support ( 70% of support on-line)

    • On-line, assisted trouble-shooting


Cisco2

Cisco

  • Communications and collaboration

    • Sales and technical training

    • Virtual classrooms

    • Company-wide meetings and broadcasts


Cisco commerce server architecture

Commerce

Server

Oracle

DBMS

Cisco Commerce Server Architecture

Client Tier

Browser

Browser

HTTP

HTTP

Physical

Middle Tier

WEB Server

Oracle

DBMS

Database

Tier

Oracle

Applications


Cisco commerce server

Cisco Commerce Server

  • Typical three-tier architecture

  • Proprietary web server

    • Performs content aggregation

    • Encryption

    • Accesses oracle DBMS

    • Runs on a dedicated SUN server

  • Proprietary commerce server

    • Proprietary application server

    • Performs variety of commerce functions


Cisco commerce server1

Cisco Commerce Server

  • Scalability and availability

    • Big servers for scalability

    • Multiple commerce server processes for load balancing

    • Databases replicated

    • Hot standby for availability


Case studies observations

Case Studies: Observations

  • Database is being used mostly for storage

  • Application in the middle-tier

  • Middle-tier also provides:

    • scalability

    • load balancing

    • large number of users


Analyzing internet applications

Analyzing Internet Applications

  • Web integration

  • Web publishing

  • Application integration

  • E-commerce


Web integration

WEB Integration

  • Heterogeneous data sources

  • Heterogeneous data types

  • 1000s of data sources

  • Dynamic data

  • Warehousing


Web publishing

Web Publishing

  • Problem: internet placing new requirements on content management

    • Heterogeneity: access different types of content from browsers e.g. Email, data warehouses, reports, HTML files

    • Personalized: structured, dynamic, customized content

    • Transactive: content blending with application

    • Aggregation: portalization via major “gateways”


Application integration

Application Integration

  • Integrating Multiple Applications (e.g. ERP/Front Office)

    • Application workflow specification

  • Asynchronous communication

    • Queuing and propagation

  • Message tracking

  • Message warehouse (persistence)

    • Message broker/server

  • Data transformation

    • Transforming messages to different application formats (e.g. SAP, CLARIFY, …I


Electronics commerce

Electronics Commerce

  • Automating business-to-business, business-to-consumer interactions

    • Selling and buying

      • Order management

      • Product catalogs

      • Product configuration

    • Sales and marketing

    • Education and training

    • Service

    • Communities


Database technology uses

Database Technology Uses

  • Business/workflow transactions

    • Support across multiple database/ERP systems

    • Transactional

    • Tools to generate compensating actions

    • Transformations

  • Queuing

    • Support for heterogeneous messages

    • Transactional

    • Querying, e.g. On attribute, value pairs

    • Indexing, e.g. On attribute, value pairs

    • Publish/subscribe


Database technology uses1

Database Technology Uses

  • Rule engines

    • Complex business processing rules

    • Customization/profiling rules

      • Business domain rules

      • Presentation rules

  • Repositories for Application Development

    • Managing Java objects, interfaces, etc.

    • Must for application integration

    • Standardized object models and protocols

    • Directories vs repositories


Database technology uses2

Database Technology Uses

  • XML support

    • XML schema/storage

    • XML caching

    • XML querying

    • Coexistence with SQL -- current efforts seem disjoint

  • Multiple caches

    • Consistency of middle-tier and database caches

  • Data mining

    • Algorithms need to become more pragmatic


Database technology uses3

Database Technology Uses

  • Internet user expectations

    • Speed vs correctness

      • (e.g. Search engines vs blade/cartridge/extender)

    • Availability vs correctness

  • Component Architecture

    • Caching

    • XML support

    • Querying

    • Transactions

    • Rule engines

    • Metadata management

    • Queueing


Database technology uses4

Database Technology Uses

  • Availability

    • Need near 100% availability

    • Must be easy to manage

    • Replication, hot standby, foolproof system?

  • Scalability

    • Number of users is orders of magnitude higher

  • Security

    • Global users

    • Managing millions of users

    • Encryption

    • Performance


Internet applications architecture future

XML

documents

on the Web

Internet Applications Architecture: Future

Client Tier

XML enabled

tools:

Browser

Browser

authoring tools

etc.

XML

XML

Logical Middle Tier

WEB/APP Server

XML enabled

Application Messages

XML Integration &

Query Server;

XML Database

Warehouse Server

XML

XML

XML

XML Transformer

& Gateway

Data Sources

XML enabled

Other

documents

OLE/DB

ORDBMS

on the Web

Data source

e.g. HTML,

WORD


Xml in the database

XML in the Database

  • XML has the potential to impact four important markets

    • Web integration

    • Web publishing

    • Application integration

    • Electronic commerce

      Xml-enable the DBMS


Xml enabled dbms

Xml-enabled DBMS

  • “Xml-enable” the database system

    • Store XML data/documents the database server

    • Querying and searching of structured and unstructured XML

    • In generate XML data from the database server

    • Add XML capabilities in supporting database facilities

DBMS

Integrate with

other facilities

Generate

XML

Store XML


Store xml data

Store XML Data

  • Enhance XML storage facilities in the database with support in utilities

    • Facilities to load XML data into the database

    • Provide more efficient database storage (componentized storage, compression, indexing,…)

    • XML export facilities from the server


Search and query xml data

Search and Query XML Data

  • Search XML data efficiently

    • Special SQL queries over structured + unstructured XML

    • Content-based indexing (e.g. Text indexes) for searching XML data efficiently

    • Support for XML query languages (e.g. XQL) on XML data


Generate xml

Generate XML

  • Generate XML from the database server

    • Map SQL92, SQL3 and PL/SQL datatypes to XML

    • Provide mappings between java, SQL and XML types

  • Script XML content from the database

    • Allow SQL queries to return XML results

    • Provide embedded XML in stored procedures

    • Java scripting: support embedded XML in java

    • Common apis to access any XML content in databases


Xml and supporting facilities

XML and Supporting Facilities

  • Provide XML capabilities in supporting database facilities

    • Support XML in database utilities - loader, export/import ..

    • Allow server-to-server replication of XML data

    • Fine grained access to XML documents


Xml caching

XML Caching

  • Need to temporarily cache it, index it, update the cached copy, transact it

  • Need to query XML caches

  • Also requires a store for managing it in the middle-tier

  • Provides XML logical views


Dbms architecture for internet applications

DBMS Architecture for Internet Applications

  • Monolithic architecture

    • Enhance the DBMS with all the features necessary for supporting internet applications

  • Component architecture

    • Provide components for supporting internet applications

    • Components can reside in the DBMS or in the middle-tier


Monolithic approach

Monolithic Approach

+ Database is the platform

+ Leverage DBMS infrastructure

+ Uniform management

- Not flexible

- Forces 2-tier architecture

- May not be suitable for high-end configurations

- Not suitable for heterogeneous application integration


Component approach

Component Approach

+ Flexible

+ Accommodates multi-tier architecture - components can be deployed in the middle or database tier

+ Facilitates heterogeneous integration of applications

- Need to manage multiple components


Looking ahead

Looking Ahead

  • Database Technology has lot to offer for building internet applications!

  • Componentized Databases?


  • Login