slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
June 10 th , 2003 Presentation by Jeong-Hun Shin PowerPoint Presentation
Download Presentation
June 10 th , 2003 Presentation by Jeong-Hun Shin

Loading in 2 Seconds...

play fullscreen
1 / 20

June 10 th , 2003 Presentation by Jeong-Hun Shin - PowerPoint PPT Presentation


  • 109 Views
  • Uploaded on

Databases and the Grid by Paul Watson University of Newcastle Grid Computing: Making the Global Infrastructure a Reality. June 10 th , 2003 Presentation by Jeong-Hun Shin. Contact. Contact (1997) directed by Robert Zemeckis story by Carl Sagan. Ancestors said that.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'June 10 th , 2003 Presentation by Jeong-Hun Shin' - ophrah


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Databases and the GridbyPaul WatsonUniversity of NewcastleGrid Computing: Making the Global Infrastructure a Reality

June 10th, 2003Presentation byJeong-Hun Shin

contact
Contact

Contact (1997)

directed by Robert Zemeckis

story byCarl Sagan

folding@home to understand protein folding protein aggregation and related diseases
Folding@hometo understand protein folding, protein aggregation, and related diseases
contents
Contents
  • Integration of databases into the Grid
  • Database requirements of Grid applications
    • Requirements above existing systems
  • The Grid and databases: the current state
  • Integrating databases across the Grid
  • Federating database systems across the Grid
integration of databases into the grid
Integration of databases into the Grid
  • The Grid
    • Publication of data in a more open manner
      • New results from separate, distributed sources
    • If the Grid is to support a wider range of applications
      • e.g., applications in the life/earth sciences, business applications
  • Difficulties in integrating databases into the Grid
    • Two main dimensions of complexity
      • Differences between server products within a database paradigm
      • Variety of database paradigms
    • Tradeoff: generic middleware for federating Grid-enabled DBs
      • Full functionality of different database paradigms
      • Common solutions to reduce effort
how can db s be integrated into the grid
How can DB’s be integrated into the Grid?
  • How to integrate existing DBMS’s into the Grid?
    • Short-term solution: holding limitations
    • cf. Long-term solution
      • Developments to the Grid middleware and DB server components
  • Three main questions
    • What are the requirements of Grid-enabled databases?
    • How far do existing Grid middleware and database servers go towards meeting these requirements?
    • How might the requirements be more fully met?
db requirements of grid applications
DB requirements of Grid applications
  • Prerequisite
    • Requires the functionality provided by current DBMS
      • query, update, indexing, transaction, recovery, replication, security, …
    • Grid-enabled DBMS from scratch is not desirable.
      • New facilities are added by enhancing existing DBMS.
      • The most commonly used DBMS are commercial: not open-source

 Enhancement by external wrapping of the DBMS

  • Two categories of requirements
    • Generic across all Grid application components
    • Database specific
requirements above existing systems
Requirements above existing systems
  • Scalability
    • Extremely demanding performance and capacity
    • Low response times for complex queries
    • Support for high access throughput
  • Handling unpredictable usage
    • Difficulty in predicting the types of accesses
    • Current DBMS:
      • Little support for controlling the sharing of finite resources
requirements cont d
Requirements (cont’d)
  • Metadata-driven access
    • Current use: relatively simple

As the Grid expands into new applications:

      • more sophisticated metadata systems and tools  Semantic Grid
    • Two-step access to data
      • Search of metadata catalogs to locate the DB’s  Data access
    • Need for a standardized interface for all DBS
  • Multiple database federation
    • Open publication of data

 Advances in applications combining info from multiple data sets

    • Federation middleware w/ standardized interface
    • Higher-level problem of the semantic integration of multiple DB’s
the grid and databases the current state
The Grid and databases: the current state
  • Globus
    • The dominant middleware used for building computational grids
      • Monitoring and Discovery Service (MDS): Grid information service
      • Globus Resource Allocation Manager (GRAM): resource management
      • Grid Security Infrastructure (GSI)
    • Limitations and possibilities
      • No direct support for database integration
      • GSI can provide a single sign-on capability
      • GridFTP can be used for

bulk database loading/bulk transfer of query results

      • MDS and GRAM can be used to locate and run DB federation middleware
integrating databases into the grid

Metadata

DBS

Query

Transaction

DBMS

Notification

Bulk loading

Scheduling

Accounting

Integrating databases into the Grid
  • Service-based framework
    • Individual operations offered by the services would be standardized
    • Standardization would be done by adding wrapper code to mapthe service operation interface to the vendor specific interface
    • Advantage: each DBS can provide a metadata service
      • Information on the range of services and operations
  • DBS with a Grid-enabled service interface

Client

Service Interfaceonto the Grid

Services

Interface code

roles of service wrapper
Roles of service wrapper
  • Metadata
    • Access to technical metadata about the DBS and the set of services
      • e.g., logical/physical name of the DBS and contents, ownership, ver., …
  • Query
    • Various DBMS  definition of type and level of query language
  • Transaction
    • Transactions involving only a single DBS
    • Allows a DBS to participate in app-wide distributed transactions
  • Bulk loading
    • For large amounts of data: optimized for transfer of large datasets
roles of service wrapper cont d
Roles of service wrapper (cont’d)
  • Notification
    • Allows clients to register some interest in a set of data
    • Receives a message when a change occurred
  • Scheduling
    • Allows users to schedule the use of the DBS
  • Accounting
    • Information for accounting and payment scheme
      • Monitors performance against agreed service levels
      • Enables users to be charged for resource usage
federating dbs across the grid

Metadata

Metadata

DBS

DBS

Query

Query

Transaction

Transaction

Notification

Notification

Bulk loading

Bulk loading

Scheduling

Scheduling

Accounting

Accounting

Federating DBS across the Grid
  • Grid application interfacing directly to a set of DBS
    • Great application complexity
    • Duplication of effort

Client

federating dbs across the grid cont d

Metadata

Metadata

DBS

DBS

Query

Query

Transaction

Transaction

Notification

Notification

Bulk loading

Bulk loading

Scheduling

Scheduling

Accounting

Accounting

Metadata

Query

Transaction

Notification

Bulk loading

Scheduling

Accounting

Federating DBS across the Grid (cont’d)
  • Virtual database system on the Grid
    • Reduces to federating each of the individual services
    • Same interface as the DBS w/ Grid-enabled service interface
      • Possibility for federating services of both “real” DBS and Virtual DBS

Client

Service Interfaceonto the Grid

Service Federation Middleware

Virtual DBS

creation of virtual dbs
Creation of Virtual DBS
  • Types of the creation of a Virtual DBS
    • A user decides to create a Virtual DBS
    • Services take a set of DBS and create a Virtual DBS
  • Challenge
    • Full standardization of all services is impossible
      • The resulting heterogeneity causes problems
    • Automatic creation of a Virtual DBS
      • The tool queries the metadata service their functionality and interface
      • Integration of the service is impossible if no options are available
  • Service federation middleware
    • Complexity varies from service to service
      • In general, increases along the degree of heterogeneity of the service
summary
Summary
  • A set of requirements for Grid databases
    • Existing Grid middleware does not meet them
  • A set of services should be offered by a Grid-integrated DBS
    • Service-based approach
      • Independent of any particular implementation technology
      • Simplifies the task of writing applicationsthat need to combine information from more than one DBS
    • Virtual DBS
      • Federating DBS across the Grid