xml indexing structure
Download
Skip this Video
Download Presentation
XML Indexing Structure

Loading in 2 Seconds...

play fullscreen
1 / 41

xml indexing structure - PowerPoint PPT Presentation


  • 378 Views
  • Uploaded on

XML Indexing Structure. by XSoumia Elghani & XHanaa Talei CSC5370. Table of Content. Introduction Motivation Full Text Indexing Graphs Natix Sphinx Lore System Index Fabric. Introduction. Motivation . Web . Billion of documents. Finding a document become impossible.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'xml indexing structure' - Antony


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
xml indexing structure

XML Indexing Structure

by XSoumia Elghani

& XHanaa Talei

CSC5370

table of content
Table of Content
  • Introduction
  • Motivation
  • Full Text Indexing
  • Graphs
  • Natix
  • Sphinx
  • Lore System
  • Index Fabric
motivation
Motivation

Web

Billion of documents

Finding a document become impossible

Need of efficient indexing techniques

full text indexing
Full Text Indexing

A full text provides standard retrieval of all text objects.

  • B+ tree.
  • Inverted list.
b tree
B+ Tree
  • It is the most widely used of several index structures that maintain their efficiency.
  • B+ Tree is a dynamic structure
    • Insertions and deletions leave tree height balanced
    • Almost always better than maintaining a sorted file
    • B+ tree is also based on rotation
    • Most widely used index in Data base mangement systems
slide7
HIW?

Original:

Insert 28:

slide8
Cont…

Insert 70:

Insert 95:

inverted list
Inverted List

They store data from the database as keys sodata content can be quickly searched on.

graphs
Graphs
  • As we can represent data as tree, we can represent it as a graph.
more details
More details

Employees

Programmers

Statisticians



Leads

Workson

Consults



Projects

problem solution
Problem, solution
  • P: Many links need to be reduced
  • S: An index graph a reduced graph that will summarizes all the paths from the root.

! Important

Language Equivalent

Project

Employee.leads

Employee.workson

Programmer.employee.leads

Programmer.employee.workson

The same thing apply to p2

implementing an index
Implementing an index??
  • Each node is a hash table containing one entry for each label at that node. Each index node has an extent: a list of pointers to all data nodes in the corresponding class.

i.e: the extent of the node h4 is the list [e1, e2]

We compute the query on the index and obtain a set of index nodes; and then we compute the union of all extents.

example
Example
  • Select x from statistician.employee.(leads|consults):x
  • This query will returns the nodes h8,h9; their extents are [p5,p6,p7] and [p8] then the result of our query is the union

Results:

Simplified form of DAG

Efficient way when it can be stored in main memory

natix
Natix
  • An efficient, native repository for storing, retrieving and managing tree structured large objects, preferably XML documents
  • It is based on split algorithm
  • Dynamically maintains physical records of size smaller than a page which contain sets of connected tree nodes.
  • It is similar to the hybrid system , but with some extensions
natix architecture
Natix Architecture
  • Record Manager: provided memory spaces divided into segments (collection of equal size pages) and each page holds one or more records.
  • Tree storage manager: operate on top of RM; it maps the tree used to model the document(topic)
slide18
Cont..
  • Index management
  • Query engine
  • Schema manager, take care of the DTD
  • Document manager (validate the schema), make the necessary index update..

But they are not implemented yet

physical model
Physical Model

In order to store our logical tree, there are two important ways to classify the physical node:

object content

  • Large tree
1 object content
1. Object content
  • The classification is based on the content of the node:
  • Aggregate: inner nodes of the tree; they contain their respective child nodes.
  • Literal:leaf nodes containing stream of bytes
  • Proxy: nodes which point to different records (thery are used in the representation of large trees.)
slide21

Large Trees

Large trees are split into subtrees, and then store each subree in a single record

Scaffolding

Objects

sphinx
Sphinx
  • Schema-conscious Path-Hierarchy Indexing of Xml.
  • Uses DTD to speed up the search process.
  • XML document Document Graph.
  • DTD  Schema Graph.
lore system
Lore System
  • DBMS designed for semistructured data
  • Uses OEM graph, a label directed graph.
  • Vertices are objects
  • Each object has a unique object identifier (e.g. &19)
indexes in lore
Indexes in Lore
  • To indentify objects with specific values:
    • Value Index
    • Text Index
  • To traverse DB graph:
    • Link Index
    • Path Index
value index vindex
Value Index (Vindex)
  • Implemented as B+trees
  • Takes a label ‘l’, a comparator ‘c’, and a value ‘v’
  • Returns all atomic objects having:
    • an incoming edge with the given label
    • a value satisfying the given operator and value
  • e.g. l=Price c=‘>’ v= 15.00

result= {&11, &15}.

text index tindex
Text Index (Tindex)
  • Implemented using inverted lists.
  • Maps a given word ‘w’ and label ‘l’ to a list of atomic values with incoming edge ‘l’ that contain word ‘w’.
  • Label can be omitted for a full search.
  • Returns a list of postings (o,n) indicating that ‘w’ appears in object ‘o’ as the nth word in the value.
  • e.g. w=“Ford” l= Name

result = {(&17,2),(&21,2)}

link index lindex
Link Index (Lindex)
  • Implemented using linear hashing
  • Used to retrieve the parents of an object
  • Takes a child object ‘c’ and a label ‘l’
  • Returns all parents ‘p’ such that there is an l-labeled edge from p to c.
  • If the label is omitted, lindex returns all parents and their labels
  • Useful because there are no inverse pointers in OEM graphs.
path index pindex
Path Index (Pindex)
  • Takes a given object ‘o’ (e.g. root) and a path ‘p’
  • Returns the set of objects reachable from ‘o’ following path ‘p’.
  • e.g. “select DB.Movie.Title”

result = {&5,&9,&14}

index fabric
Index Fabric
  • Optimizes searches over semi-structured databases
  • Based on Patricia tries
  • Assigns a designator to each tag in the XML document.
  • To interpret the designators a designator dictionary is used
patricia tries
Practical Algorithm to Retrieve Information Coded in Alphanumeric

Nodes are labelled with their depth

Patricia Tries
conclusion
Conclusion
  • A number of indexing techniques
  • Different approaches
  • Under construction (e.g. Natix)
  • Still developing and improving
references
References
  • Graphs: S. Abiteboul, P. Buneman, D. Suciu, “Data on the Web: from relations to semistructured data and XML”, Morgan Kuafman, 2000.
  • Natix:C.C Kanne, Guido Moerkotte. “Efficient storage of xml data“. Proc. of ICDE, California, USA, page 198, 2000.http://citeseer.nj.nec.com/kanne99efficient.html  .
  • Sphinx: L. K. Poola and J. R. Haritsa. "SphinX: Schema-conscious XML Indexing", Indian Institute of Science, 2001. http://citeseer.nj.nec.com/poola01sphinx.html
references40
References
  • Lore: J. McHugh, J. Widom, S. Abiteboul, Q. Luo, and A. Rajamaran. “Indexing semistructured data “. Technical report, Stanford University, Computer Science Department, 1998.http://citeseer.nj.nec.com/mchugh98indexing.html.
  • Index Fabric: B. Cooper, N. Sample, M. J. Franklin, G. R. Hjaltason, and M. Shadmon. “A fast index for semistructured data”. In Proceedings of VLDB, 2001. http://citeseer.nj.nec.com/cooper01fast.html.
ad