1 / 19

Research topics in data warehouse

Research topics in data warehouse . Directed By : Dr Rahgozar Mostafa h.Chehreghani. List of research topics. Lineage tracing Incremental view maintenance Indexing in data warehouse Data quality. Lineage tracing. List of papers : Using AutoMed Metadata in Data Warehousing Environments

varuna
Download Presentation

Research topics in data warehouse

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Research topics in data warehouse Directed By : Dr Rahgozar Mostafa h.Chehreghani

  2. List of research topics • Lineage tracing • Incremental view maintenance • Indexing in data warehouse • Data quality

  3. Lineage tracing • List of papers : • Using AutoMed Metadata in Data Warehousing Environments • A Tutorial on the IQL Query Language • Practical Lineage Tracing in Data Warehouses • Incremental view maintenance and data lineage tracing in heterogeneous database environments • A Framework for supporting data integration using the materialized and virtual approaches

  4. Lineage tracing • Automed: model for metadata in data warehouse • Use tag for relations • Use a language such as IQL • Node , Edge , Constraint • IOL: • Functional and typed language • Prefix and Infix functions • New functions by lambda • lambda {x,y,z} ((*) ((+) x y) z)

  5. IQL • let v = q1 in q2 • let v = ((+) 200 500) in ((*) v v) • union : R ++ S • duplicate elimination: distinct (R) • setUnion R S Ξdistinct (R ++ S) • difference : R – S • projection : [{x,z} | {x,y,z} <- R] • Cartesian product and Joins • gc agFun xs • map f xs • Grouping and Aggregation Operations

  6. Using IQL in Automed • Example : Enforce unique key constraint: (=) (count (distinct [n | {s,n} <- <<Student,name>>])) (count <<Student>>) • Name : field • Student : table

  7. Example of lineage tracing

  8. Example of lineage tracing • TS1,S2 = addNode (dept,{“Maths”,“CompSci”}); • addNode (person, [x| x mathematician] ++ [x| x compScientist]); • addNode (avgDeptSalary, {avg [s| (m,s)«_, mathematician, salary»]} ++ {avg [s| (c,s)«_, compScientist, salary»]}); • addEdge («_, dept, person», [( “Maths”, x)| x mathematician] ++ [(“CompSci”, x) | x compScientist]); • addEdge («_, person, salary», «_, mathematician,salary» ++ «_, compScientist, salary»); • addEdge («_, dept, avgDeptSalary», {( “Maths”, avg [s| (m,s) «_, mathematician, salary»]),

  9. Example of lineage tracing • (“CompSci”, avg [s| (c,s)«_, compScientist, salary»])}); • delEdge («_, mathematician, salary», [(p, s)| (d, p) «_, dept, person»; (p’, s) «_, person, salary»; d = “Maths”; p = p’]); • delEdge («_, compScientist, salary», [(p, s)| (d, p) «_, dept, person»; (p’, s) «_, person, salary»; d = “CompSci”; p = p’}); • delNode (mathematician, [p| (d, p) «_, dept, person»; d = “Maths”]); • delNode («compScientist», [p| (d, p) «_, dept, person»; d = “CompSci”]);

  10. Incremental view maintenance • List of papers • Incremental view maintenance and data lineage tracing in heterogeneous database environments • View maintenance in a warehousing environment • A System Prototype for Warehouse View Maintenance

  11. Incremental view maintenance • Di : set of base relations • ΔDi : bags inserted into Di • ⌂Di : bags deleted from Di • V : materialized view • ΔV : bags inserted into V • ⌂V : bags deleted from V • Vnew = (V ++ ΔV) -- ⌂V • Minimality condition • ΔV C V • ΔV∩ ⌂V = Ø

  12. Incremental view maintenance

  13. Incremental view maintenance

  14. Indexing in data warehouse • Paper • Bitmap Index Design and Evaluation • Advantages : • Compact size • Efficient hardware support for bitmap operations (AND, OR, XOR, NOT) • Fast search

  15. Bitmap Index

  16. Bitmap Index

  17. Data quality in data warehouse • List of papers • Towards Quality-Oriented Data Warehouse Usage and Evolution • Data Quality Problems and Proactive Data Quality Management in Data-Warehouse-Systems • Data Warehouse Data Policy • Fitness for use • Subjective : • Related to end users • Objective : • Definition of system • Models: • GQM : Goal Question Metric • English

  18. GQM • Goal factor • Importance of each factor determined respect to Goal • Quality dimension : • Data coherence • Data Completeness • Data freshness

  19. GQM

More Related