1 / 31

Completeness of Queries over Incomplete Databases

Explore the concept of data completeness and the challenges of query answering over incomplete databases. Learn about query completeness and its reasoning, table completeness, and different semantics for aggregate queries.

tblanchard
Download Presentation

Completeness of Queries over Incomplete Databases

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CompletenessofQueriesoverIncomplete DatabasesSimon Razniewski Joint work with Werner Nutt Free University of Bozen-Bolzano

  2. Introduction • Data completeness: important aspect of data quality • Query answering over incomplete data: extensively studied • Query Completeness: little work Completeness of Queries over Incomplete Databases

  3. Bolzano is in the Province of South Tyrol Autonomous, trilingual province in the north of Italy Bolzano Completeness of Queries over Incomplete Databases

  4. School Data in South Tyrol Decentrally maintained database Statistical reports ?? notoriouslyincompletecorrectnessimportant Completeness of Queries over Incomplete Databases

  5. Example Database Schema • Pupil(pname, age, sname) • School(sname, type, language) Completeness of Queries over Incomplete Databases

  6. Completeness Reasoning Example Supposewehavedataaboutpupilsfrom all • German schools • Italianschools, exceptthe high school “Da Vinci“ • Ladin schools, exceptthemiddleschool “Gherdëna“ Will thefollowingqueryget a correctanswer? “HowmanypupilsareatGerman primaryschools?“ • Yes • (ifwe also have all German primaryschools) Completeness of Queries over Incomplete Databases

  7. Completeness Reasoning Example (Cntd) Supposewehavedataaboutpupilsfrom all • German schools • Italianschools, exceptthe high school “Da Vinci“ • Ladin schools, exceptthemiddleschool “Gherdëna“ Will thefollowingqueryget a correctanswer? “Howmany Ladin pupilsarethere? • Maybe not, pupilsfrom “Gherdëna“ couldbemissing Completeness of Queries over Incomplete Databases

  8. Overview • Formalization • IncompleteDatabase • Query Completeness • Table Completeness • ReasoningforConjunctiveQueries • BagSemantics • Set Semantics • Aggregate Queries Completeness of Queries over Incomplete Databases

  9. Incomplete Database (Motro 1989) Incompletenessneeds a completereference Incompletedatabasesarepairsof an ideal databaseDiand an availabledatabaseDa D = (Di, Da) such that DaDi Completeness of Queries over Incomplete Databases

  10. Incomplete Database - Example D = (Di, Da) “Paul and Andrea are pupils in the ideal database” Di = { pupil(‘Paul‘, 11, ‘Da Vinci‘), pupil(‘Andrea‘, 14, ‘Gherdëna‘) } “Ouravailabledatabasemissesthefactthat Andrea is a pupil“ Da = { pupil(‘Paul‘, 11, ‘Da Vinci‘) } Completeness of Queries over Incomplete Databases

  11. Query Completeness (Motro 1989) Query Q “The setofanswersto Q iscomplete“ Notation: Compl(Q) Semantics: (Di, Da) Compl(Q) iff Q(Di) = Q(Da) Completeness of Queries over Incomplete Databases

  12. Table Completeness (Levy 1996) Table pupil(pname,age,sname) “Our available db contains all pupils from Ladin schools” Formally: “If (p, a, s) is a Ladin pupil according to the ideal db, then (p, a, s) is a pupil in the available db” This is a full TGD (= tuple generating dependency) Completeness of Queries over Incomplete Databases

  13. Table Completeness (Cntd) “Our available db contains all pupils from Ladin schools” TGD: c Notation: Compl(pupil(p, a, s); school(s, t, ‘Ladin’) Semantics: (Di, Da) Compl(pupil(p,a,s); school(s, t, ‘Ladin‘)) iff (Di, Da)c Completeness of Queries over Incomplete Databases

  14. CompletenessReasoning Wehavecompletedataaboutpupilsfrom all • German schools • Italianschools, exceptthe high school “Da Vinci“ • Ladin schools, exceptthemiddleschool “Gherdëna“ Query “HowmanypupilsareatGerman primaryschools? TC-QC entailment CCompl(Q) ? TC Statements C QC Statement Compl(Q) Completeness of Queries over Incomplete Databases

  15. CompletenessReasoning (Cntd) • TC-QC: tablecompletenessentailsquerycompleteness Compl(R1; G1), …, Compl(Rn; Gn) Compl(Q) - bagsemanticsComplbag(Q) - setsemanticsComplset(Q) • QC-QC:querycompletenessentailsquerycompleteness Compl(Q1), …, Compl(Qn) Compl(Q) • TC-TC: tablecompletenessentailstablecompleteness Compl(R1; G1), …, Compl(Rn; Gn) Compl(R; G) Completeness of Queries over Incomplete Databases

  16. WhatisKnown? • Characterizing QC-QC entailment: Compl(Q1), …, Compl(Qn) Compl(Q) • Existence of a rewriting is a sufficient condition (Motro 1989) • Deciding TC-QC entailment: Compl(R1; G1), …, Compl(Rn; Gn) Compl(Q) • Decision procedure for trivial cases (Levy 1996) • For reasoning w.r.t. a concrete database instance, data complexity is coNP-complete for first-order queries and TC statements (Denecker et al. 2007) Completeness of Queries over Incomplete Databases

  17. TC-QCbag – Canonical TC Statements “How many 12-year old pupils are at the Italian schools?'' Q(COUNT(p)) :− pupil(p, 12, s), school(s, t, ‘Italian')‏ Q can be answered correctly if - every 12-year old pupil from an Italian school is there - every Italian school with a 12-year old pupil is there That is, if the database satisfies - Compl(pupil(p, a, s); school(s, t, ‘Italian'), a = 12)‏ - Compl(school(s, t, l); pupil(p, 12, s), l = ‘Italian')‏ canonical completeness statementsforQ Completeness of Queries over Incomplete Databases

  18. TC-QCbag – Canonical TC Statements (Cntd) Query Q() :− A1(), …, An(n), The canonicaltablecompletenessstatementforatom Aiis Compl(Ai; A1, …, An-1, An+1, …, An) CanQisthesetofcanonicalcompletenessstatementsfor all atomsof Q • Proposition: (Di, Da) CanQimplies(Di, Da) Complbag (Q) Completeness of Queries over Incomplete Databases

  19. TC-QCbagReducesto TC-TC Wesaw: CanQComplbag (Q) (Complset (Q)) • ForanysetCof TC-statements: CComplbag(Q)iffCCanQ Theorem: Complbag(Q)CanQ TC-QC TC-TC Completeness of Queries over Incomplete Databases

  20. TC-TC Entailment = Query Containment C1 = Compl(pupil(n, a, s); True) C2 = Compl(pupil(n, a, s); a = ‘12') Obviously, C1entailsC2 Q1(n) :−pupil(n, a, s) Q2(n) :− pupil(n, a, s), a = ‘12' Q2iscontainedin Q1 C1entailsC2becauseQ2iscontainedin Q1 Completeness of Queries over Incomplete Databases

  21. TC-TC Entailment = Query Containment (Cntd) TC statements describe parts of tables that are complete TC statements entail each other if the parts described are contained • Entailment of TC from TC can naturally be reduced to query containment Theorem: Let L be a class of conjunctive queries that (i) contains for every relation the identity query (ii) is closed under intersection Then TC-TC entailment and containment of unions of queries can be reduced to each other in linear time. Completeness of Queries over Incomplete Databases

  22. Complexity Classes of conjunctive queries: • CQ: Conjunctive queries with comparisons over dense orders • RQ: Relational conjunctive queries (i.e., without comparisons) • LCQ: Linear conjunctive queries (i.e., without self-joins) • LRQ: Linear relational conjunctive queries Completeness of Queries over Incomplete Databases

  23. TC-QCbag - Complexity Completeness of Queries over Incomplete Databases

  24. TC-QCset TC-QCsetis • Containment w.r.t. toTCstatements CQiQaiffCQi Qa (monotonicityof Q) • Containment w.r.t. TGDs CQiQaiffcQi Qa More complexthanTC-TC Completeness of Queries over Incomplete Databases

  25. TC-QCset TC-QCbag - Complexity Completeness of Queries over Incomplete Databases

  26. CompletenessReasoningfor Aggregate Queries • SUMandCOUNT: similartobagsemantics • MINandMAX: similartosetsemantics Completeness of Queries over Incomplete Databases

  27. QC-QC and Query Determinacy Motro’s idea: Look for rewritings Given Q1(x) :− R(x), S(x) Q2(x) :− T(x) Suppose we know Compl(Q1) and Compl(Q2) Consider Q(x) :− R(x), S(x), T(x) We see: Q can be rewritten as Q(x) :− Q1(x), Q2(x) Therefore, weconcludeCompl(Q) Completeness of Queries over Incomplete Databases

  28. QC-QC andQuery Determinacy (Cntd) Queries Q1, …, Qn, Q • Determinacy: Q1, …, Qndetermine Q, written Q1, …, Qn Q, iff • Q1(D) = Q1(D’), …, Qn(D) = Qn(D’) implies Q(D) = Q(D’) • for all pairs of dbsD, D’ • QC-QC Entailment: Compl(Q1), …, Compl(Qn) entails Compl(Q), iff • Q1(Di) = Q1(Da), …, Qn(Di) = Qn(Da) implies Q(Di) = Q(Da) • for all pairs of dbsDi, Da where Da Di • Proposition: Q1, …, Qn Q implies Compl(Q1), …, Compl(Qn)Compl(Q) Completeness of Queries over Incomplete Databases

  29. QC-QC and Query Determinacy (Cntd) However: • Decidability of determinacy for conj. queries is open (Segoufin/Vianu ‘05) • Necessity of determinacy for QC-QC entailment is open Theorem: For boolean queries, existence of rewritings, determinacy and QC-QC entailment coincide Completeness of Queries over Incomplete Databases

  30. Where Can Completeness Statements Come From? Any conclusion only as correct as the statements it is derived from ~> On which basis can someone give a completeness statement? • Someone knows some part of the real world E.g., a class teacher knows all his students • The method of data collection is known to be complete E.g., at the deadline for enrolment all forms must be present • Cardinalities of parts of the real world areknown and the method of data collection is correct E.g., no nonexisting schools are registered and the number of schools in South Tyrol is known Completeness of Queries over Incomplete Databases

  31. Conclusion • Framework formodellingcompleteness • queryanswers (Motro: QC statements) • partsofdatabases (Levy: TC statements) • Reasoning • ComplexityanalysisofTC-TCandTC-QC • ConnectionbetweendeterminacyandQC-QC • Reasoning in thepresenceofinstances • Currentwork • Schema constraints (keys, foreignkeys, finite domains) • Null values • Prototypicalimplementation Completeness of Queries over Incomplete Databases

More Related