Containment of Conjunctive Queries on Annotated Relations TJ Green University of Pennsylvania

1. Containment of Conjunctive Queries on Annotated RelationsTJ GreenUniversity of Pennsylvania Symposium on Database Provenance University of Edinburgh May 21, 2008

2. Provenance and Query Optimization • Many kinds of semiring-based provenance annotations to choose from: • lineage • why-provenance • minimal witness why-provenance • provenance polynomials • ... • These seem to keep track of more/less information • A fundamental question: how does this affect query optimization?

3. Conjunctive Queries on K-Relations • Datalog-style syntax for conjunctive queries (CQs): Q(x,y) :- R(x,z), R(z,y) • Semantics of applying the CQ to a K-relation R : D£DK: Q(a,b) = z2D R(a,z)¢R(z,b) • # of repetitions of an atom in the body matters • For unions of conjunctive quereis (UCQs) (equivalent to positive RA), sum over CQs: P(x,y) :- R(x,z), R(z,y) P(x,y) :- R(x,w), R(y,w) • Semantics of UCQ applied to R ― a sum over CQs: P(a,b) = z2D R(a,z)¢R(z,b) + w2D R(a,w)¢R(b,w)

4. Choice of K Affects Query Optimization K = N (bag semantics) differs from K = B (set semantics) e.g., the conjunctive queries Q1(x) :- R(x,y), R(x,z) Q2(u) :- R(u,v) are set-equivalent, but not bag-equivalent

5. Our Contributions • We make a systematic study of query containment and query equivalence for various provenance models • We show that K-containment and K-equivalence of CQs and UCQs are decidable for lineage, why-provenace, and the provenance polynomials N[X], as well as a new model, B[X] • The decision procedures are based on interesting variations of containment mappings • We analyze the complexity in each case

6. Our Contributions • As a corollary of the decidability result for N[X]-equivalence of UCQs, we also fill in a gap in the chart for bag semantics:

7. K-Containment for Queries • For semiring K, define a·Kb,9c . a + c = b. If ·K is a partial order, it is called the natural order, and K is said to be naturally-ordered • B, N, lineage, why-provenance, B[X], and N[X] are all naturally-ordered • We define K-containment using the natural order: Q1vKQ2 , 8I8tQ1(I)(t) ·K Q2(I)(t) Q1´KQ2, 8I8tQ1(I)(t) = Q2(I)(t)

8. A Hierarchy of Semiring Provenance (1) • Provenance polynomials (N[X], +, ¢, 0, 1) – tracks calculations abstractly; most general e.g., 2p2r + 3ps + ps3 • Drop coefficients to get (B[X], +, ¢, 0, 1) p2r + ps + ps3 • Drop exponents to get why-prov. (P(P(X)), [, d, ;, {;}) {{p,r},{p,s}} • Flatten set-of-sets to get lineage (P(X), +, ¢, ?, ;) {p,r,s} • Drop, flatten, etc. correspond to surjective semiring homomorphisms

9. A Hierarchy of Semiring Provenance (2) • Suppose h : K1K2 is a semiring homomorphism. Then a·K1b implies h(a) ·K2h(b).If h is also surjective, then h(a) ·K2h(b) implies a·K1b. • Definition:K1¹ K2 means PvK2Q implies PvK1Q • Proposition: for any positive K B¹K¹N[X] (All those we consider are positive.) Moreover: • Proposition (Provenance Hierarchy): B¹ lineage ¹ Why-Prov. ¹B[X] ¹N[X]

10. Containment Mappings • Acontainment mapping from CQ Q to CQ P is a function h : Vars(Q)  Vars(P) such that • head of Q is mapped to head of P • every atom in body of Q is mapped to an atom in body of P • Theorem [CM77]: For CQs P,Q we have PvBQ iff there is a containment mapping from Q to P • e.g. Q1(x) :- R(x,y), R(x,z) Q2(u) :- R(u,v) • h which sends ux and vy is a containment mapping • Checking for existence of containment mapping is NP-complete

11. Canonical Databases • Take body of CQ, “freeze” into database instance [CM77], and tag each tuple with a “tuple id” • We’ll denote by canK(Q) the canonical database for Q with abstract tags from K • e.g., Q(w) :- R(u,v), R(v,w) canN[X](Q) = canB[X](Q) = R canlin(Q) = R canwhy(Q) = R

12. Lineage-Containment of CQs • Covering set of containment mappings: for every atom A in the body of P there is a containment mapping h : QP with A in the image of h • Theorem: For CQs P, Q the following are equivalent: • PvlinQ • P(canlin(P)) µlinQ(canlin(P)) • there is a covering set of containment mappings from Q to P • Note: covering sets of containment mappings were identified in [CV 93] as a necessary (but not sufficient) condition for bag-containment of CQs

13. Why-Containment of CQs • A containment mapping is onto if it induces a surjection on atoms • Theorem: For CQs P, Q the following are equivalent: • PvwhyQ • P(canwhy(P)) µwhyQ(canwhy (P)) • there is an onto containment mapping h : QP • Note: onto containment mappings were identified in [CV 93] as a sufficient (but not necessary) condition for bag-containment of CQs

14. B[X], N[X]-containment of CQs • A containment mapping is exact if it induces a bijection on atoms • Theorem: For CQs P, Q and for K 2 {B[X], N[X]} the following are equivalent • PvKQ • P(canK(P)) µKQ(canK(P)) • there is an exact containment mapping h : QP • Another way to think of exact containment mappings: by unifying variables in Q, you get a query isomorphic to P

15. So Far • K-containment of CQs is decidable for all the provenance models in the hierarchy • Next, we indicate which steps in the hierarchy are strict, and which collapse: BÁ lineage Á Why-Prov. ÁB[X] ¼N[X]

16. Separating the Models for v of CQs • BÁ lineage: Q1(x,y) :- R(x,y), R(x,z) Q2(x,y) :- R(x,y) Q1 vBQ2 but Q1vlin Q2 • lineage Á why: Q1(x) :- R(x,y), R(x,z) Q2(x) :- R(x,y) Q1 vlin Q2 but Q1vwhy Q2 • why ÁB[X]: Q1(x,y) :- R(x,y) Q2(x,y) :- R(x,y), R(x,z) Q1 vwhy Q2 but Q1vB[X]Q2

17. From Containment to Equivalence • {Onto|exact} containment mappings in both directions implies CQs are isomorphic, so why-provenance, B[X], and N[X] collapse to: P´whyQ, P´B[X]Q,P´N[X]Q, PQ • In contrast, for lineage, having sets of covering containment mappings in both directions does not imply isomorphism (but still decidable)

18. From CQs to UCQs • For idempotent semirings (where + is idempotent) this is easy. B, PosBool(B), lineage, why-provenance, and B[X] are idempotent; N[X] is not (omitted) • Proposition [after SY80]: If K is idempotent, then for UCQs P, Q we have PvKQ iff for every CQ P in P there is a CQ Q in Q such that PvK Q • Corollary: For idempotent K, the problems of checking K-equivalence of CQs and K-equivalence of UCQs are polynomially equivalent

19. N[X]- and Bag-Equivalence of UCQs • As with CQs, N[X]-equivalence of UCQs turns out to be the same as isomorphism: Theorem: For UCQs P, Q, P´N[X]Q iff PQ • But, it turns out that N[X]-equivalence and N-equivalence of UCQs are intimately related: Theorem: for UCQs P, Q, P´N[X]Q iff P´NQ Thus: Corollary: for UCQs P, QP´NQ iff P Q

20. Summary: Complexity Results • Theorem: checking for {covering set of|onto|exact} containment mappings is NP-complete • Checking for query isomorphism: believed >P, <NP

21. Summary: Provenance Hierarchy

22. Related Work • Already mentioned • Set-cont. and equiv. of CQs [Chandra&Merlin 77] • Set-cont. and equiv. of UCQs [Sagiv&Yannakakis 80] • Bag-cont. of UCQs [Ioannidis&Ramakrishnan 95] • Bag-equiv. of CQs [Chaudhuri&Vardi 93] • Containment of CQs with where-provenance [Tan 03] • Bag-set semantics [CV 93], combined semantics [Cohen 06] • For K-relations: support operator of [Geerts&Poggi 08] generalizes duplicate elimination • Bag-containment of CQs [Jayram+ 06]

23. Future Work • Loose ends: • Lower bound for N[X]-containment of UCQs (we gave only a PSPACE upper bound) • Generalize results for specific semirings to semirings with certain properties? • Beyond UCQs: Datalog • is K-containment of Datalog programs the same as set-containment when K is a distributive lattice? • is bag-equivalence/N[X]-equivalence undecidable for Datalog? • Could semiring framework give any insight into bag-containment of CQs? • Query optimization for annotated XML