107 Views

Download Presentation
## ILP for Mathematical Discovery

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**ILP for Mathematical Discovery**Simon Colton & Stephen Muggleton Computational Bioinformatics Laboratory Imperial College**The Automation of Reasoning**• Aims for the talk • Discuss a new ILP algorithm (ATF) • and its implementation in the HR system • Promote maths as a domain for ILP research Maths Automated Theorem Proving Automated Reasoning Bioinformatics Machine Learning**From Prediction to Description**Predictive tasks Supervised learning Know what you’re looking for Don’t know what you’re looking for Don’t know you’re even looking Descriptive tasks Unsupervised learning**A Partial Characterisation of Learning Tasks**• Concept learning • Outlier/anomaly detection • Clustering • Concept formation • Conjecture making • Theory formation**The HR Program in Overview**• Embodies a novel ILP algorithm • We call this “Automated Theory Formation” (ATF) • Designed for descriptive tasks (in maths) • But has had applications to concept learning tasks • Incrementally builds a theory • Containing association and classification rules • HR has numerous tools for the user • To extract information from the theory generated • Which is relevant to the task at hand**ATF Overview**• Invent new concepts • Derive classification rule from concept • Induce hypotheses relating the concepts • Prove/disprove the relationships • Deductively • Using state of the art ATP/model generators • Extract association rules • From the hypotheses**Input to HR**• Five inputs to HR • Objects of interest (graphs, groups, etc.) • A labelling of the objects • If the task at hand is predictive… • Background predicates (Prolog style) • Axioms relating predicates (ATP style) • Termination conditions • HR works as an any time algorithm • User can supply • numerous different combinations of these } See Paper**Representation of Theory Contents**• Three types of frames • All have a clausal definition slot • Example frame • Concept frame • Slot 1: range-restricted program clause • Slot 2: success set • Slot 3: classification rule afforded by definition • Other slots: measures of value • Hypothesis frame • Slot 1: clauses (association rules) • Slot 2: proof/counterexample • Other slots: details of the concepts related**Cut Down Algorithm Description**• Build new concept definition from old • Using one of 12 generic production rules [PR] (see paper) • Find the success set, S, of new concept • If S is empty, derive non-existence hypothesis, H • Extract association rules from H, try to prove/disprove • If S is a repeat, derive equivalence hypothesis, H • Extract association rules from H, try to prove/disprove • If S is new • Add new concept to theory • Derive classification rule • Derive implication & near-equivalence hypotheses • Extract association rules, try to prove/disprove • Measure concepts in theory**Concept Space Searched**• Space determined by PRs, not language bias • Clausal definition is: • range-restricted, fully typed program clauses • Definition: n-connectedness • Every variable appears in a body predicate with head variable n, or with a n-connected variable • Example: • c(X,Y) :- p(X), q(Y), p(Z), r(X,Y), s(Y,Z) is 1-connected • c(X,Y) :- r(Y), s(X,Z) is not 1-connected • HR’s definitions are all 1-connected**Deriving Classification Rules**• Given definition D • Arity = n, head predicate = p, success set = S • Classifying function over constants, o, is: • Classification, C, afforded by D: • Put two objects in the same class if f(o1) = f(o2) • Theorem: • If a definition D is not 1-connected, then a literal can be removed without changing the classifiction afforded by D • So, HR’s search space is non-redundant with respect to C**Illustrative Example**• concept17(X,Y) :- integer(X), integer(Y), divisor(X,Y), ¬ divisor(Y,2). • S17 = {(1,1), (2,1), (3,1), (3,3), (4,1), (5,1), (5,5), (6,1), (6,3)} • Classifying function: • f17(1)={(1)} f17(2)={(1)} f17(3)={(1),(3)} f17(4)={(1)} • f17(5)={(1),(5)} f17(6)={(1),(3)} • Classification afforded by concept 17: • [ [1,2,4] [3,6] [5] ]**Mathematics Applications**• Two applications given here • Both from external research groups • Data sets available online • See paper for details • Of two more applications**FindingDiscriminants**• Finding discriminants of residue classes • Work with Sorge and Meier • Overall goal: classify algebraic structures • Bottleneck: showing non-isomorphism • Learning task: • Given two multiplication tables • Find a property true of only one • Which doesn’t refer to individual elements • Data set: 817 pairs of tables (size 5, 6, 10)**Results**• HR given 500 steps per task (~22 secs) • Worked with four production rules • Found discriminants • For 791 out of 817 pairs (~97%) • Average of 20 discriminants per pair • 517 distinct discriminants in total • Example above: • Idempotent element (a*a=a) • Appearing once on diagonal • Only one of two discriminants found for pair**Reformulation of CSPs**• Work with Miguel and Walsh • Constraint satisfaction solving • Very powerful general purpose technique • Specifying a problem is still highly skilled • Learning task: • Given solutions to small problems • Find concepts to specialise the problem specification • Find implied constraints to increase efficiency • Data set: QG-quasigroups (5 types) • Multiplication tables up to size 6**Results**• HR ran for an hour for each problem class • Produced on average 150 association rules • And 10 specialisation concepts • In each case, a better reformulation was derived (with human interpretation) • Up to 10 times speed up in some cases • Nice example: QG3: (a*b)*(b*a)=a • These are Anti-abelian, i.e., a*b=b*a a=b • Symmetry relation: a*b=b b*a=a**Some Other Applications**• Concept learning tasks: • Extrapolation of integer sequences: ICML’00 • Mutagenesis regression unfriendly • Anomaly detection task: • Analysis of Bach Chorale melodies (current MSc.) • Conjecture making tasks: • Generating TPTP library theorems: CADE’02 (& paper) • Finding links in the Gene Ontology (current MSc.) • Making Graffiti-style conjectures (current MSc.) • Theory formation task: • Invention of integer sequences: AAAI’00, JIS’01 (& paper)**Conclusions**• Presented the ATF algorithm • Involves induction & deduction • Presented for first time in ILP terminology • Characterised the concept search space • Presented two learning tasks • In mathematics • More in paper (and in previous work) • Shown that HR can make discoveries**Future Work**• Apply HR to bioinformatics • Needs more efficient implementation • Look into the conglomeration of • Creative reasoning techniques • Relate HR to other descriptive programs • CLAUDIEN and WARMR • Can these programs • Do better than HR in maths applications?**A Drosophilia for Descriptive Induction?**• “Something from (nearly) nothing” • Can give HR only 1 concept • Multiplication in number theory • Invents the concept of refactorable nums • Number of divisors is itself a divisor • 1, 2, 8, 9, 12, … • A nice hypothesis it produces is: • Odd refactorable numbers are square