Complexity of Polynomial Systems

1. Complexity of Polynomial Systems David PritchardWaterloo URA SeminarMay 16, 2007

2. 3 2 1 0 0 ( ) b b 4 1 5 1 0 1 0 3 2 0 0 7 ¡ ¡ + ¡ + x x y y z x a y z x y a = = = = = = = = ; ; ; ; ; ; : Preliminaries • Polynomial equation: any expression using variables, constants, addition/subtraction, multiplication, whole powers, and one “=” • Polynomial system: multiple equations in a common set of variables. E.g., • Solution: a value for each variable, so after substituting, all equations are true. E.g.,

3. Motivation • Polynomial systems come up all over the place. Want general technique to answer: is there a solution? what is it? • Special case, e.g.: quadratic formula. • For this talk “general technique” means “computer program” • Want programs to be fast. Leads us to talk about computational complexity

4. 2 ( ) 3 5 1 3 5 0 + + + ¡ x y x y z x x = = = ; ; Terminology • Important special case of polynomial is linear: no multiplication/powers of variables allowed. E.g., • In high school, learn how to solve 2x2 systems of linear equalities þ ý ý

5. Game Plan • Will discuss equalities and inequalities * or complex

6. C ( ) C I i 2 s z e 1 Crash Course in Computational Complexity • A computer algorithm (program) is polynomial* time (P-time) if its running time on input I is at mostfor some choice of constants C1, C2 • Math: “time” = steps on Turing machine. Us: “time” = ms of (say) Java program. Both notions are the same! * Distinct usage from polynomial system

7. Real Linear Equality Systems • Mentioned 2 var, 2 equation case • General strategy: Gaussian Elimination • solve for one variable x in terms of others • eliminate all occurrences of x • repeat: solve & eliminate another var, etc • Elimination ever gives “0=1”: no sol'n • Else back-substitute at end, get sol'n

8. Gaussian Elimination: Time • Say n vars, m eqns. Input size = n+m • # arithm’c operations to perform one round of substitution: 2nm • Each round elims a var à n rounds • Total operations: 2n2m < 2(input size)3 hence Gaussian elim’n is P-time • (We cheated - coefficient sizes, inter-mediate expression swell! Still P-time)

9. Real Linear Inequality Systems • Related to “linear programs:” maximize f(x, y, …) subject to c1(x, y, …) ≥ 0, c2(x, y, …) ≥ 0,… where f, c1, c2, …, are linear functions • In fact optimizing is no harder than solving; use binary search like solve f(x, y, …) ≥ 10, f(x, y, …) ≤20, c1(x, y, …) ≥ 0, c2(x, y, …) ≥ 0,…

10. = b f l b l i 3 4 1 3 4 0 5 2 2 7 5 ¸ + + + + ¢ ¢ ¢ e e a m o = $ $ $ b f l b l i i i i 3 4 1 + + : : ¢ ¢ ¢ m n m z e e e a m o ( ) b f l b l i 1 1 ¸ t + + s e e a m o . . = ( ) b f l b 3 4 2 ¸ + e e a m ( ) b f l b 2 4 2 3 ¸ + ¢ ¢ e e a m Linear Program Duality • Notice (beef=1/2, lamb=1/4, oil=1/4) gives cost $2.75. Is it optimal? • Yes! Add equations (1)+(2)+0.5*(3): • We can get a P-time solution, tricky

11. Scorecard þ [Khachiyan 1972]

12. Integer Linear Systems • Application: similar to LP food example, but where can't break units (e.g., scheduling employees) • Hard! Even in special case where all variables must be 0 or 1 • Naive algorithm: try all possibilities • n vars: need to try 2ncombinations, not P-time algorithm. Is there one?

13. What is NP-hardness? (NPh) • A problem (not algorithm) can be NPh • Doesn’t mean Non-Polynomial — actually “n” stands for nondeterministic • Yet, accepted evidence that no polynomial algorithm exists for the problem • Proof: reduce another NPh problem to it. Many, many problems are known to be NPh • Why evidence? Despite trying, no known P-time solution to any NPh problem • Suspicion: none exist! (i.e., P != NP)

14. Integer Linear Programming • 0-1 integer linear programming is NPh (Karp, 1972) • Removing “0-1” only makes harder! • Effort still focused on practical solutions; for NPh problems, “real-life” instances may not be the hardest ones • E.g., used to solve traveling salesman problem (NPh) on 100000s of cities • One difficulty: no duality =(

15. Scorecard þ þ [Khachiyan 1972] [Karp 1972]

16. Real/Complex Polynomial Systems • Real vs complex leads to vastly different approaches to solve a polynomial system • Both are NP-hard! Proof… • Reduce to 0-1 integer linear programming: • Suppose we can solve real polynomial systems • Then we could solve 0-1 integer linear programs too: take linear constraints, add constraint x2=x for each variable x • Increases problem size (#constraints, #vars) by only a little bit (a polynomial amount)

17. Systems in C: Gröbner Bases • Simplest Q: does a system have sol’n? • Recall: if we can manipulate system’s equations to get “0=1”, no solution • Basic idea: normalize system so that we can compute remainders (like mod) • Division by a set of polynomials is straightforward but can give multiple answers if set is not a Gröbner basis

18. Sketch of Complex Case 1. Normalize system to a Gröbner basis 2. Compute remainder of 1 in radical 3. If remainder is 0, there’s no solution; else remainder is 1, there’s a solution! • Depends on the fact that complex numbers are algebraically closed • Not as straightforward to numerically compute a solution

19. 4 3 2 2 4 2 3 2 0 ¡ + ¡ + y y y x y x = Reals: Cylindrical Algebraic Decomposition • Idea: polynomial system decomposes Euclidean space into several parts. • Compute parts recursively: use elimination (resultant) to reduce #vars e.g. from [Arnon, Collins, McCallum 1984]

20. 4 3 2 2 4 ( ) f 2 3 2 ¡ + ¡ + x y y y y x y x = ; 1 2 1 0 8 6 ( ) l 2 0 4 8 4 6 0 8 3 7 1 2 t t ¡ + + y - r e s u a n g x x x x x ) = Reals: Cylindrical Algebraic Decomposition ð Decomp’n of R by g: Cylinders ð Decomp’n of R2 by f:

21. Scorecard þ þ [Khachiyan 1972] [Karp 1972] þ [Buchberger 1965; Tarski 1930s]

22. Integer Variables + Polynomials = Diophantine Equations • 1900, David Hilbert gave 23 problems to the international mathematics congress in Paris • Problem 10: “Find an algorithm to determine whether a given polynomial Diophantine equation with integer coefficients has an integer solution” • Exactly what we want to do!

23. (Different Formulations) • Whole number variables vs integer variables? Can write one using other: (whole number) – (whole number) = any integer [obvious] (integer)2 + (integer)2 + (integer)2 + (integer)2 = any whole number [Lagrange’s 4-squares theorem] • Combine multiple eqn’s: {a=b, c=d}  (a-b)2+(c-d)2=0

24. How to Solve Diophantine Eq’ns? • Can’t just try all possible solutions: if none found, can’t tell when to stop looking. • Any other technique works? • No! • Matiyasevich, 1970: No computer program exists that can determine solvability of Diophantine systems

25. Matiyasevich’s Theorem • Main contribution: proved anything a computer can do, a Diophantine sys’m can do • Random e.g.:this system has solutionif and only ifk+2 is prime 0 = wz + h + j - q 0 = (gk + 2g + k + 1)(h + j) + h - z 0 = 16(k + 1)3(k + 2)(n + 1)2 + 1 - f2 0 = 2n + p + q + z - e 0 = e3(e + 2)(a + 1)2 + 1 - o2 0 = (a2 - 1)y2 + 1 – x2 0 = 16r2y4(a2 - 1) + 1 - u2 0 = n + l + v - y 0 = (a2 - 1)l2 + 1 - m2 0 = ai + k + 1 - l - i 0 = ((a + u2(u2 - a))2 - 1)(n + 4dy)2 + 1 - (x + cu)2 0 = p + l(a - n - 1) + b(2an + 2a - n2 - 2n - 2) - m 0 = q + y(a - p - 1) + s(2ap + 2a - p2 - 2p - 2) - x 0 = z + pl(a - p) + t(2ap - p2 - 1) - pm

26. Halting problem: Is there a computer program which takes other computer programs as input, and outputs whether or not they ever stop? • Answer: No! Proof: pretty! [not here](Turing, 1936) • Hence: no program to solve Diophantine systems since (using simulation result) it would solve halting problem

27. Scorecard þ þ [Khachiyan 1972] [Karp 1972] þ ý [Tarski 1930s; Buchberger 1965] [Matiyasevich 1970]

28. Endgame • Many linear program solvers exist and simplex method doable by hand • Many integer LP solvers; can be much slower • Gröbner bases in Maple and… • CAD in Mathematica and… • Too bad for Diophantus and Hilbert =(