Algorithms

1 / 28

# Algorithms - PowerPoint PPT Presentation

Solution. exact. approximate. fast. Speed. slow. Algorithms. Tradeoff : Execution speed vs. solution quality. “Short & sweet”. “Quick & dirty”. “Slowly but surely”. “Too little, too late”. Computational Complexity. Problem : Avoid getting trapped in local minima. Global optimum.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Algorithms' - oshin

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Solution

exact

approximate

fast

Speed

slow

Algorithms

Tradeoff: Execution speed vs. solution quality

“Short & sweet”

“Quick & dirty”

“Slowly but surely”

“Too little, too late”

Computational Complexity

Problem: Avoid getting trapped in local minima

Global optimum

Approximation Algorithms

Idea: Some intractable problems can be efficiently approximated within close to optimal!

Fast:

• Simple heuristics (e.g., greed)
• Provably-good approximations

Slower:

• Branch-and-bound approaches
• Integer Linear Programming relaxation

Approximation Algorithms

Wishful:

• Simulated annealing
• Genetic algorithms

Minimum Vertex Cover

Minumum vertex cover problem: Given a graph, find a minimum set of vertices such that each edge is incident to at least one vertex of these vertices.

Example:

Input graph

Heuristic solution

Optimal solution

Applications: bioinformtics, communications,

civil engineering, electrical engineering, etc.

• One of Karp’s original NP-complete problems

Approximate Vertex Cover

Theorem: The minimum vertex cover problem is NP-complete (even in planar graphs of max degree 3).

Theorem: The minimum vertex cover problem can be solved exactly within exponential time nO(1)2O(n).

Theorem: The minimum vertex cover problem can not be approximated within £ 1.36*OPT unless P=NP.

Theorem: The minimum vertex cover problem can be approximated (in linear time) within 2*OPT.

Idea: pick an edge, add its endpoints, and repeat.

x

y

Approximate Vertex Cover

Algorithm: Linear time 2*OPT approximation for the minimum vertex cover problem:

• Pick random edge (x,y)
• Add {x,y} to the heuristic solution
• Eliminate x and y from graph
• Repeat until graph is empty

Best approximation

bound known for VC!

Idea: one of {x,y} must be in any optimal solution.

Þ Heuristic solution is no worse than 2*OPT.

A

B

A

B

A

B

E

C

E

E

D

C

C

D

D

Maximum Cut

Maximum cut problem: Given a graph, find a partition

of the vertices maximizing the # of crossing edges.

Example:

cut size = 2

cut size = 4

cut size = 5

Input graph

Heuristic solution

Optimal solution

Applications: VLSI circuit design, statistical physics,

communication networks.

• One of Karp’s original NP-complete problems.

Maximum Cut

Theorem [Karp, 1972]: The minimum vertex cover

problem is NP-complete.

Theorem: The maximum cut problem can be solved

in polynomial time for planar graphs.

Theorem: The maximum cut problem can not

be approximated within £ 17/16*OPT unless P=NP.

Theorem: The maximum cut problem can be

approximated in polynomial time within 2*OPT.

Theorem: The maximum cut problem can be

approximated in polynomial time within 1.14*OPT.

=1.0625*OPT

A

B

E

C

D

A

B

A

B

E

E

C

C

D

D

Maximum Cut

Algorithm: 2*OPT approximation for maximum cut:

• If moving an arbitrary node across the partition

will improve the cut, then do so

• Repeat until no further improvement is possible

cut size = 2

cut size = 3

cut size = 5

Input graph

Heuristic solution

Optimal solution

Idea: final cut must contain at least half of all edges.

Þ Heuristic solution is no worse than 2*OPT.

Approximate Traveling Salesperson

Traveling salesperson problem: given a pointset, find

shortest tour that visits every point exactly once.

2*OPT metric TSP heuristic:

• Compute MST
• T = Traverse MST
• S = shortcut tour
• Output S

Analysis:

S < T

=

2*

MST < OPT TSP

2*

triangle inequality!

T covers minimum spanning tree twice

TSPminus an edge is a spanning tree

Non-Approximability

• NP transformations typically do not preserve the

approximability of the problem!

• Some NP-complete problems can be approximated

arbitrarily close to optimal in polynomial time.

Theorem: Geometric TSP can be approximated in

polynomial time within (1+e)*OPT for any e>0.

• Other NP-complete problems can not be approximated

within any constant in polynomial time (unless P=NP).

Theorem: General TSP can not be approximated

efficiently within K*OPT for any K>0 (unless P=NP).

Graph Isomorphism

Definition: two graphs G1=(V1,E1) and G2=(V2,E2) are

isomorphic iff \$ bijection ƒ:V1®V2such that

"vi,vjÎV1 (vi,vj)ÎE1Û (ƒ(vi),ƒ(vj))ÎE2

Isomorphism ºedge-preserving vertex permutation

Problem: are two given graphs isomorphic?

Note: Graph isomorphism ÎNP, but not known to be in P

Graph Isomorphism

G2

G1

G

Zero-Knowledge Proofs

Idea: proving graph isomorphism without disclosing it!

Premise: Everyone knows G1 and G2 but not ≈

≈must remain secret!

Create random G ≈ G1

Note: ≈ is≈(≈)

Verifier checks G≈G1 or G≈G2

Repeat k times

ÞProbability of cheating: 2-k

Approximating a “proof”!

G2

G1

χ

Zero-Knowledge Proofs

Idea: prove graph 3-colorable without disclosing how!

Premise: Everyone knows G1 but not its 3-coloring χ

which must remain secret!

χ'

Create random G2≈ G1

Note: 3-coloring χ'(G2) is ≈(χ(G1))

Verifier checks G1≈G2 or χ'(G2)

Repeat k times

ÞProbability of cheating: 2-k

χ

Interactive proof!

χ

Zero-Knowledge Caveats

• Requires a good random number generator
• Should not use the same graph twice
• Graphs must be large and complex enough

Applications:

• Identification friend-or-foe (IFF)
• Cryptography

Zero-Knowledge Proofs

Idea: prove that a Boolean formula P is satisfiable

without disclosing a satisfying assignment!

Premise: Everyone knows P but not its secret

satisfying assignment V !

P = (x+y+z)(x'+y'+z)(x'+y+z')

ƒ

Convert P into a graph 3-colorability

instance G =ƒ(P)

Use zero-knowledge protocol

to show that G is 3-colorable

Þ P is satisfiable iff G is 3-colorable

Þ P is satisfiable with probability1-2-k

G =

Interactive proof!

Interactive Proof Systems

• Prover has unbounded power and may be malicious
• Verifier is honest and has limited power

Completeness: If a statement is true, an honest verifier

will be convinced (with high prob) by an honest prover.

Soundness: If a statement is false, even an omnipotent

malicious prover can not convince an honest verifier that

the statement is true (except with a very low probability).

• The induced complexity class depends on the verifier’s

abilities and computational resources:

Theorem: For a deterministic P-time verifier, class is NP.

Def: For a probabilistic P-time verifier, induced class is IP.

Theorem [Shamir, 1992]: IP = PSPACE

Concepts, Techniques, Idea & Proofs

• 2-SAT
• 2-Way automata
• 3-colorability
• 3-SAT
• Abstract complexity
• Acceptance
• Algebraic numbers
• Algorithms
• Algorithms as strings
• Alice in Wonderland
• Alphabets
• Alternation
• Ambiguity
• Ambiguous grammars
• Analog computing
• Anisohedral tilings
• Aperiodic tilings
• Approximate min cut
• Approximate TSP
• Approximate vertex cover
• Approximations
• Artificial intelligence
• Asimov’s laws of robotics
• Asymptotics
• Automatic theorem proving
• Autonomous vehicles
• Axiom of choice
• Axiomatic method
• Axiomatic system
• Babbage’s analytical engine
• Babbage’s difference engine
• Bin packing
• Binary vs. unary
• Bletchley Park
• Bloom axioms
• Boolean algebra
• Boolean functions
• Bridges of Konigsberg
• Brute fore
• Busy beaver problem
• C programs
• Canonical order
• Cantor dust
• Cantor set
• CAPCHA
• Cardinality arguments
• Cartesian coordinates
• Cellular automata
• Chaos
• Chatterbots
• Chess-playing programs
• Chinese room
• Chomsky hierarchy
• Chomsky normal form
• Chomskyan linguistics
• Christofides’ heuristic
• Church-Turing thesis
• Clay Mathematics Institute

Concepts, Techniques, Ideas & Proofs

• Clique problem
• Cloaking devices
• Closure properties
• Cogito ergo sum
• Colorings
• Commutativity
• Complementation
• Completeness
• Complexity classes
• Complexity gaps
• Complexity Zoo
• Compositions
• Compound pendulums
• Compressibility
• Computable functions
• Computable numbers
• Computation and physics
• Computation models
• Computational complexity
• Computational universality
• Computer viruses
• Concatenation
• Co-NP
• Consciousness and sentience
• Consistency of axioms
• Constructions
• Context free grammars
• Context free languages
• Context sensitive grammars
• Context sensitive languages
• Continuity
• Continuum hypothesis
• Contrapositive
• Cook’s theorem
• Countability
• Counter automata
• Counter example
• Cross- product
• Crossing sequences
• Cross-product construction
• Cryptography
• DARPA Grand Challenge
• DARPA Math Challenges
• De Morgan’s law
• Decidability
• Deciders vs. recognizers
• Decimal number system
• Decision vs. optimization
• Dedekind cut
• Denseness of hierarchies
• Derivations
• Descriptive complexity
• Diagonalization
• Digital circuits
• Diophantine equations
• Disorder
• DNA computing
• Domains and ranges

Concepts, Techniques, Ideas & Proofs

• Dovetailing
• DSPACE
• DTIME
• EDVAC
• Elegance in proof
• Encodings
• Enigma cipher
• Entropy
• Enumeration
• Epsilon transitions
• Equivalence relation
• Euclid’s “Elements”
• Euclid’s axioms
• Euclidean geometry
• Euler’s formula
• Euler’s identity
• Eulerian tour
• Existence proofs
• Exoskeletons
• Exponential growth
• Exponentiation
• EXPSPACE
• EXPSPACE
• EXPSPACE complete
• EXPTIME
• EXPTIME complete
• Extended Chomsky hierarchy
• Fermat’s last theorem
• Fibonacci numbers
• Final states
• Finite automata
• Finite automata minimization
• Fixed-point theorem
• Formal languages
• Formalizations
• Four color problem
• Fractal art
• Fractals
• Functional programming
• Fundamental thm of Algebra
• Fundamental thm of Arith.
• Game of life
• Game theory
• Game trees
• Gap theorems
• Garey & Johnson
• General grammars
• Generalized colorability
• Generalized finite automata
• Generalized numbers
• Generalized venn diagrams
• Generative grammars
• Genetic algorithms
• Geometric / picture proofs
• Godel numbering
• Godel’s theorem
• Goldbach’s conjecture
• Golden ratio
• Grammar equivalence

Concepts, Techniques, Ideas & Proofs

• Grammars
• Grammars as computers
• Graph cliques
• Graph colorability
• Graph isomorphism
• Graph theory
• Graphs
• Graphs as relations
• Gravitational systems
• Greibach normal form
• “Grey goo”
• Guess-and-verify
• Halting problem
• Hamiltonian cycle
• Hardness
• Heuristics
• Hierarchy theorems
• Hilbert’s 23 problems
• Hilbert’s program
• Hilbert’s tenth problem
• Historical perspectives
• Household robots
• Hung state
• Hydraulic computers
• Hyper computation
• Hyperbolic geometry
• Hypernumbers
• Identities
• Immerman’s Theorem
• Incompleteness
• Incompressibility
• Independence of axioms
• Independent set problem
• Induction & its drawbacks
• Infinite hotels & applications
• Infinite loops
• Infinity hierarchy
• Information theory
• Inherent ambiguity
• Initial state
• Intelligence and mind
• Interactive proofs
• Intractability
• Irrational numbers
• JFLAP
• Karp’s paper
• Kissing number
• Kleene closure
• Knapsack problem
• Lambda calculus
• Language equivalence
• Law of accelerating returns
• Law of the excluded middle
• Lego computers
• Lexicographic order
• Linear-bounded automata
• Local minima
• LOGSPACE
• Low-deg graph colorability
• Machine enhancements

Concepts, Techniques, Ideas & Proofs

• Machine equivalence
• Mandelbrot set
• Manhattan project
• Many-one reduction
• Matiyasevich’s theorem
• Mechanical calculator
• Mechanical computers
• Memes
• Mental poker
• Meta-mathematics
• Millennium Prize
• Minimal grammars
• Minimum cut
• Modeling
• Multiple tapes
• Mu-recursive functions
• Nanotechnology
• Natural languages
• Navier-Stokes equations
• Neural networks
• Newtonian mechanics
• NLOGSPACE
• Non-approximability
• Non-closures
• Non-determinism
• Non-Euclidean geometry
• Non-existence proofs
• NP
• NP completeness
• NP-hard
• NSPACE
• NTIME
• Occam’s razor
• Octonions
• One-to-one correspondence
• Open problems
• Oracles
• P completeness
• P vs. NP
• Parallel postulate
• Parallel simulation
• Dovetailing simulation
• Parallelism
• Parity
• Parsing
• Partition problem
• Paths in graphs
• Peano arithmetic
• Penrose tilings
• Physics analogies
• Pi formulas
• Pigeon-hole principle
• Pilotless planes
• Pinwheel tilings
• Planar graph colorability
• Planarity testing
• Polya’s “How to Solve It”
• Polyhedral dissections

Concepts, Techniques, Ideas & Proofs

• Polynomial hierarchy
• Polynomial-time
• P-time reductions
• Positional # system
• Power sets
• Powerset construction
• Predicate calculus
• Predicate logic
• Prime numbers
• Principia Mathematica
• Probabilistic TMs
• Proof theory
• Propositional logic
• PSPACE
• PSPACE completeness
• Public-key cryptography
• Pumping theorems
• Pushdown automata
• Puzzle solvers
• Pythagorean theorem
• Quantifiers
• Quantum computing
• Quantum mechanics
• Quaternions
• Queue automata
• Quine
• Ramanujan identities
• Ramsey theory
• Randomness
• Rational numbers
• Real numbers
• Reality surpassing Sci-Fi
• Recognition and enumeration
• Recursion theorem
• Recursive function theory
• Recursive functions
• Reducibilities
• Reductions
• Regular expressions
• Regular languages
• Rejection
• Relations
• Relativity theory
• Relativization
• Resource-bounded comput.
• Respect for the definitions
• Reusability of space
• Reversal
• Reverse Turing test
• Rice’s Theorem
• Riemann hypothesis
• Riemann’s zeta function
• Robots in fiction
• Robustness of P and NP
• Satisfiability
• Savitch’s theorem
• Schmitt-Conway biprism
• Scientific method
• Sedenions

Concepts, Techniques, Ideas & Proofs

• Self compilation
• Self reproduction
• Set cover problem
• Set difference
• Set identities
• Set theory
• Shannon limit
• Sieve of Eratosthenes
• Simulated annealing
• Simulation
• Skepticism
• Soundness
• Space filling polyhedra
• Space hierarchy
• Spanning trees
• Speedup theorems
• Sphere packing
• Spherical geometry
• Standard model
• State minimization
• Steiner tree
• Stirling’s formula
• Stored progam
• String theory
• Strings
• Strong AI hypothesis
• Superposition
• Super-states
• Surcomplex numbers
• Surreal numbers
• Symbolic logic
• Symmetric closure
• Symmetric venn diagrams
• Technological singularity
• Theory-reality chasms
• Thermodynamics
• Time hierarchy
• Tinker Toy computers
• Tractability
• Transcendental numbers
• Transfinite arithmetic
• Transformations
• Transition function
• Transitive closure
• Transitivity
• Traveling salesperson
• Triangle inequality
• Turbulance
• Turing complete
• Turing degrees
• Turing jump
• Turing machines
• Turing recognizable
• Turing reduction
• Turing test
• Two-way automata
• Type errors
• Uncomputability

Concepts, Techniques, Ideas & Proofs

• Uncomputable functions
• Uncomputable numbers
• Uncountability
• Undecidability
• Universal Turing machine
• Venn diagrams
• Vertex cover
• Von Neumann architecture
• Von Neumann bottleneck
• Wang tiles & cubes
• Zero-knowledge protocols

.

.

.

.

“Make everything as simple

as possible, but not simpler.”

- Albert Einstein (1879-1955)