Generating Satisfiable Problem Instances
1 / 24

AAAI00 Austin, Texas - PowerPoint PPT Presentation

  • Uploaded on

Generating Satisfiable Problem Instances Dimitris Achlioptas Microsoft Carla P. Gomes Cornell University Henry Kautz University of Washington Bart Selman Cornell University. AAAI00 Austin, Texas. Introduction.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' AAAI00 Austin, Texas' - jayme-whitley

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Generating Satisfiable Problem InstancesDimitris AchlioptasMicrosoftCarla P. Gomes Cornell University Henry KautzUniversity of WashingtonBart SelmanCornell University

  • AAAI00

  • Austin, Texas


  • An important factor in the development of search methods is the availability of good benchmarks.

  • Sources for benchmarks:

    • Real world instances

      • hard to find

      • too specific

    • Random generators

      • easier to control (size/hardness)

Random generators of instances
Random Generators of Instances

  • Understanding threshhold phenomena lets us tune the hardness of problem instances:

  • At low ratios of constraints -

    • most satisfiable, easy to find assignments;

  • At high ratios of constraints -

    • most unsatisfiableeasy to show inconsistency;

  • At the phase transition between these two regions

    • roughly half of the instances are satisfiable and we find a concentration of computationally hard instances.

Limitation of random generators
Limitation of Random Generators

  • PROBLEM: evaluating incompletelocal search algorithms

  • Filtering out Unsat Instances - use a complete method and throw away unsat instances.

    Problem: want to test on instances too large for any complete method!

  • “Forced” Formulas

    Problem: the resulting instances are easy – have many satisfying assignments


  • I Generation of only satisfiable instances

  • II New phase transition in the space of satisfiable instances

  • III Connection between hardness of satisfiable instances and new phase transition

  • IV Conclusions

Generation of only satisfiable instances

Generation of only satisfiable instances

Quasigroup or Latin Squares

Given an N X N matrix, and given N colors, color the matrix in such a way that:

-all cells are colored;

- each color occurs exactly once in each row;

- each color occurs exactly once in each column;

Quasigroup or Latin Square

Quasigroup Completion Problem (QCP)

Given a partial assignment of colors (10 colors in this case), can the partial quasigroup (latin square) be completed so we obtain a full quasigroup?


32% preassignment

Qcp a framework for studying search
QCP: A Framework for Studying Search

  • NP-Complete.

  • Random instances have structure not found in random k-SAT

    Closer to “real world” problems!

  • Can control hardness via % preassignment

  • BUT problem of creating large, guaranteed satisfiable instances remains…

(Anderson 85, Colbourn 83, 84, Denes & Keedwell 94, Fujita et al. 93, Gent et al. 99, Gomes & Selman 97, Gomes et al. 98, Shaw et al. 98, Walsh 99 )

Quasigroup with holes qwh

32% holes

Quasigroup with Holes(QWH)

  • Given a full quasigroup, “punch” holes into it

Difficulty: how to generate the full quasigroup, uniformly.

Question: does this give challenging instances?

Markov chain monte carlo mcmm
Markov Chain Monte Carlo (MCMM)

  • We use a Markov chain Monte Carlo method (MCMM) whose stationary (egodic) distribution is uniform over the space of NxN quasigroups (Jacobson and Matthews 96).

    • Start with arbitrary Latin Square

    • Random walk on a sequence of Squares obtained via local modifications

Generation of quasigroup with holes qwh
Generation of Quasigroup with Holes (QWH)

  • Use MCMM to generate solved Latin Square

  • Punch holes - i.e.,uncolor a fraction of the entries

    • The resulting instances are guaranteed satisfiable

    • QWH is NP-Hard

      Is there % holes where instances truly hard on average?

Easy hard easy pattern in backtracking search

Complete (Satz) Search

Order 30, 33, 36

Easy-Hard-Easy Pattern in Backtracking Search

QWH peaks near 32%

(QCP peaks near 42%)

Computational Cost

% holes

Easy hard easy pattern in local search

Local (Walksat) Search

Order 30, 33, 36

Easy-Hard-Easy Pattern in Local Search

Computational Cost

% holes

First solid statistics for overconstrainted area!

Phase transition in qwh
Phase Transition in QWH?

  • QWH - all instances are satisfiable - does it still make sense to talk about a phase transition?

    • The standard phase transition corresponds to the area with 50% SAT/UNSAT instances

    • Here all instances SAT

      Does some other property of the wffs show an abrupt change around “hard” region?


Preassigned cells


Backbone is the shared structure of all

solutions to a given instance (not counting preassigned cells)

Number sols = 2

Backbone size = 2

Phase transition in the backbone
Phase Transition in the Backbone

  • We have observed a transition in the size of backbone

    • Many holes – backbone close to 0%

    • Fewer holes – backbone close to 100%

    • Abrupt transition – coincides with hardest instances!

Sudden phase Transition in Backbone

and it coincides with the hardest area

New Phase Transition in Backbone

% Backbone

% of Backbone



% holes

Why correlation between backbone and problem hardness
Why correlation between backbone and problem hardness?

  • Intuitions: Local Search

  • Near 0% Backbone = many solutions = easy to find by chance

  • Near 100% Backbone = solutions tightly clustered = all the constraints “vote” in same direction

  • 50% Backbone = solutions in different clusters = different clauses push search toward different clusters

(Current work – verify intuitions!)

Why correlation between backbone and problem hardness1
Why correlation between backbone and problem hardness?

  • Intuitions:Backtracking search

  • Bad assignments to backbone variables near root of search tree cause the algorithm to deteriorate

  • For the algorithm to have a significant chance of making bad choices, a non-negligible fraction of variables must appear in the backbone

Reparameterization of backbone
Reparameterization of Backbone

Backbone for different orders (30 - 57)

% of Backbone

Reparameterization computational cost
ReparameterizationComputational Cost

Computational Cost different orders (30, 33, 36)

% of Backbone

Local Search


Local Search

(normalized & reparameterized)


  • QWH is a problem generator for satisfiable instances (only):

    • Easy to tune hardness

    • Exhibits more realistic structure

    • Well-suited for the study of incomplete search methods (as well as complete)

    • Confirmation of easy-hard-easy pattern in computational cost for local search

  • New kind of phase transition in backbone

    • Reparameterization

  • GOAL – new insights into practical complexity of problem solving

QWH generator, demos, available soon (< one month)