210 likes | 213 Views
An Empirical Study of a New Restart Strategy for Randomized Backtrack Search Venkata P. Guddeti and Berthe Y. Choueiry Constraint Systems Laboratory University of Nebraska-Lincoln. Outline. Summary of contributions Context: Graduate Teaching Assistants Assignment Problem (GTAAP)
E N D
An Empirical Study of a New Restart Strategy for Randomized Backtrack Search Venkata P. Guddeti and Berthe Y. Choueiry Constraint Systems Laboratory University of Nebraska-Lincoln CSPIA 04
Outline • Summary of contributions • Context: Graduate Teaching Assistants Assignment Problem (GTAAP) • Randomized BT search with restarts • Empirical evaluations • Conclusions & future research directions CSPIA 04
Summary of contributions • An improved restart strategy for randomized backtrack search (RDGR) • Evaluation & characterization • Comparison with BT, LS, ERA, RGR • Problem types: GTAAP & random CSPs • Criterion: solution quality distribution • Conclusions • Identified regimes where a given technique dominates • Building blocks for cooperative, hybrid search CSPIA 04
Context: GTAAP Hiring & managing GTAs as instructors + graders • Given • A set of courses • A set of GTAs • A set of constraints that specify allowable assignments • Find a consistent & satisfactory assignment • Consistent: assignment breaks no (hard) constraints • Satisfactory: assignment maximizes • number of courses covered • happiness of the GTAs CSPIA 04
Constraint-based model • Variables (typically70 courses) • Grading, conducting lectures, labs & recitations • Values (30 GTAs) • Hired GTAs (+ preference for each value in domain) • Constraints • Unary, binary, global (e.g., capacity) • Objective • longest consistent solution (primary criterion) • maximize geometric mean of preferences (secondary criterion) CSPIA 04
Backtrack search (BT) • In theory, complete. In practice... forget it • Huge branching factor causes thrashing backtrack never reaches early variables • Tested 12 ordering heuristics • No significant difference • Use randomization & • restarts [Gomes et al. 98] CSPIA 04
Randomized BT with restarts • On stagnation of backtrack search • Interrupt search, then restart • Explore wider areas of search space: randomized variable-value ordering • Restart strategies • Fixed-cutoff, universal strategy [Luby et al. 93] • RRR: randomization & rapid restarts [Gomes et al. 98] • Fixed optimal cutoff value • Priori knowledge of cost distribution required • RGR:randomization & geometric restarts[Walsh 99] CSPIA 04
RGR • Static restart strategy • As cutoff value increases, RGR degenerates into BT • Ensures completeness (utopian in our setting) • But… restart is obstructed • … and thrashing reappears diminishing the probability of finding a solution CSPIA 04
RDGR • Randomization & Dynamic Geometric Restarts • Cutoff value • Depends on the progress of search • Never decreases • Increases at a much slower rate than RGR • Feature: restart is ‘less’ obstructed CSPIA 04
Experiments: 3 sets • Effect of run time on RGR & RDGR • Choiceofr in RGR & RDGR • Relative performance of RDGR versus • Backtrack search (BT) [Glaubius 01] • Local Search (LS) [Zou 03] • Multi-Agent Search (ERA) [Liu et al. 02, Zou 03] • RGR All implementations use same platform and executed to the best of our abilities (internal competition) CSPIA 04
Evaluation criteria • Solution Quality Distribution (SQD) • cumulative distributions of solution quality • measured as percentage deviation from best known solution • Descriptive statistics • Mean, median, mode, std dev, max, min • 95% confidence interval using Mann-Whitney test CSPIA 04
Data sets • 6 real-world data sets (GTAAP) • 3 solvable, 3 over-constrained • Experiment repeated 500 times • 4 sets of randomly generated problems • Model B, 100 instances, each instance runs for 3 minutes Solvable <25,15,0.5,0.36> Unsolvable <25,15,0.5,0.36> <40,20,0.5,0.2> <40,20,0.5,0.5> CSPIA 04
1. Effect of varying run time • RDGR consistently outperforms RGR • Running time does not affect the relative dominance CSPIA 04
2. Choice ofrin RGR r = 1.1 for RGR for GTAAP & random CSPs CSPIA 04
2. Choice of rin RDGR r = 1.1 for GTAAP r = 2 for random CSPs CSPIA 04
3. Performance: SQDs • Under-constrained: ERA > RDGR > RGR > BT > LS • Over-constrained: RDGR > RGR > BT > LS > ERA CSPIA 04
3. SQDs at phase transition • Solvable: ERA still wins for smallest deviations • Unsolvable: RDGR > RGR > BT > ERA > LS CSPIA 04
3. Performance: RDGR vs. RGR • RDGR allows more restarts than RGR • RDGR is more stable than RGR CSPIA 04
Summary: algorithm dominance On GTAAP and randomly generated CSPs • Solvable instances ERA > RDGR > RGR > BT > LS • Over-constrained instances RDGR > RGR > BT > LS > ERA CSPIA 04
Future research • More evaluations on real-world problems • Design ‘progress-aware’ restart strategies • Where cutoff value is changed during search • Design new search strategies • Hybrids: a solution from a given technique is fed to another • Cooperative: strategies applied where most appropriate within a given problem instance CSPIA 04