Shortcomings of Traditional Backtrack Search on Large, Tight CSPs: A Real-world Example

1. The Graduate Teaching Assistant (GTA) assignment project [1] aims at matching GTAs to academic tasks according to their abilities and availability. It is , is an on-going applied research project conducted at the Constraint Systems Laboratory as a service to the Department of Computer Science and Engineering and as a hands-on tool for training undergraduate and graduate students in advanced Constraint Processing techniques. In practice, the problem is always tight and often over-constrained, in that there are more courses to serve than there are GTAs hired to help. Also there are many non-convex constraints that restrict allowable assignments and combinations of GTAs to classes. Finally. we have enriched the application by allowing GTAs to express their preferences on a scale from 0 (unable to handle) to 5 (highest preference). Our goal is to explore and compare the performance of various backtrack search techniques: systematic search [1] and random backtrack search [2]. • Variables in the spring 2001 data set: 69 • Average shallowest level of BT: 48 • Thrashing in 30% of the search tree. Thrashing of BT search: • Variable Ordering Heuristics: 2 variable selection heuristics were implemented using least domain and domain-degree ratio as selection criteria. These heuristics were ordered statically and dynamically. Thus, we had 4 variable ordering heuristics: • Static Least Domain (SLD) • Static Domain Degree Ratio (SDD) • Dynamic Least Domain (DLD) • Dynamic Domain Degree Ratio (DDD) Shortcomings of Traditional Backtrack Search on Large, Tight CSPs: A Real-world Example Venkata Praveen Guddeti and Berthe Y. Choueiry Constraint Systems Laboratory • Department of Computer Science and Engineering, University of Nebraska-Lincoln • {vguddeti|choueiry}@cse.unl.edu Methodology Abstract • We compared the ordering heuristics according to five criteria: • Unassigned courses: the number of courses that are not assigned a GTA. • CC: the number of constraint checks. • NNV:the number of nodes visited. • Number of backtracks:the number of backtracks done. Problems of systematic BT search Unassigned variables: The table below shows the number of unassigned variables and the CPU run time taken for the solution. Systematic Search Many ordering heuristics have also been implemented to improve the performance of search and the embedded forward checking (FC) mechanism implemented for the GTA problem [1]. We experimented with various data sets of the GTA assignment problem to analyze and compare the performance of the various ordering heuristics implemented. We report our observations and summarize our analysis of the shortcomings of traditional backtrack search methods on large over-constrained CSPs. We show that the performance of value and variable ordering heuristics depend on the problem instance. And the BT mechanism gets “thrashed” in a small portion of the search space; unable to undo early choice, regardless of the ordering heuristic used. Systematic Search: Forward checking algorithm augmented with various ordering heuristics. Problems of random BT search The search tree of the GTA problem is so large that even random BT search is not enough to avoid the thrashing during search. Although the solutions vary with each run, the shallowest BT level does not improve. • Value Ordering Heuristics: 3 value ordering heuristics: • First in last out (FIL): The first available value is selected. • Highest preference value (PREFERENCE): The value having the highest preference is selected. • Least occurring value (OCCURRENCE): The value occurring the least number of times in the future variables is selected. Current investigations • We are testing various restart strategies based on • Fixed restarts and • Dynamic restarts [2]. • Limited BT search: Keeping a count of the amount of backtracking done so far and abandoning the search if more than the cutoff value [3]. • Credit based search: Each search path is assigned some credit. This credit may or may not be uniform. If credit is over than perform deterministic search and/or small amount of local search [3]. • Random Ordering Heuristics: 2 heuristics using randomness: • All Random: The value and variable selection were done randomly. • Hybrid heuristic: Random heuristics in combination with the above mentioned ordering heuristics. References Experiments • R. Glaubius and B. Y. Choueiry, “Constraint Modeling and Reformulation in the Context of Academic Task Assignment”. In Working Notes of the Workshop Modeling and Solving Problems with Constraints, ECAI 2002. • H. Kautz, E. Horvitz, Y. Ruan, C. Gomes, B. Selman; “Dynamic Restart Policies”. In Proceedings of the Eighteenth National Conference on Artificial Intelligence, Edmonton, Alberta, July 2002. AAAI Press. • H. Simonis, Invited Talk at the Constraint Systems Laboratory, UNL, November 2000. The combination of the 4 variable ordering heuristics and the 3 value ordering heuristic yielded 12 ordering heuristics. Three real-world data sets of spring 2001, fall2001 and fall2002 were used for the experiments. Thus, 48 experiments were carried out. Each experiment was run for 10 minutes. Results were noted for both the best solution found and for full 10 minute run.