1 / 33

An Interactive System for Hiring & Managing Graduate Teaching Assistants

An Interactive System for Hiring & Managing Graduate Teaching Assistants. Ryan Lim Venkata Praveen Guddeti Berthe Y. Choueiry Constraint Systems Laboratory University of Nebraska-Lincoln. Outline. Task & Motivation System Architecture & Interfaces Scientific aspects Problem Modeling

michel
Download Presentation

An Interactive System for Hiring & Managing Graduate Teaching Assistants

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. An Interactive System for Hiring & Managing Graduate Teaching Assistants Ryan Lim Venkata Praveen Guddeti Berthe Y. Choueiry Constraint Systems Laboratory University of Nebraska-Lincoln

  2. Outline • Task & Motivation • System Architecture & Interfaces • Scientific aspects • Problem Modeling • Problem Solving • Comparing & Characterizing Solvers • Motivation revisited & Conclusions

  3. Task • Hiring & managing GTAs as instructors + graders • Given • A set of courses • A set of graduate teaching assistants • A set of constraints that specify allowable assignments • Find a consistent & satisfactory assignment • Consistent: assignment breaks no (hard) constraints • Satisfactory: assignment maximizes • number of courses covered • happiness of the GTAs • Often, number of hired GTAs is insufficient

  4. Motivation • Context • “Most difficult duty of a department chair” [Reichenbach, 2000] • Assignments done manually, countless reviews, persistent inconsistencies • Unhappy instructors, unhappy GTAs, unhappy students • Observation • Computers are good at maintaining consistency • Humans are good at balancing tradeoffs • Our solution • An online, constraint-based system • With interactive & automated search mechanisms

  5. Outline • Task & Motivation • System Architecture & Interfaces • Scientific aspects • Problem Modeling • Problem Solving • Comparing & Characterizing Solvers • Motivation revisited & Conclusions

  6. Password Protected Access for GTAs http://cse.unl.edu/~gta Password Protected Access for Manager http://cse.unl.edu/~gta • Web-interface for applicants • Web-interface for manager • View / edit GTA records • Setup classes • Specify constraints • Enforce pre-assignments Visualization widgets Local DB Other structured, semi-structured, or unstructured DBs Interactive Search Automated Search Heuristic BT Stochastic LS Multi-agent Search Randomized BT • A local relational database • Graphical selective queries Cooperative, hybrid Search Strategies • Drivers for • Interactive assignments • Automated search algorithms In progress System Architecture

  7. GTA interface: Preference Specification

  8. Manager interface: TA Hiring & Load

  9. Outline • Task & Motivation • System Architecture & Interfaces • Scientific aspects • Problem Modeling • Problem Solving • Comparing & Characterizing Solvers • Motivation revisited & Conclusions

  10. Constraint-based Model • Variables • Grading, conducting lectures, labs & recitations • Values • Hired GTAs (+ preference for each value in domain) • Constraints • Unary: ITA certification, enrollment, time conflict, non-zero preferences, etc. • Binary (Mutex): overlapping courses • Non-binary: same-TA, capacity, confinement • Objective • longest partial and consistent solution (primary criterion) • while maximizing GTAs’ preferences (secondary criterion)

  11. Outline • Task & Motivation • System Architecture & Interfaces • Scientific aspects • Problem Modeling • Problem Solving • Comparing & Characterizing Solvers • Motivation revisited & Conclusions

  12. Problem Solving • Interactive decision making • Seamlessly switching between perspectives • Propagates decisions (MAC) • Automated search algorithms • Heuristic backtrack search (BT) • Stochastic local search (LS) • Multi-agent search (ERA) • Randomized backtrack search (RDGR) • Future: Auction-based, GA, MIP, LD-search, etc. • On-going: Cooperative/hybrid strategies

  13. Manager interface: Interactive Selection

  14. Dual perspective Task-centered view Resource-centered view

  15. Shallowest level reached by BT after … Number of variables: 69 24 hr: 51 (26%) 1 min: 55 (20%) Max depth: 57 Depth of the tree: 69 Heuristic BT Search • Since we don’t know, a priori, whether instance is solvable, tight, or over-constrained • Modified basic backtrack mechanism to deal with this situation • We designed & tested various ordering heuristics: • Dynamic LD was consistently best • Branching factor relatively huge (30) • Causes thrashing, backtrack never reaches early variables

  16. Stochastic Local Search • Hill-climbing with min-conflict heuristic • Constraint propagation: • To handle non-binary constraints (e.g., high-arity capacity constraints) • Greedy: • Consistent assignments are not undone • Random walk to avoid local maxima • Random restarts to recover from local maxima

  17. Multi-Agent Search (ERA)[Liu et al. 02] • “Extremely” decentralized local search • Agents (variables) seek to occupy best positions (values) • Environment records constraint violation in each position of an agent given positions of other agents • Agents move, egoistically, between positions according to reactive Rules • Decisions are local • An agent can always kick other agents from a favorite position even when value of ‘global objective function’ is not improved • ERA appears immune to local optima • Lack of centralized control • Agents continue to kick each other • Deadlock appears in over-constrained problems

  18. Randomized BT Search • Random variable/value selection allows BT to visit a wider area of the search space [Gomes et al. 98] • Restarts to overcome thrashing • Walsh proposed RGR [Walsh 99] • Our strategy, RDGR, improves RGR with dynamic choice of cutoff values for the restart strategy [Guddeti & Choueiry 04]

  19. Optimizing solutions • Primary criterion: solution length • BT, LS, ERA, RGR, RDGR • Secondary criterion: preference values • BT, LS, RGR, RDGR • Criterion: • Average preference • Geometric mean • Maximum minimal preference

  20. More Solvers… • Interactive decision making • Automated search algorithms • BT, LS, ERA, RGR, RDGR. • Future: Auction-based, GA, MIP, LD-search, etc. • On-going: Cooperative / hybrid strategies

  21. Outline • Task & Motivation • System Architecture & Interfaces • Scientific aspects • Problem Modeling • Problem Solving • Comparing & Characterizing Solvers • Motivation revisited & Conclusions

  22. Comparing Solvers • Using the same CSP encoding, students implements solvers separately and competed for best results • Experience lead to the identification of behavioral criteria and regimes that characterize the performance of the various solvers in the context of GTAP

  23. Characterizing Solvers • General criteria • Stability, solution length, vulnerability to local optima, deadlock, thrashing, etc. • Tight but solvable instances • ERA  RDGR  RGR  BT  LS • Over-constrained instances • RDGR  RGR  BT  ERA  LS

  24. Outline • Task & Motivation • System Architecture & Interfaces • Scientific aspects • Problem Modeling • Problem Solving • Comparing & Characterization Solvers • Motivation revisited & Conclusions

  25. Motivation (revisited) • “Most difficult duty of a department chair” • Keeps the manager in the decision loop while removing the need for tedious and error-prone manual assignments • Helps producing quick (3 weeks down to 2 days) and satisfactory (stable) assignments • Initially, assignments were manually done on paper • Now, on-line data acquisition process • Enabled department to streamline & standardize GTA selection, hiring, and assignment • Overworked staff, unhappy GTAs • Overjoyed staff (relieved from handling application forms and massive paperwork) • Enthusiastic anonymous online reviews from applicants

  26. History & Evaluation • System entirely built by students • Modeling started in January 2001 • Prototype system used since August 2001 • Features improved and added as needs arised • No formal longitudinal study • Since August 2003: 109 GTA users, 23 feedback responses • Since April 2004, CSE implemented on-line GTA evaluation by faculty on top of GTAAP

  27. GTA Online Feedback Navigation Data entry 23 responses

  28. Conclusions • Integrated interactive & automated problem-solving strategies • Reduced the burden of the manager • Lead to quick development of ‘stable’ solutions • Our efforts • Helped the department • Trained students in CP techniques • Paved new avenues for research • Cooperative, hybrid search • Visualization of solution space

  29. <<< end of presentation I welcome your questions Please contact me for a live demo

  30. Manager interface: Course Load Specification

  31. Manager interface: Preassignment

  32. Manager interface: Constraint Specification

More Related