1 / 18

Challenges Promoting Theoretical EC Research

Challenges Promoting Theoretical EC Research. William E. Hart Optimization Uncertainty/Estimation Dept Sandia National Laboratories wehart@sandia.gov http://www.cs.sandia.gov/~wehart. Overview. A reminder: EC theory is relevant! Publishing theoretical conference papers is difficult

kasie
Download Presentation

Challenges Promoting Theoretical EC Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Challenges Promoting Theoretical EC Research William E. Hart Optimization Uncertainty/Estimation Dept Sandia National Laboratories wehart@sandia.gov http://www.cs.sandia.gov/~wehart

  2. Overview • A reminder: EC theory is relevant! • Publishing theoretical conference papers is difficult • … some personal reflections • Some ideas/suggestions EC Theory - GECCO 2002

  3. The Role of EC Theory • I. Modeling and Abstraction • Goal: identify/validate essential features of successful EAs • Simple EC methods allow us to explore their limitations/possibilities • Analysis of abstract EC methods enables these results to be applied to many specific implementations • Q: What are effective abstract EA formulations? EC Theory - GECCO 2002

  4. The Role of EC Theory • Unification and Linkage • Goal: develop linkages between different disciplines • Transferring ideas between disciplines is easier in simple EC algorithms • Reformulating EC methods as more traditional algorithmic techniques clarifies their strengths and weaknesses • Analytic tools from other disciplines can be validated on simple EC models • Q: Can analysis of direct search methods be applied to EAs? EC Theory - GECCO 2002

  5. The Role of EC Theory • Assessing Fundamental Algorithmic Capabilities • Goal: what EC methods can solve what classes of problems? • Rigorous evaluation of EC methods is required • Negative results must be formulated mathematically • Mathematical analysis provides the strongest assurance of robustness in EC methods • Q: Is randomization an essential feature of EC methods? EC Theory - GECCO 2002

  6. The Role of EC Theory • Identifying New Algorithmic Directions • Goal: anticipate and evaluate potential EC methods • Can consider EC methods for which practical implementations are not readily available • Abstract EC models can identify promising research directions and desirable EC capabilities • The design of practical EC methods can be guided by our analysis of abstract EC models • Q: How can EC methods effectively exploit derivative information? EC Theory - GECCO 2002

  7. What Theory (Probably) Cannot Do • Guarantee fast global convergence • Explain why your code does not seem to work on your application • Provide convergence guarantees for your new EC heuristic • Prove that a particular EC heuristics will not work in general EC Theory - GECCO 2002

  8. Publishing Theoretical EC Results - Journals • EC journals generally encourage theoretical results • Journal articles are usually long enough to provide a complete analysis • Reviewers do not always appreciate the strategic value of theoretical results • Providing reasonable turnaround to authors of theoretical papers is challenging! EC Theory - GECCO 2002

  9. Publishing Theoretical EC Results - Conferences • Conference papers have very tight space limitations! • Short presentation of theoretical results is quite difficult • What background can you assume of an EC audience? • How can you explain, state and discuss the results? • Conferences typically have a very loose review process • Quick reviews needed • Rough match of papers to reviewers • Limited feedback to authors EC Theory - GECCO 2002

  10. My Own Experiences • Publishing in: • ICGA, GECCO, CEC, FOGA • EC, IEEE TEC • General theme: local convergence on • Evolutionary pattern search EA framework • Self-adaptive EAs • Challenges: • Effectively meeting page limitations! • Poor reviewer feedback • Distinguishing between conference/journal publications EC Theory - GECCO 2002

  11. Meeting Conference Page Limits • Idea 1: State the results without proofs • Usually can condense analysis to fit page limits • Some reviewer always complains about missing proofs • Note: any negative feedback can make it difficult to publish a conference paper! EC Theory - GECCO 2002

  12. Meeting Conference Page Limits (cont’d) • Idea 2: Include proofs in an optimal appendix for the reviewers • Reviewers get confused • It is easy to ‘ding’ a paper for exceeding page limits • Organizing and writing this appendix well can be challenging! EC Theory - GECCO 2002

  13. Poor Reviewer Feedback • Typical reviewer comments: • I don’t believe these results • I’m confused by these results… • The analysis considers an algorithm/problem that is too simple • The authors should rework their paper to … • Poor review of theoretical results: • A simple ‘bug’ in my ICGA97 paper was not caught • Many reviewers fail to appreciate the difference between an approximate analysis of an EA’s search dynamics and an exact characterization of an EA’s search dynamics EC Theory - GECCO 2002

  14. Publication in Journals • EC Publication Policy • Dilemma for EC theory • Can ‘simple’ theoretical conference papers be ‘republished’ in journals? • How about more extended conference papers? (e.g. FOGA?) • We should encourage theoretical analyses to be published in journals (they get more careful reviews there). • Current policies effectively discourage publication of theoretical papers in conferences! Papers should describe mature work that is original in nature and has not been published elsewhere. EC Theory - GECCO 2002

  15. Some Ideas/Suggestions • Encourage the submission of “Extended Abstracts” for conferences • Idea: a paper that explicitly omits technical details to focus on the ‘big picture’ • Commonly used in theoretical computer science conferences STOC, FOCS, SODA • Special Sessions/Minisymposiums • Idea: sessions of presenters who do not prepare a paper • Organized about a specific topic or theme • Approved by the conference organizing committee • Commonly used within mathematical conferences (e.g. SIAM) EC Theory - GECCO 2002

  16. Some Ideas/Suggestions (cont’d) • Incorporate EC Theory tracks into EC conferences • Idea: explicitly encourage theoretical papers • This would hopefully help focus the reviewers • Problem: this track probably cuts across other tracks • Work to clarify editorial policies for EC journals • Idea: allow ‘revised’ conference papers • Example: SIAM J Computing • In general, only original papers will be considered. However, papers previously presented in conference proceedings of limited circulation may be submitted in revised form. EC Theory - GECCO 2002

  17. Outstanding Issues • How do we ensure that qualified reviewers are available for theoretical papers? • What constitutes an adequate review of theoretical papers? • How can we best educate the EC community about the impact of EC theory? EC Theory - GECCO 2002

  18. References • Challenges for Theoretical Computer Science • http://www.research.att.com/~dsj/nsflist.html • SIAM: Society for Industrial and Applied Mathematics • http://www.siam.org/ • ACM SIGACT • http://sigact.acm.org/sigact • Advice to Authors of Extended Abstracts http://www.acm.org/sigsoft/conferences/pughadvice.html EC Theory - GECCO 2002

More Related