Hierarchical Allelic Pairwise Independent Function by David Iclănzan

1. Hierarchical Allelic Pairwise Independent Functionby David Iclănzan Present by Tsung-Yu Ho At Teilab, 2011.08.08

2. Reference Paper • Hierarchical Allelic Pairwise Independent Functions • Author : David Iclănzan • Department of Electrical Engineering, Sapientia Hungarian University of Transylvania, Romania • GECCO 2011, Best Paper Award Winnersin Estimation of Distribution Algorithms

3. Abstract • Current multivariate EDAs rely on computationally efficientpairwiselinkage detection mechanisms to identify higher order linkage blocks. Historical attempts to exemplify the potential disadvantage of this computational shortcut were scarcely successful. • In this paper we introduce a new class of test functions to exemplify the inevitable weakness of the simplified linkage learning techniques. Specifically, we show that presently employed EDAsare not able to efficiently mix and decidebetween building-blocks with pairwise allelic independent components. These problems can be solved by EDAs only at the expense of exploring a vastly larger search space of multivariable linkages.

4. Outline • Basic knowledge for 碩一 • Genetic algorithm (GA) • Estimation of distribution algorithm (EDA) • What’s the problem? • Motivation • Goal of this paper • Testing allelic pairwise ibdependant function • Concatenated parity function (CPF) • Concatenated parity / trap function (CP/TF) • Proposed Function • Difference between Teil’s work and this paper • Conclusion

5. Basic Knowledge (1) • Genetic algorithm (GA), or evolutionary algorithm (EA) • Stochastic search for optimization • Operate on the population, numerous solutions (search space) • Evolve solutions generation by generation • A GA flow in one Generation • Evaluation • Selection • Crossover • Mutation • Replacement • Convergence on expected condition

6. Basic Knowledge (2) • Estimation of distribution algorithms (EDAs) • Or probabilistic model-building genetic algorithm (PMBGAs) • Extend of traditional GAs • GAs with linkage learning and model building (not exactly) • Main idea • Extract information from promising solutions • Exploit information to build probabilistic model • Generate next population from building model.

7. Basic Knowledge (3) • Classes of EDAs based on linkage learning mechanisms • Univariate • No linkage between any variables • cGA, PBIL, UMDA • Bivariate • Most two variables with linkage • BMDA, MIMIC • Multivariate • K variables with linkage • Detect from 2 order to k order. • ECGA, BOA, hBOA, EBNA, DSMGA

8. Basic Knowledge (4) • cGA (univariate EDA) • Vector [P1,P2,P3,P4] = [0.5, 0.5 ,0.5 ,0.5] • Generate two individuals 0 1 1 1 (f=3) 1 0 0 1 (f=2) • Update Vector=[0.25,0.75,0.75,0.5] • ECGA (multivariate EDA) • [0] [1] [2] [3] -> [0,1] [2] [3] -> [0,2] [1] [3] -> [0,2,1] [3] [0,2] [1] [3] [0,2,3] [1] [0,3] [1] [2]

9. Motivation (1) • Multivariate EDAs • Most suited EDAs for solving nearly decomposable problems. • Exploitation of Pairwise linkage for higher-order model • Pairwise , otherwise • BOA : only one addition or deletion of an edge in each step • It is critical for precision of the linkage model in early iteration • If Some testing problems lead to wrong accuracy. 3 4 1 3 4 1 2 2 5 5

10. Motivation (2) • Innovation time • Crossover operator achieve a solution better than any solutions at this point. • Takeover time • Selection operator converge a solution • If innovation time < takeover time • New innovation will be generated • If innovation time > takeover time • Results in premature convergence. • Innovation time is heavily affected by linkage model (1) (2)

11. Motivation (3) • The precision of the linkage model • Model Accuracy • Overfitting • Spurious linkage • Underfiiting • Missing important linkage (???) • Even without important linkage, EDA can solve problem under polynomial time • What is necessary linkage ? • Define linkage by Nfe (number of function evaluation) • The bonding will be probabilistic? • Sometime link, or sometime not • If there exists a nearly decomposable function to make inaccuracy linkage • Parity Function?

12. CPF Function • Concatenated Parity Function 2 3 Length = 20, k=4, m=5 X = 0010 1101 1100 0001 1001 3 2 2 3 3 CPF(X) =

13. CP/TF Function • Concatenated Parity / Trap Function 2 3 X = 011 001 110 110 000 000 111 3 2 2 1 2 0 3 CPF(X) = 1 1 1 110 0 0 0 0 001 1 000 2

14. Wash Function

15. Goal of this paper (1) • Coffin and Smith found parity functions, where variables appear to be independent when observing only two of them, to fail EDAs. • It was believed to be difficult for EDAs. • Chen and Yu found that cGA and ECGA can solveparity functions in the polynomial time. • cGA, without linkage learning, can solve parity function. • The author think that parity function can’t present the allelic pairwise independent functions.

16. Goal of this paper (2) • Author says, “This paper focuses on settling the open question, whether or not the heavy reliance on pairwise exploitation in present EDAsimplies a weakness on linkage learning for some nearly decomposable problems --- a class of problems for which EDAs are considered well-suited.” • The author think “CPF” is too easy to be the typical problems. • Therefore, his goal is to propose a newallelic pairwise independent functions to fail current EDAs

17. From Teil’s View • True • EDAs(hBOA) cannot solve CPF efficiently. • cGA and ECGA can solve CPF efficiently. • Give correct linkage model, EDAs can solve CPF efficiently (???) • [Author’s view] • CPF is easy, because cGA can solve it. • Find a new functions to fail ECGA (or other EDAs). • ECGA with pairwise linkage learning must fail. • Must give correct linkage, then ECGA can solve. (???) • [Teil’s View] • CPF is easy, because cGA can solve it. • Other EDAs can’t solve CPF? • Why? => Find the reason. • We found the replacement mechanism.

18. The summary of this paper’s flow EDAs Pairwise Linkage Detect Computational must Uni- variae Multi- variae (5) Weakness (2)Hard (?) (2)Easy (1)Well (4)Hard Some famous nearly decomposable problems CPF without pairwise linkage New Function (3)CPF Is easy Nearly Decomposable Problems

19. Hierarchical Pairwise Allelic Independent Function (1) 11 -> 1 00 -> 0 Other -> -

20. Hierarchical Pairwise Allelic Independent Function (2) 0010 1101 0011 0001 1001 1 1 0 0 0 0010 1101 0011 0001 1001 --- 0 1 --- ---

21. CPF and HPAIF • CPF has too many optima • global optima • HPAIF • optima

22. Experiment

23. Conclusion • EDAs on some parity function may scale exponentially. • cGA and ECGA can scale CPF on polynomial time. • CPF, CP/TF, Walsh Function may be easy on allelic pairwise independent function • The author proposed a new function to detect if EDAs with pairwise detection has weakness. • ECGA failed with it. • Current EDAs with pairwise linkage learning has weakness on some nearly decomposable function. • He propose a new allelic pairwise independent function to support this.

24. Discussion • There is no need to design a new testing function that most EDAs cannot solve. • Moreover, the only way to solve this testing function is to give EDAs the information of linkage. • The author said that CPF is easy testing problem. • Hence, ECGA can solve CPF. • However, no explain on why hBOA failed on it. • If we believe that CPF is an easy testing function. • We should first clear why hBOA failed on CPF