Myopic Policies for Budgeted Optimization with Constrained Experiments

1. Myopic Policies for Budgeted Optimization with Constrained Experiments JavadAzimi, Xiaoli Fern, Alan Fern Oregon State University AAAI, July 2010

2. Motivation: Electricity Production in a Microbial Fuel Cell (MFC) This is how an MFC works Nano-structure of anode significantly impact the electricity production. e- e- SEM image of bacteria sp. on Ni nanoparticle enhanced carbon fibers. Fuel (organic matter) O2 bacteria H+ H2O Oxidation products (CO2) Cathode Anode • We should optimize anode nano-structure to maximize power.

3. Experiment Selection • Experiments are costly and there is a fixed budget. • How to select the best sequence of experiments. Current Experiments Scientist selects Experiment Run Experiment

4. Bayesian Optimization (BO) • Since Running experiment is very expensive we use BO. • Select one experiment to run at a time based on results of previous experiments. Gaussian Process Surface Current Experiments Select Single Experiment Run Experiment

5. Space of Experiments Averaged Area Average Circularity Bayesian Optimization (BO) • BO assumes that we can ask for specific experiment. • This is unreasonable assumption in many applications. • In Fuel Cell it takes many trials to create a nano-structure with specific requested properties. • Costly to fulfill.

6. Constrained Experiments It is less costly to fulfill a request that specifies ranges for the nanostructure properties E.g. run an experiment with Averaged Area in range r1 and Average Circularity in range r2 We will call such requests “constrained experiments” Space of Experiments Averaged Area Average Circularity • Constrained Experiment 1 • large ranges • low cost • high uncertainty about which experiment will be run • Constrained Experiment 2 • small ranges • high cost • low uncertainty about which experiment will be run

7. BO for Constrained Experiment • Given a fixed budget, select the best constrained experiments. Gaussian Process Surface Current Experiments Select Single Experiment Select Constrained Experiment Run Experiment

8. Constrained Experiment • We generalized BO heuristics to constrained experiments. • Two challenges: • How to compute heuristics for constrained experiment? • How to take experimental cost into account?(which has been ignored by most of the approaches in BO).

9. Standard BO Heuristics • Standard heuristics are statistics of the posterior p(y|x,D) where D is our current observation. • Maximum Upper bound Interval (MUI) • Select point with highest 95% upper confidence bound • Purely explorative approach. • Maximum Probability of Improvement (MPI) • It computes the probability that the output is more than (1+m) times of the best current observation , m>0. • Explorative and Exploitative. • Maximum Expected of Improvement (MEI) • Similar to MPI but parameter free • It simply computes the expected amount of improvement after sampling at any point.

10. Generalizing BO for Constrained Experiment • Having the posterior distribution of p(y|x,D) and px(.|D) we can calculate the posterior of the output of each constrained experiment which has a closed form solution. • Therefore we can compute standard BO heuristics for constrained experiments. • There are closed form solution for these heuristics. Input space Discretization Level

11. Budgeted Constrained Experiments • We are limited with Budget B. • Unfortunately heuristics will typically select the smallest and most costly constrained experiments which is not a good use of budget. • How can we consider the cost of each constrained experiment in making the decision? • Cost Normalized Policy (CN) • Constraint Minimum Cost Policy(CMC) -Low uncertainty -High uncertainty -Better heuristic value -Lower heuristic value -Expensive -Cheap

12. Cost Normalized Policy • It selects the constrained experiment achieving the highest expected improvement per unit cost. • We report this approach for MEI policy only.

13. Constraint Minimum Cost Policy (CMC) • Motivation: • Approximately maximizes the heuristic value. • Has expected improvement at least as great as spending the same amount of budget on random experiments. • Example: Cost=10 random Cost=5 random Cost=4 random Very expensive: 10 random experiments likely to be better Selected Constrained experiment Poor heuristic value: not select due to 1st condition

14. Space of Experiments Experimental Results(Setup) • Gaussian process is used as our model with squared exponential kernel. • Cost function is defined as: • There is a constant cost for running any constrained experiment plus an additional cost depending on the size of the experiment. • The value of slope dictates how fast the cost increases as the size of a constrained experiment decreases. r1 r2

15. Experimental Results(Real Applications) • Two Real data sets: • Fuel Cell: • Fuel Cell electricity generation • Hydrogene: • Biosolar hydrogen production

16. Experimental Results(Benchmarks Functions) • 3 popular benchmarks used in BO literature. Discontinuous Cosines Rosenbrock

17. Overall Performance • The normalized regret of each framework is shown which is calculated as y*- ymaxover Random performance for budget 15 where y* is the highest possible output. Average regret of each approach over all frameworks.

18. Different Budget(1) Real Fuel Cell Random CMC-MUI Cosines Rosenbrock

19. Different Budget(2) Real Fuel Cell CN-MEI Cosines Rosenbrock

20. Different Budget(3) Real Fuel Cell CMC-MPI(0.2) Cosines Rosenbrock

21. Different Budget(4) Real Fuel Cell Cosines Rosenbrock CMC-MEI

22. Conclusion • We introduced a new constrained experiment framework which asks for hyper-rectangle rather than exact point. • We extended model-free BO heuristics to our frame work. • We introduced two approaches to optimize our budgeted framework. • CMC-MEI is working better than other approaches.

23. Thanks for attendanceQuestion?