Learning to Satisfy Actuator Networks

1. Learning to Satisfy Actuator Networks Mark Coates National Science and Engineering Research Council of Canada (NSERC)

2. A Journey • “And what is a journey?  Is it just… distance traveled?  Time spent?  No.  It's what happens on the way, the things that happen to you.  At the end of the journey you're not the same.”

3. Plan your Journey, Learn • “It's what happens on the way, it the things that happen to you.” • Sensor Networks SANETs • Local Actuation: • Control sensors, control objects. • Modify the environment. • Learn causal relationships between actuations and environmental (model) variables. • Plan behaviour to optimize performance

4. Sensor/Actuator Network • Set of actuators ( ) + associated sensors ( ). • Actuators perform a physical (modifying) action. • Sensors monitor the response of the system. • Quantify the net effect on the system (positive or negative) • Design actuation strategy to optimize response

5. Causal Analysis • How can we infer the impact of an actuation based on a set of observations? • In particular, how do we derive: Manipulated Probability P(Y | X := x, Z=z) From (observations based on) Unmanipulated Probability P(Y | X = x, Z=z)

6. Example Problem • We wish to evaluate the average effectiveness of a fertilizer • Local background variables (for example soil moisture, temperature, salinity, weed density) affect: • The successful reception of the fertilizer • The impact of the fertilizer on the crop

7. Causal Graph uj : local realizations of background variables with global distribution g zj : action by actuator (0/1 = off/on) [known or measured] dj : actuation received (0/1 = no/yes) [unobserved] yj : response (0/1 = negative/positive) [unobserved] xj , wj: observed measurements, dependent on dj and yj.

8. Average Causal Effect (ACE) • Expectation (over latent variables) of: • [ Prob. of positive response given fertilizer ― Prob. of positive response withoutfertilizer ]

9. Model the mapping not the variable • Problem: • Latent variables u can be high dimensional; • Probability distribution g(u) can have complex structure. • Approach: • We have binary variables Z, D, Y • We don’t care about the value of uj and how that directly influences dj • What we do care about is how u impacts the mappings ZD and DY

10. New Causal Graph g • cr: sixteen states • c= • Much easier to estimate this distribution 0: inhibit 1: pass 2: flip 3: activate crj zj dj xj yj wj

11. Evaluating ACE • Estimate ACE by applying distributed EM algorithm across the graphical model (model g(cr) as multinomial) • Locally maximize the likelihood function: • Expectation: calculate expected crjat each node • Maximization: average the expected crito estimate g(cr).

12. Sensor Network Evaluation • Tree network topology: An efficient mechanism for data aggregation and dissemination. • Data aggregation (bottom-up) • Leaf nodes: Transmit E[crj] to parent node • Parent node: Performs aggregation and relays result to its own parent • Root node: Performs maximization • Result dissemination (top-down) • Each node broadcasts result to its children nodes

13. Influencing the Environment • Design an actuation strategy • Set of decision rules • Map from (current and past) sensor measurements to an actuation • Possible Objectives: • Maximize expected response of system • Provide probabilistic bounds on worst-case behaviour • Possible Scenarios • Accurate models of probability distributions  Bayesian networks • Uninformative models  Learning approaches

14. Problem Formulation • Epoch of T discrete time intervals • At times t = 0,…,T, node i measures a set of environmental variables • Chooses an actuation belonging to a discrete set of actuations • At the end of the epoch, measure a local response variable

15. Maximization Approach • Single binary actuation decision without a good model of p(y|v,a) • Consider p(y|v,a) = f(y|v,a) + n(v,a) • We have a set of points (vi,ai,yi) and want to learn the best actuation strategy A(v), i.e., that which maximizes f(y|v,a). • Approach: regression + subsequent maximization

16. Robustness concerns • Maximization amplifies regression errors. • Multi-stage planning implies repeated regression + maximization Proliferation of error

17. Relaxed Problem • Identify the largest set of environmental conditions and actuations such that: • Expected response exceeds threshold • Probability of terrible response is very low

18. LSAT Formulation • Learning to Satisfy (LSAT) • Given points find the set G that solves: subject to where Ci(G,P) are constraints.

19. Two types of constraints • Point-wise constraints: C(G,P) = C(x,G,P) is a function of the input variable x and the constraint takes the form • Example: • Set-average constraints: C(G,P) > 0 is only satisfied on-average over the entire set. • Example:

20. Solution • Derive equivalent empirical constraints • Consider solution to empirical constraints subject to • If empirical constraints are close to ideal constraints then solution is satisfactory. • Algorithm: extension of support vector machine • Lagrangian formulation allocating different penalties to violations of individual constraints.

21. Comments • Actuator networks present a host of problems • Assessment of whether causal relationships exist and evaluation of their strength • Design of actuation strategies that yield a satisfactory (optimal?) environmental response • These problems are difficult in a centralized setting – the extension to distributed algorithms poses an even greater challenge.