Control and the Synchronous Model of Computation

1. Control and the Synchronous Model of Computation Raja Sengupta Systems Program, Civil & Environmental Engineering University of California, Berkeley http://path.berkeley.edu/~raja Joint Work with S. Dickey, J. Misener, D. Nelson, S. Tan, S. Rezaei, P. Seiler, Q.Xu, M. Zennaro, UC Berkeley H. Krishnan, General Motors

2. Relevant Papers • Xu Q., Sengupta R. Real-time Estimation of a Markov Process over a Noisy Digital Communication Channel. Submitted to the IEEE Transactions on Automatic Control, June 2005. • Zennaro M. Sengupta R. Distributing Synchronous Programs Using Bounded Queues. Accepted by EMSOFT 2005. • Seiler P., Sengupta R. An H_infinity Approach to Networked Control. IEEE Transactions on Automatic Control, vol.50, no.3, March 2005, pp.356-64. • Ergen M., Lee D., Sengupta R., Varaiya P. Wireless Token Ring Protocol. IEEE transactions on Vehicular Technology, vol.53, no.6, Nov. 2004, pp.1863-81. • Seiler P., Sengupta R. A Bounded Real Lemma For Jump Systems. IEEE Transactions on Automatic Control, vol.48, no.9, Sept. 2003, pp.1651-4.

3. With the advent of Digital Computing Control linked to the Synchronous Model of Computation Verified (stability,safety, fairness,optimality) Syntax Semantics Execution Platform compiled relevance P. Varaiya. A question about hierarchical systems.System Theory: modeling, analysis and control, Kluwer, 2000.

4. With the advent of Digital Computing Control linked to the Synchronous Model of Computation Caspi P. Embedded Control: From Asynchrony to Synchrony and Back, EMSOFT 2001

5. With the advent of Digital Computing Control linked to the Synchronous Model of Computation Caspi P. Embedded Control: From Asynchrony to Synchrony and Back, EMSOFT 2001

6. Extend Control to Networked Environments?

7. Extend Control to Networked Environments?

8. Distributing the Synchronous Model • Both order and timing would have to be enforced across networks Cascade Composition Berry ’91 y=f(u) u=g(y) Feedback Composition: Fixpoint Semantics

9. Tension: Execution time in each round could become highly variable IEEE 802.11b Synchronous Broadcast Performance Data (DCF)

10. The Logical Order can be enforced Without Scheduling! – Zennaro, Sengupta EMSOFT’05 • Implementation problem: given a map  from RA to STS* traces we want an implementation map  such that, for all STS* s and RA r the following holds: (r = (s)  rt)  s  (t) • Modularity preservation: we seek a composition operator xRA with respect to which  is a monomorphism between (STS, xSTS) and (RA, xRA). We want this operator to be implementable across a network;

11. BDSP library cont’d

12. Time Triggered Architecture (Kopetz etal.) is a wired example Simulink to Lustre to TTA - Tripakis etal. ‘04 The token ring protocol is a wireless example (Sengupta etal. ‘04) ftp ftp ftp ftp ftp ftp In certain environments the timing can be enforced as well Token Rotation Time Rotation number

13. The DES Supervisory Control Theory Response: Control in a CSP-style Model of Computation Plant Supervisor !c1 • The sender blocks till the receiver receives • Ramadge ‘87 ?c1 !c2 X ?u1 !u1 ?u2 !u2

14. The DES Supervisory Control Theory Response: Control in a CSP-style Model of Computation Plant Supervisor Network !c1 • The rendezvous time could be variable • Not the right model for real-time computation • A real-time computation should not block for non-deterministic time ?c1 !c2 X ?u1 !u1 ?u2 Network !u2

15. The Hybrid Systems Response:A non-blocking Synchronous thread with a blocking Asynchronous thread • The hybrid system paradigm responded to this by making each component have two concurrent threads of computation • Deshpande SHIFT IEEETAC, Sifakis rtss’ 99 • All dataflows between the asynchronous process and the synchronous process are local x >= 1  !output x’=f(x) x’=g(x)

16. The Hybrid Systems Response:A non-blocking Synchronous thread with a blocking Asynchronous thread • The hybrid model semantics and execution might differ significantly if one tried to send data over a network from the synchronous thread ?output (z) x >= 1  !output (x) x’=f(x) x’=g(x) Rcv Rndzvs Ack Req rndzvs Read x Rndzvs Ack Rcv x comm delay

17. Networked Control Systems • DES/Hybrid Systems is generalizing into NCS • DES/Hybrid Model of Computation is mostly CSP-style (RPC) • Sender blocks message by message • Generalization • Kahn-style models of computation enabling a sender to stream data • Semantics explicitly accounting for network transport performance • Loss/Erasure, delay, distortion send receive

18. Networked Control Systems • DES/Hybrid Systems is generalizing into NCS • DES/Hybrid Model of Computation is mostly CSP-style (RPC) • Sender blocks message by message • Generalization • Kahn-style models of computation enabling a sender to stream data • Partially blocking sender: Obtained by TCP, messages flow FIFO, blocks when buffers full (Tilbury etal. ’01) • Synchronous non-blocking: Unreliable periodic FIFO message flow (Seiler & Sengupta etal. ‘01) • Asynchronous non-blocking: Aperiodic FIFO message flow (Hespanha etal. ’04)

19. NCS: Non-blocking synchronous senderMarkovian Jump Linear Semantics Global clock sender x x Erasure: Network delivers each message with some probability receiver Sync state estimate Sync state output Intermittent output • Estimator: An asynchronous to synchronous converter in the interior of an end-to-end synchronous system Network Estimator Controller Vehicle 1 Vehicle 2 System 2

20. Problem Setup y(k) u(k) Network A B yc(k) C Bc Ac Cc • Plant: • Network Model: • Augmented Plant:

21. The Optimal Estimator in the LQG case(Seiler, PhD thesis 2001) • Problem: Given all past measurements, {y(0),…,y(k)}, and network observations, {q(0),…,q(k)}, find the state estimate, which minimizes: • There is a separation theorem • Theorem: The optimal state estimate is given by the Kalman Filter:

22. Stability of Kalman Filter • Typically (A,C) detectable ensures that the Kalman filter is stable: M(k) stays bounded as k. For this problem, M(k) is a stochastic process: • If E[M(k)] grows unbounded as k, then the infinite time LQG cost is infinite and plant cannot be stabilized by any controller. • What conditions must the network satisfy to ensure that E[M(k)] stays bounded?

23. Stability of Kalman Filter • Theorem: Assume (A,Bw) is stabilizable. E[M(k)] grows unbounded if p r(A)2  1. If C is nonsingular then p r(A)2 < 1 is sufficient for E[M(k)] to stay bounded. • Proof: (1)Use the following matrix inequality and induction: (2) If C is nonsingular, . Use this to derive an upper bound:

24. H Conditions • Plant (P): • If the plant is stable, define the H norm as: • Theorem: If pij=pj for i,j=1,…,N then the MJLS is mean square stable and satisfies iff there exists a matrix G>0 such that:

25. Platoon Example x0 x1 d1 • Use feedback linearization and model each vehicle as: • Spacing error: • State space vehicle dynamics: • Demo controller:

26. Vehicle Following Performance vs. Packet Loss

27. Classical Information Theory • If the source is ergodic the minimal communication rate to transmit the source with an arbitrarily small probability of error is set by its entropy • Shannon 1948 • Rate-Distortion function (Shannon 1959) • The minimal bit rate R to represent a continuous alphabet source to achieve a given distortion D • Distortion is the limit of average error D= • These rates are achieved as communication delays go to infinity • We care about finite time performance

28. Literature • Classical rate-distortion theory • Infimum of the rate required to achieve given distortion • Distortion is defined in asymptotic way • Channel is not considered • Neuhoff and Gilbert, ‘82 • Problem:Causal source code • Difference • Source code, don’t consider channel • Study asymptotic property • Varaiya and Walrand ’83 • Problem: Optimal causal coding/decoding over memoryless channel • Difference • Discrete alphabet and Hamming distance • Channel with noiseless feedback • Farvardin, Vaishampayan, ’87, ’91 • Problem: Quantization over noisy channel • Difference: Consider memoryless quantization, algorithm solution to reach local optimality • Brockett and Wang, ’97 • Problem: Design coder and estimator that have stable mean square error • Difference: Bit-limited error free channel. • Nair and Evens ’97, ’98, ’00 • Problem: Causal state estimation with bit rate constraint • Difference: Perfect channel with only bit rate limitation

29. Literature • Sahai, ’00 • The highest achievable data rate (“anytime capacity”) of a channel such that the bit error probability decays with delay at given rate. • Tatikonda, ‘01 • Problem: The lower bound of rate required for controllability, observability and stability • Difference: Bit-rate constraint without channel error • Seiler, Sengupta, ‘01 • Problem: Control and estimation over wireless channel • Difference: Real erasure channel, no quantization • Tunc and Varaiya, ’02, ’03 • Problem: State estimation when the measurement is transmitted over binary symmetric channel • Difference • Look for the coder/decoder to stabilize estimation error • Noiseless channel feedback • Gastpar, Rimoldi, and Vetterli ’03 • Problem: Optimal causal source/channel coding • Difference: Consider asymptotic property • Tenekietzis, ’04 • Problem: Real-timeoptimal coding and decoding over noisy channel • Difference: Discrete alphabet source

30. The MMSE Problem Statement Given and design and to minimize the mean square error

31. Our Problem Statement Markovian System Encoder Memoryless Digital Channel Receiver Memory Update Decoder Note perfect memory case is included with

32. Structure of the optimal encoder and decoder • Theorem 1: For fixed decoder, there is no loss of optimality if one restricts attention to an encoder of the form • Encoder is a function of the current state and the probability density function of the state of the receiver • Generalization of Teneketzis’04 to continuous valued case • Theorem 2: For fixed encoder, the optimal decoder is the expectation of conditioned on all received symbols, i.e. and when receiver has perfect memory • Standard result

33. The optimal encoder has a threshold structure • Theorem 3: The optimal encoder partitions with hyperplanes • For scalar case, there is an optimal encoder that is a threshold encoder • Optimal encoder design is a finite dimensional optimization problem of dimension no greater than the channel alphabet • Extends Farvardin ’87 result from IID process to Markov process 0 1 … … X

34. Iterative algorithm for a locally optimal threshold: Scalar system, Binary symmetric channel 0 0 1- 1- 1 1 • The best threshold T, for fixed reconstruction levels, is the mid-point of R0 and R1. • The best reconstruction levels R0 and R1, for fixed threshold T, are the expectation of X conditioned on received symbol and receiver memory. • Iteratively optimize T and R0 and R1. • The MSE decreases with each iteration and therefore the algorithm converges. • Algorithm is similar to Farvardin ’87, Blahut-Arimoto ‘72 for rate-distortion functions, Lloyd-Max quantization without noisy channel

35. Conjecture • We think it may be a good idea to encode the innovation, i.e., • We examined the problem of optimal encoding of a memoryless Gaussian source

36. Memoryless Gaussian source over binary symmetric channel: Problem 0 0 1- 1- 1 1 Gaussian Random vector to be transmitted Encoder Binary Symmetric Channel Decoder For any given Q(·), the optimal decoder is conditional expectation Objective function

37. Memoryless Gaussian source over binary symmetric channel: Results • Theorem 4: • To transmit a zero-mean non-singular n-dimension Gaussian random vector X over the binary symmetric channel, the optimal encoder is a hyperplane through the origin and orthogonal to the principal component • Encode the principal component X1 0 k 1 X2 1

38. Example: 2-D Vector Case Result X1 0 k 1 X2 1

39. Control Is Still Exploring New Models Of Computation Computer Sciences Operations Research Control Sciences • Procedures of component integration are well defined for compilation • Mathematically derive system behavior from component behavior Petri Nets/ Stroboscope/ CSIM Simulink/ Esterel/ S/R, SDF Asynchronous Product CCS CSP Strongest composition and mathematical properties Hard to enforce at short time scales over large distances Weak composition and mathematical properties. Easy to realize over distances even within milliseconds