External Control in Markovian Genetic Regulatory Networks

1. External Control in MarkovianGeneticRegulatory Networks ANIRUDDHA DATTA datta@ee.tamu.edu ASHISH CHOUDHARY cdry@ee.tamu.edu MICHAEL L. BITTNER mbittner@nhgri.nih.gov EDWARD R. DOUGHER

2. Presentation Overview • Terminologies • Problem statement • Solution • Applications • Future scope • Conclusion

3. Gene Regulatory Networks

4. Probablistic Boolean Networks • Each gene, each input, and each output is represented by a node in a directed graph • Each node in the graph can be in one of two states: on or off. • x denotes activity level • ON = Gene is expressed • OFF = Gene is not expressed

5. Gene Activity Profile - GAP • activity level of gene ‘i’ at time step ‘k’ = xi(k) • Eg : • xi(k) = 0 • xi(k) = 1 • overall expression levels of all the genes = x(k) = [x1(k), x2 (k), . . . , xn(k)]

6. Transition • Expression level of the ithgene transitions according to the equation: xi(k + 1) = fj(i)(x(k)) with probability cj(i)

7. Markov Chain • AMarkov process is a random process in which the future is independent of the past, given the present. • This specific kind of "memorylessness" is called the Markov property.  • Transition Probabilities – binary to decimal • Transition Matrix

8. Controlled Markov Chain • The objective is to come up with a sequence of control inputs = Control Strategy • Cost function is minimized over the entire class of allowable control strategies.

9. Usage of PBN’s and Controlled Markov Chain for GRN • Transition probabilities between various states in PBN can be altered. • For altering = Auxiliary Variables • Auxiliary variables are called CONTROL INPUTS • Markov Theory = toggling between ON and OFF using mean first passage time

10. How to choose Control Inputs for the PBN? • To increase the likelihood that the network will transition from an undesirable state to a desirable one. • Eg: Treatment of Cancer • Control input = current state of the Therapeutic treatment (Radiation, Chemo) • ON state indicating that a particular intervention is being actively applied at that point in time • OFF state indicating that the application of that particular intervention has ceased.

11. Application : Treatment of Cancer • STEP 1 : Choose control inputs • STEP 2 : “optimally” apply one or more treatments to move the state probability distribution vector away from one which is associated with uncontrolled cell proliferation • STEP 3 : cost function is minimized over a finite number of steps (Treatment Window) • treatment is typically applied over a finite time horizon. • STEP 4: Controlling the Markov Chain • Dynamic Programming by Bellman

12. Problem Formulation : Control in PBNs

13. Solution using dynamic programming

14. A real world example based on gene expression data • The network chosen = metastatic melanoma • Metastatic competence = reduce it • Metastatic Melanoma : • The term 'metastatic melanoma', also known as stage IV melanoma, is used when melanoma cells of any kind (cutaneous, mucosal or ocular) have spread through the lymph nodes to distant sites in the body and/or to the body's organs. • Metastasis, or metastatic disease, is the spread of a cancer from one organ or part to another non-adjacent organ or part. 

15. Treatment for metastatic melanoma • The abundance of messenger RNA for the gene WNT5A = high metastatic competence • Disruption of this influence could reduce the chance of a melanoma metastasizing, a desirable outcome. • Intervention blocked the Wnt5a protein from activating its receptor • The use of an antibody that binds Wnt5a protein, could substantially reduce Wnt5a’s ability to induce a metastatic phenotype. • Objective : WNT5A is down regulated

16. Data from other Papers • Kim et al. (2002) = the WNT5A network was obtained by studying the predictive relationship between 587 genes • The expression status of each gene = TERNARY • −1 (down-regulated) • 0 (unchanged) • 1 (unregulated) A network with 587 genes will have 3587 states • COD (Coefficient of Determination) technique (Dougherty, Kim, & Chen,2000; Kim et al., 2000a, 2000b)

17. Controlling the 10-gene network using dynamic programming would require us to design a control algorithm for a system with 310(=59, 049) states

18. 7-gene network • Best two-gene predictors • Corresponding COD’s • 37 ×37matrix of transition probabilities for the Markov Chain corresponding to the dynamic evolution of the gene-activity profile of the seven gene network. • Optimal Control Problem specification : • (i) the treatment/intervention window = window of length 5 • (ii) the terminal penalty • (iii) the types of controls and the costs associated with them.

19. Treatment Penalty • zero to all states for which WNT5A equals −1, • a penalty of 3 to all states for which WNT5A equals 0 • a penalty of 6 to all states for which WNT5A equals 1.

20. Application of Control : Case 1 WNT5A Controlled Directly • Biologically such a control could be implemented by using a WNT5A inhibitory protein. • Control variable : • 0 indicating that the expression status of WNT5A has NOT been forcibly altered • 1 indicates that such a forcible alteration has taken place. • arbitrarily assigned a cost of 1 to each such forcible change and solved for the optimal control using dynamic programming.

21. Results for Case 1 • The net result was a set of optimal control inputs for each of the 2187 (=37) states at each of the five time points. • Probability of WNT5A being equal to −1 was higher with control than that without control. • WNT5A always reached −1 at the final time point

23. Case 2 : WNT5A Controlled Through pirin • Control objective is the same • Forcing pirin to −1 (corresponding to a control input of 1) or • Letting it remain wherever it is (corresponding to a control input of 0) • Control cost of 1

24. Results : Case 2 • No definite ordering of probabilities between the controlled and uncontrolled cases at the intermediate time points. • probability of WNT5A being equal to −1 at the final time point was not, in general, equal to 1 • Trying to control expression status of WNT5A using another gene and the control horizon of length 5 simply may not be adequate for achieving the desired objective with such a high probability.

26. Conclusions • Probabilistic Boolean networks with one or more control inputs. • Treatment of Cancer • Control inputs can themselves be chosen so that the genes evolve in a more “desirable fashion.” • Control can be introduced into a PBN leading to a controlled Markov Chain. • Control inputs can be optimally chosen using the Dynamic Programming technique • Published in 2003

27. QUESTIONS