Generating fuzzy models from qualitative models: robustness and interpretability issues

1. Generating fuzzy models from qualitative models: robustness and interpretability issues Liliana Ironi IMATI - CNR, Pavia (Italy) Acknowledgement: Raffaella Guglielmann - Dip. Mathematics, Univ. Pavia Riccardo Bellazzi - Dip. Informatica e Sistemistica, Univ. Pavia Cesare Patrini, Andrea Nauti - Istituto FisiologiaUmana, Univ.Pavia

2. Goal A method for the construction of a robust and interpretablemodel of the nonlinear dynamics of systems characterized by incomplete knowledge and/or inadequate data set Biomedical domain

3. Case Study: Thiamine kinetics 2 x2 f32 f21 f13 f03 f01 1 x1 x3 3 Cell membrane f10 f30 u1 Plasma u2 u1, u2: signal inputs x1: Thiamine x2: Th.piro-phosphate x3: Th.mono-phosphate • chemical reactions are nonlinear and incompletely known; • linearity assumptions are too strong and turn out to fail; • poor data set: 17 samples for each chemical form

4. BUT Challenging issues • Poor generalization capabilities [Bellazzi,Guglielmann, Ironi, Patrini; 2001] • Numerically unstable Robustness Interpretability Motivations Learning methods from data, known as input-output methods, are able to reconstruct the unknown system dynamics without requiring prior knowledge • The model structure is NOT transparent • The parameters do NOT have physical meaning

5. Í Í Model : input space output space R R f : X Y X Y n A system dynamics model is given by : PROBLEM: find a nonlinear function that approximates f , given N samples • structure identification: search for in a space with good properties • parameter estimation:search for • measured computed Input-outputapproaches

6. Given M rules Fuzzy systems - I A fuzzy setF in U is characterized by: Fuzzy rule: IF x1 is F1 and …and xnis Fn THEN y is Fy Inference Engine - fuzzifier: singleton - inference rule: product - defuzzifier: center average - composition rule: summation

7. BUT To define a meaningful rule base is difficult and even impracticable The rule base is mostly built and initialized from data only Fuzzy systems - II • universal approximator • capable to deal with prior knowledge of the system dynamics through linguistic descriptions (IF - THEN) rules • better performance over Neural Networks when a meaningful rule base is available • understandable results in terms of fuzzy rule base

8. Data - driven approach Structure identification is NOT separated from parameter estimation

9. Variable partition: domain splitted into L regions; inizialized around M data, etc. No physical meaning of parameters • incomplete • inconsistent • indistinguishable Fuzzy rule generation: combination of all m’s; combinations of m’s with maximum degree on data pairs,etc. • incomplete, inconsistent • exponential grows of rules and parameters • sub-optimal structure • ill-posed problem Data - driven approach

10. IDEA: use available knowledge to build the fuzzy structure FS-QM: The need for a hybrid approach • Although incomplete, structural knowledge of underlying processes is often available • A qualitative differential model may be formulated and qualitative system dynamics may be simulated - QSIM models

11. QSIM (Kuipers, Univ. Texas - Austin, 1984) QS (t0) = < l1,> , QS(t0, t1) = <(l1 , l2), > , QS(t1) = < l2,° >,QS(t1, t2) = <(l1 , l2), > , QS(t2 ) = <l1 ,, ° > • Physical variable: • functions of time • f  C 1([a,b]) • in [a,b] a finite number of critical points Landmark values L:={lk}R ordered Time-points T:={t[a,b]| f (t)= lk , lk L} Qualitative values R QL ={lk , (lk , lk+1), k = 1, m} Qualitative state qs(f, t)=<qval(f, t),qdir(f, t)> • Constraints • arithmetic and derivative operators • functional dependencies • M+ / -, S+ / -, etc. Simulation algorithm: all of the possible behaviors are generated from QS(t0) B:= QS(t0), QS(t0, t1),…,QS(tn)

12. FS-QM approach

13. R QL nQF Assumptions: FS-QM:structure identification - I Variable partition:landmark-based

14. FS-QM: structure identification - II Dynamics representation: from behaviors to rules t = T5: IF Tht is low AND ThPPt is medium THEN ThPPt+1 is low

15. Back to Thiamine kinetics 2 x2 f32 f21 f13 f03 f01 1 x1 3 x3 Cell membrane f10 f30 u1 Plasma u2 Goal: build a fuzzy model for each subsystem

16. Construction of the fuzzy model 2. Map landmark values into fuzzy ones. Choice of . Variable u1: #QB’s #AQB’s 20 2 • Possible choice for (bounded support) • landmark triangular, etc • intervaltrapezoidal, etc Subsystem 1 Simulation Results 1. Build QDE model

17. 1. IF Tht is L and ThMPt is L and u1t is V-H THEN Tht +1 is L 2. IF Tht is L and ThMPt is L and u1t is V-H THEN Tht +1 is M - 3. IF Tht is M and ThMPt is M and u1t is V-H THEN Tht +1 is M 4. IF Tht is M and ThMPt is M and u1t is H THEN Tht +1 is M 5. IF Tht is M and ThMPt is M and u1t is M THEN Tht +1 is M - 6. IF Tht is M and ThMPt is H and u1t is M THEN Tht +1 is M 7. IF Tht is M and ThMPt is L and u1t is L THEN Tht +1 is M - 8. IF Tht is M and ThMPt is L and u1t is L THEN Tht +1 is L 9. IF Tht is M and ThMPt is M and u1t is M THEN Tht +1 is H 10. IF Tht is H and ThMPt is M and u1t is M THEN Tht +1 is M 11. IF Tht is M and ThMPt is M and u1t is L THEN Tht +1 is M 12. IF Tht is L and ThMPt is L and u1t is H THEN Tht +1 is M rules 4. Fuzzy Model 3. Map AQB’s 1. IF Tht is L and ThMPt is L and u1t is V-H THEN Tht +1 is L 2. IF Tht is M and ThMPt is M and u1t is V-H THEN Tht +1 is M 3. IF Tht is M and ThMPt is M and u1t is H THEN Tht +1 is M 4. IF Tht is M and ThMPt is H and u1t is M THEN Tht +1 is M 5. IF Tht is M and ThMPt is L and u1t is L THEN Tht +1 is L 6. IF Tht is M and ThMPt is M and u1t is M THEN Tht +1 is H 7. IF Tht is H and ThMPt is M and u1t is M THEN Tht +1 is M 8. IF Tht is M and ThMPt is M and u1t is L THEN Tht +1 is M 9. IF Tht is L and ThMPt is L and u1t is H THEN Tht +1 is M

18. FS-QM: Distinct phases Structure identificationclearly separated fromparameter estimation

19. Problem:search for Originaldata Perturbeddata (random noise o.m. 10-7) Parameter estimation problem Ill-posed nonlinear least squares problem: thesolutions do not necessarily depend continuously on data or numerically unstable Regularization techniques must be applied to get a stable solution

20. q0 inizialized with prior knowledge q* should be“close” to q0 Regularized problem FS-QM: Parameter estimation Landmark-based partitionclear physical meaning of q Each component is associated with the locations of the partitions of the variable xi , i.e. with the landmarks

21. Numerical stability or continuous dependence on data Originaldata Perturbeddata (random noise o.m. 10-7) FS-QM: Robustness Generalization properties hold independently of the shape of m’s, identified model is capable to reproduce new data set Model validation results • Complete, consistent rule base • n. of parameters linear with n.of qualitative values or partitions

22. Remark: To preserve consistency,m’s supports must be bounded FS-QM: Interpretability Fuzzy rule base: complete, consistent, and intelligible in terms of state transition rules Variable partitions: completeness, consistency, and separability are preserved

23. Initial Identified Normal condition: the identified m’s lie in their initial supports New application perspectives • Modelinterpretability in actual physical terms makes it possible to use it in a diagnostic context: • to test different diagnostic hypotheses • to evaluate deviations of parameters from their nominal values Nominal model: initial model structure of the system under normal conditions

24. IF the optimization procedure fails THEN relax the contraints on q Initial Identified Insulin-treated diabetic subjects (quasi-normal) IF another failure occurs on a larger region THEN change initialization InitialIdentified Untreated diabetic subjects New initial partition Same initial structure and new data set

25. Robustness and interpretability Conclusions • QR-based partition and fuzzy rule base construction allow us to • embed prior knowledge into the model • give a physical meaning to the parameter vector q • define a trust regionqbelongs to • Future work: • explore application potential of FS-QM, e.g. diagnosis • further improve computational efficiency

27. 1. Tentative initialization Initial Identified 2. Refined initialization Initial Identified

28. Initial model N. of Iterations: 46N. of Iterations: 36