Download
1 / 33
330 likes | 468 Views
Outline. Research Objectives MethodologyTowards an Automated MethodologyProcessCase Study: The Pacemaker exampleConclusions. Architectural-Level Risk Assessment Methodology at the early stages of development(S. Yacoub, H. Ammar. ISSRE'00, IEEE Comp. Soc., October, 2000)Automated Environment.
E N D
2. Outline Research Objectives
Methodology
Towards an Automated Methodology
Process
Case Study: The Pacemaker example
Conclusions
3. Architectural-Level Risk Assessment Methodology at the early stages of development(S. Yacoub, H. Ammar. ISSRE'00, IEEE Comp. Soc., October, 2000)
Automated Environment Automated Risk Assessment Research Objectives
4. Automated Risk Assessment(continued) Architectural-Level Risk Assessment Methodology (S. Yacoub, H. Ammar. ISSRE'00, IEEE Comp. Soc., October, 2000) Utilizes:
Dynamic Metrics: Component Complexity cpxi Connector Complexity cpxij (S. Yacoub, H. Ammar, and T. Robinson. Metrics'99, November 1999)
Failure Mode Effect Analysis FMEA (MIL_STD 1629A to define Component Severity svrtyi Connector Severity svrtyij)
Component Dependency Graphs CDG (adopted from: S. Yacoub, B. Cukic, and H. Ammar. ISSRE'99 November 1999)
Defines:
Heuristic Component Risk Factor hrfi = cpxi x svrtyi
Heuristic Connector Risk Factor hrfij = cpxij x svrtyij
Risk Aggregation Algorithm that produces HRFappl
5. Model the architecture of the system using simulation models (UML-RT).
Perform complexity analysis using simulation traces.
Perform severity analysis using FMEA and simulation runs.
Develop heuristic risk factors for components and connectors.
Develop Components Dependency Graph for risk assessment purposes. (System/Subsystems)
Aggregate the risk factors using the graph traversal algorithm. Automated Risk Assessment Architectural-Level Risk Assessment Methodology (continued) 6 Steps
7. Model the architecture of the system together with the risk logging capability using Rose RealTime.
Adjust the simulation runs in the observer as desired.
Run the simulation and get two log files containing:
Component complexities.
Component Execution Time.
A log of all the messages exchanged. Automated Risk Assessment Automated Environment (continued) Process
8. Process the log with Excel Risk Macro and get:
Transition Probabilities.
Connector complexities.
CDG where Risk Factors = Severity Factors * Complexity Factors (hrfi = cpxi x svrtyi )
Perform severity analysis using FMEA and simulation runs.
Traverse the CDG using the Excel traversal macro. Automated Risk Assessment Automated Environment Process (continued)
9. Example: Pacemaker Main Use Case Diagram
10. Example: Pacemaker
11. Case Study: Pacemaker (continued)
12. Case Study: Pacemaker (continued)
15. 2) Perform Complexity Analysis
16. 2) Perform Complexity Analysis (contd) A) Quantify Component Complexity Factors using dynamic complexity metrics.
17. 2) Perform Complexity Analysis (contd)
18. 2) Perform Complexity Analysis (contd) B) Quantify Connector Complexity Factors using dynamic coupling metrics.
19. 3) Perform Severity Analysis In performing severity analysis, each potential failure mode is ranked according to the consequences of that failure mode.
Steps:
Identifying Failure Modes
Failure modes of individual components. (Functional faults and state-based faults)
Failure modes of individual connectors. (Interface fault analysis)
20. 3) Perform Severity Analysis (contd) Steps (contd):
Conducting Effect Analysis
Inject the fault.
Simulate the faulty model.
Monitor output and compare to expected output.
Identify the effect of the fault.
Rank Severity
Identify category: Minor, Marginal, Critical, or Catastrophic.
Assign severity index to each component i as (svrtyi), which takes a value of 0.25, 0.50, 0.75, and 0.95
21. Worst case severity found for the RS, CD, CG, VT, and AR are Minor(0.25), Minor(0.25), Marginal(0.50), Catastrophic(0.95) and Catastrophic (0.95), respectively
23. 4) Develop Risk Factors hrfi = cpxi x svrtyi
where:
0 <= cpxi <= 1, is the normalized complexity level (dynamic complexity for components or dynamic coupling for connectors), and
0<= svrtyi < 1 , is the severity level for the architecture element.
24. 4) Develop Risk Factors (contd)
25. 5) Constructing the CDG
26. 6) Risk Aggregation Algorithm The algorithm expands all branches of the CDG starting from the start node.
The breadth expansions of the graph represent logical "OR" paths.
translated as the summation of aggregated risk factors weighted by the transition probability along each path.
The depth of each path represents the sequential execution of components:
is given by the aggregate: HRF = 1 - ?i(1-hrfi)
27. Risk Aggregation Algorithm
Procedure AssessRisk
Parameters
consumes CDG, AEappl,(average execution time for the application)
produces Riskappl
Initialization:
Rappl = Rtemp = 1 (temporary variables for (1-RiskFactor) )
Time = 0
Algorithm
push tuple <C1, hrf1, EC1 >, Time, Rtemp
while Stack not EMPTY do
pop < Ci, hrfi , ECi >, Time, Rtemp
if Time > AEappl or Ci = t; (terminating node)
Rappl += Rtemp ;(an OR path)
else
? < Cj ,hrfj , ECj > ? children(Ci)
push (<Cj, hrfj ,ECj>, Time += ECi ,
Rtemp = Rtemp*(1-hrfi)*(1-hrfij )*PTij ) ( AND path)
end
end while
Riskappl = 1- Rappl
end Procedure AssessRisk
28. Risk Aggregation Algorithm The algorithm can be used for
System-level Risk Assessment
The risk of the pacemaker that is found to be ~ 0.9
Subsystem-level Risk Comparison
Complex systems are composed of many subsystems.
The algorithm can be used to obtain a risk factor for a subsystem using risk factors of its individual components.
Compare risk factors of individual subsystems.
Sensitivity Analysis
Sensitivity to Uncertainties in Component Risk Factors
Sensitivity to Uncertainties in Connector Risk Factors
29. Sensitivity Analysis
30. Benefits The approach helps in:
Deciding which components in the architecture require more development resources.
Deciding which connectors in the architecture are of highest risk. A high risk connector indicates that the interfaces between the corresponding components and the messaging protocol should be carefully designed.
Studying how uncertainties in component risk factors affect the overall risk value of the system.
Studying how uncertainties in connector risk factors affect the overall risk value of the system.
31. Conclusion : Benefits The methodology is applicable early at the architectural level.
The methodology is based on dynamic metrics. We use dynamic metrics to account for the fact that a fault in a frequently executed component will frequently manifest itself into a failure.
The methodology is based on simulation of architecture models. Simulation helps in:
Performing FMEA procedures .
Calculating the CDG parameters such as probability of transitions.
Obtaining dynamic metrics.
32. Conclusion : Issues Using ordinal scale for measuring severity.
Effect of uncertainties in the scenario probabilities and the estimated average execution times.
Scalability issues, applying the methodology to a larger case study.
Methodology is limited to systems with statechart and sequence diagram specifications.
33. Questions...
34. Main Use Case Diagram
