Formal Methods

Formal Methods PowerPoint PPT Presentation


  • 188 Views
  • Uploaded on
  • Presentation posted in: General

OMSE 535 - Manny Gatlin. 2. Overview. Formal methods (requirements)"Cleanroom" developmentFuture of software testing. OMSE 535 - Manny Gatlin. 3. Formal Methods. Languages/notations for requirements"High-level"Declarative (?)ExamplesPredicate logic (Prolog) (PVS)Set/function theory (Z)Finite

Download Presentation

Formal Methods

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


1. Formal Methods OMSE 535 Lecture 10 Manny Gatlin

2. OMSE 535 - Manny Gatlin 2 Overview Formal methods (requirements) "Cleanroom" development Future of software testing

3. OMSE 535 - Manny Gatlin 3 Formal Methods Languages/notations for requirements "High-level" Declarative (?) Examples Predicate logic (Prolog) (PVS) Set/function theory (Z) Finite-state machine with temporal logic (SCR) Algebraic equations for ADT (OBJ) Tables (ad hoc) (Parnas) (Leveson) What is "high-level" changes over time - Pascal was once a "high-level design language. Most formal methods have a declarative character, but this may only be the current fashion. Tabular methods of Parnas and Leveson try to hide their mathematics to be "engineer friendly".What is "high-level" changes over time - Pascal was once a "high-level design language. Most formal methods have a declarative character, but this may only be the current fashion. Tabular methods of Parnas and Leveson try to hide their mathematics to be "engineer friendly".

4. OMSE 535 - Manny Gatlin 4 Exploring Formal Methods The dream Write requirements with user's help Compile into executable code Problems Efficiency Test/validation just shifted forward in development There might be a "requirements" phase with the user, followed by a more mathematical "specification" phase. Some formal methods are more executable than others (OBJ, Prolog: yes; Z: no). The inefficiency typically involves brute-force search methods. But parts of the system could be hand-implemented. It it any easier to test at requirements time? Without a separate implementation, development is deprived of an "orthogonal" view of the system that might expose misunderstandings.There might be a "requirements" phase with the user, followed by a more mathematical "specification" phase. Some formal methods are more executable than others (OBJ, Prolog: yes; Z: no). The inefficiency typically involves brute-force search methods. But parts of the system could be hand-implemented. It it any easier to test at requirements time? Without a separate implementation, development is deprived of an "orthogonal" view of the system that might expose misunderstandings.

5. OMSE 535 - Manny Gatlin 5 Formal Methods Today "Mathematics is good for you" Precision enhances understanding Concise notation fosters communication Proofs of specification properties General statements/questions for the user Computer assistance is required Effective for conventional testing It is said that tying one hand behind a mountain climber's back focuses his mind. Sometimes "concise" = "obscure". Not everyone is literate in mathematics. But there have been no other proposals... The use of a formal specification as effective oracle doesn't suffer from the problems associated with the Formal Methodists' dream.It is said that tying one hand behind a mountain climber's back focuses his mind. Sometimes "concise" = "obscure". Not everyone is literate in mathematics. But there have been no other proposals... The use of a formal specification as effective oracle doesn't suffer from the problems associated with the Formal Methodists' dream.

6. OMSE 535 - Manny Gatlin 6 Mills' "Cleanroom" Development Combines existing ideas... Emphasis on front-end of development Iterative enhancement Step-wise refinement ... with new ideas No developer compile or unit test "Box function" proofs Obtaining a user profile is an essential part of requirements! Proponents later backed down on "no compile" by designer/coder. Issues are the same as for inspection. But unit test remains solidly a no-no. The real technical heart of the method is Mills' "functional proof method," which is unfortunately not easy to describe. Dr.Harlan Mills, an accomplished mathematician and IBM Fellow, developed an approach to software engineering based on fundamental ideas in mathematics, statistics, and engineering. Mills offered his ideas to the software industry at large as "Cleanroom"---a term borrowed from the semi-conductor industry to reflect an emphasis on defect prevention rather than defect removal.Obtaining a user profile is an essential part of requirements! Proponents later backed down on "no compile" by designer/coder. Issues are the same as for inspection. But unit test remains solidly a no-no. The real technical heart of the method is Mills' "functional proof method," which is unfortunately not easy to describe. Dr.Harlan Mills, an accomplished mathematician and IBM Fellow, developed an approach to software engineering based on fundamental ideas in mathematics, statistics, and engineering. Mills offered his ideas to the software industry at large as "Cleanroom"---a term borrowed from the semi-conductor industry to reflect an emphasis on defect prevention rather than defect removal.

7. OMSE 535 - Manny Gatlin 7 The Argument for Cleanroom Adam's data "Most" failures result from fixes MTTF distribution must result from subjective classification of failures? The IEEE Software presentation hides the technical content...MTTF distribution must result from subjective classification of failures? The IEEE Software presentation hides the technical content...

8. OMSE 535 - Manny Gatlin 8 Cleanroom Claims 62% of faults result in 3% of failures 1% of faults result in 54% of failures MTTF improved "20 times" more by operational test than by bug fix Productivity improved 3-10 times MTTF improved by 10-100 times The faults/failures percentages are obtained from Adams' data by combining the two classes nearest each end of the MTTF distribution. Empirical productivity and MTTF claims are much disputed. Opponents say that the data includes too many projects on which no evident use of any well defined "cleanroom" method was used. In principle, claims depend on an accurate user profile.The faults/failures percentages are obtained from Adams' data by combining the two classes nearest each end of the MTTF distribution. Empirical productivity and MTTF claims are much disputed. Opponents say that the data includes too many projects on which no evident use of any well defined "cleanroom" method was used. In principle, claims depend on an accurate user profile.

9. OMSE 535 - Manny Gatlin 9 Cleanroom Process Requirements (best current practice) Formal specification (mathematical) Design ("box functions") Code (conventional) Test (operational) Fix problems (conventional) But currently only a few requirements' processes include getting the user profile! The formal method is apparently left open, but Mills had definite ideas (his "box functions") and he would probably not have approved of some methods; some would be too hard to use (e.g., formal logic), others too tied to a tool (e.g., SCR).But currently only a few requirements' processes include getting the user profile! The formal method is apparently left open, but Mills had definite ideas (his "box functions") and he would probably not have approved of some methods; some would be too hard to use (e.g., formal logic), others too tied to a tool (e.g., SCR).

10. OMSE 535 - Manny Gatlin 10 "Box Functions" Design "Black box" Conventional what-only system design "State box" Migrate black-box sequences to state Verify by reducing to a black-box form and comparing with original "Clear box" Bottom-up ADT designs from state-box Verify by reducing to state-box form and comparing with original Instead of pure input-output behaviors, Mills suggests sequences of inputs to avoid any mention of state. To do the last two steps one really has to take Mills' course. It is difficult to convey the ideas even over a whole term. Mills was always able to ignore any quantity of evidence that "functional verification" was too difficult for most people.Instead of pure input-output behaviors, Mills suggests sequences of inputs to avoid any mention of state. To do the last two steps one really has to take Mills' course. It is difficult to convey the ideas even over a whole term. Mills was always able to ignore any quantity of evidence that "functional verification" was too difficult for most people.

11. OMSE 535 - Manny Gatlin 11 Short-term Future of Testing State of the practice: Haphazard unit test Functional system test State of the art: Formal-method specification Structural-coverage adequacy Operational test In the practice, the purpose of testing is to find failures. The quality of the testing (and of the software) depends far more on the people involved and the process they use than on any technical, engineering ideas. The operational test in the art is usually only a functional test, since it proves impossible to obtain a user profile.In the practice, the purpose of testing is to find failures. The quality of the testing (and of the software) depends far more on the people involved and the process they use than on any technical, engineering ideas. The operational test in the art is usually only a functional test, since it proves impossible to obtain a user profile.

12. OMSE 535 - Manny Gatlin 12 Long-term Futures Formal requirements Inspection or automatic unit test eliminates all "typos" Operational testing determines "good enough" quality Engineering comes to software... Perhaps only safety-critical software would benefit from the best process. But everyone could pick and choose what might help. Research problems: 1. Unit-test tools that are real competition for inspection. 2. Profile estimation techniques, or profile-independent theory of random test. 3. Understanding of the fault/failure connection.Perhaps only safety-critical software would benefit from the best process. But everyone could pick and choose what might help. Research problems: 1. Unit-test tools that are real competition for inspection. 2. Profile estimation techniques, or profile-independent theory of random test. 3. Understanding of the fault/failure connection.

13. OMSE 535 - Manny Gatlin 13 Conclusions Current testing practices are not the best they can be Good testing practices are part of good engineering practices

  • Login