what s the exit strategy n.
Skip this Video
Loading SlideShow in 5 Seconds..
What’s the Exit Strategy? PowerPoint Presentation
Download Presentation
What’s the Exit Strategy?

Loading in 2 Seconds...

play fullscreen
1 / 30

What’s the Exit Strategy? - PowerPoint PPT Presentation

  • Uploaded on

What’s the Exit Strategy?. Verifying a Design. Agenda – Design Verification. Concepts and implications The specification connection Verification before test Test thoroughness Tiered verification Concluding the process Summary. Verification Concepts.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'What’s the Exit Strategy?' - conner

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
what s the exit strategy

What’s the Exit Strategy?

Verifying a Design

agenda design verification
Agenda – Design Verification
  • Concepts and implications
  • The specification connection
  • Verification before test
  • Test thoroughness
  • Tiered verification
  • Concluding the process
  • Summary
verification concepts
Verification Concepts
  • Verification – Confirmation, through the provision of objective evidence, that the specified requirements have been fulfilled (Q9000-2000)
  • Confirmation – the act of ratifying or corroborating (AH)
verification implications
Verification Implications
  • The verification process is not intended to be a “craps shoot”
    • Verification is primarily intended to ratify correctness
    • Verification is NOT primarily intended to reveal incorrectness
  • The mindset is important!
    • Doubt that a design will work expresses a lack of confidence in the designer and the design process
    • Waiting until “verification” to guarantee correctness is an excuse for being sloppy
  • We should expect our designs to work! Verification simply puts the seal of confirmation on the expectation
verification implications cont
Verification Implications (cont.)
  • Verification addresses two processes
    • Design
      • Primarily a one-time, iterative process in which the developed concept is proven sound
      • Fundamental correctness can be proven analytically
        • Inspection confirms logical correctness in all circumstances
          • Peer reviews are a manual form of inspection
          • Functional simulation is an automated form of inspection
          • Inspections must be tailored to design
        • Analysis verifies correctness in the presence of variability
          • Reliability (WC, Derating, FMEA, etc.)
          • Timing over environments and age
verification implications cont1
Verification Implications (cont.)
  • Verification addresses two processes (cont.)
    • Instantiation
      • Particular instance must be sound
      • Particular correctness must be demonstrated experimentally
        • Inspection confirms that instructions are followed
          • Correct components installed
          • Correct configuration selected
          • Correct processes performed
        • Test and demonstration confirms that operation matches expectations
          • Functionally and parametrically
          • Predicted by nominal analytical predictions
verification implications cont2
Verification Implications (cont.)
  • Verification “after the fact” is too late
    • Analysis and test are NOT equivalent
    • Design analysis and inspection must proceed in lockstep with design processes
    • Implementation tests and inspections should simply confirm what is known
  • Design efforts fail because they occur in a vacuum
    • Creativity takes primacy over correctness
    • Functionality comes first with operability being “shoehorned” after the fact
      • The “monster” simulation syndrome
  • Conclusion: Verify as you proceed
verification implications cont3
Verification Implications (cont.)
  • Correctness is a matter of personal integrity for the designer
    • Verification is not simply externally imposed task
    • Verification is something that the designer must want to do (a part of his/her ethos)
    • Verification is confirmation that the designer is “worth his/her salt”
the specification connection
The Specification Connection
  • “Requirements” are confirmed during verification
    • The designer does not have a free hand with respect to how a design is confirmed
    • Specification provides the constraint set under which verification is prosecuted
    • Therefore, verification must be defined prior to start of design
      • Not simply in a general manner
      • Specifically
    • Since specification is a customer and vendor document
      • Both customer and vendor must be involved in the verification process
      • “Trust me” is not a reasonable phrase when it comes to verification
the specification connection cont
The Specification Connection (cont.)
  • Specification contains a complete [ideally] set of requirements for the design
    • Some requirements are very specific
      • “All discrete outputs shall have a source impedance of 2 kOhms.
      • An adequate test is fairly obvious
    • Some requirements are not very specific
      • “The unit shall provide 512 kbytes of SRAM”
      • Note the implied requirement – The SRAM shall function correctly
      • What is an adequate functional test for this requirement? (walking 1’s, 0’s addressing, etc?)
    • Some requirements lend themselves to quantitative analysis – “The unit shall provide a .1° C accuracy at end of life”
    • Some requirements lend themselves to simple inspection – “The unit shall provide 4 analog output channels”
    • When do we decide adequacy of the method of verification and the individual verification cases? – Before we start designing
the specification connection cont1
The Specification Connection (cont.)
  • Therefore, verification planning must begin with specification creation
    • Each requirement must be assigned a method or methods
    • Each requirement must be assigned a test or analysis case
      • Must answer question – What is an adequate verification?
      • Must establish answer to question – When are we done?
    • Each requirement must be verified at one or more levels [FPGA, board, box, sub-system, system]
    • Customer must concur with decision
  • Note that modern verification databases make this relatively straightforward
the specification connection cont2
The Specification Connection (cont.)
  • Advantages to this type of verification planning
    • More verifiable / verified designs
      • Up-front focus on programmatically difficult verification can be instituted
        • Earlier analysis
        • Simplification of overly complex circuit designs
      • Guidance for lower-level verification efforts can be established
        • Sub-module vs. module level
        • Module vs. unit level
      • Integral self-test provisions can be included in design
      • Earlier simulation vector definition (FPGAs / logic)
the specification connection cont3
The Specification Connection (cont.)
  • Advantages (cont.)
    • More robust test sets
      • Early knowledge regarding end-item function communicated
      • Capability of test set matched to need
        • Precision (timing, voltage, current, etc.)
        • Speed
        • Test level (component, unit, system)
      • Test flow design considerations included in test set design
    • Knowing verification requirements early makes the entire development process more efficient
the specification connection cont4
The Specification Connection (cont.)
  • Defining requirements and test cases at the same time as the specification clarifies and simplifies the FPGA verification process
    • Allows early start to simulation vector design
    • Creates early knowledge of additional functional models (SRAM, peripherals, etc.) needed
    • Clarifies decision regarding what can be functionally simulated and what must be simulated with back-annotation
    • Defines levels at which simulations must be run
verification before test
Verification Before Test
  • It is a relatively bad thing to discover a design flaw during test (better than nothing)
    • Late in the development cycle
    • Correction is more complicated / expensive
    • Personal stress is higher
  • Yet … a surprising lack of verification occurs early
  • Types of pre-test verification
    • Inspection - conformity evaluation by observation and judgment accompanied, as appropriate by measurement or testing (ISO)
      • Personal self-check
      • Peer Review
      • Basic functional verification (Signal audits, functional simulation, etc.)
verification before test cont
Verification Before Test (cont.)
  • Types of pre-test verification (cont.)
    • Analysis – Conformity evaluation of the predictions produced by realistic models of the system
      • Worst case analysis (voltage, current, frequency, time) using models that incorporate real world degradation of function
      • Derating analysis using models that reduce stress
      • Other
  • Note that pre-test verification is fundamentally conceptual
    • Not tool dependent
    • Not design dependent
    • Can be accomplished many ways
verification before test cont1
Verification Before Test (cont.)
  • Two approaches to pre-test verification
  • Verify full design at once (one big peer review, code walkthrough, analysis, simulation)
    • Benefits
      • Complete design reduces need to develop assumptions
      • When proved correct, does not need to be repeated
      • When design is solid, less work
      • Reduce final documentation for verification
    • Drawbacks
      • Allows small design flaws to propagate for a long time
      • Complexity reduces visibility (verification more prone to mistakes and oversights)
      • When design is flawed, it increases work
      • Leads to corrections that are global (hard to test effectively and predict side effects)
      • May take longer to execute than an aggregate of smaller verification activities
verification before test cont2
Verification Before Test (cont.)
  • Two approaches to pre-test verification (cont.)
    • Verify incremental designs
      • Benefits
        • Supports development of “known good” sub-designs
        • Meshes with a partitioned design approach
        • Facilitates visibility (fewer mistakes, clearer goals)
        • Allows small flaws to be fixed quickly (minimizes downstream impact)
      • Drawbacks
        • Is more work in the ideal case (no/few flaws)
        • Increases documentation if formality is vital
  • Either approach can be made to work, but one or the other must be chosen and implemented
test thoroughness
Test Thoroughness
  • Principles for test thoroughness
    • Every “shall” in the specification that meets the following criteria must be tested
      • Items that may be affected by the instantiation process (subject to fabrication error)
      • Items for which the only extant verification is model based
      • Items for which the testing does not pose unacceptable risk (radiation, overstress, etc.)
    • Every instance of a “shall” must be tested
      • 18 interfaces require 18 specific tests
    • Requirements that have an exclusive character
      • Must be tested for correct operation
      • Should be tested at some level for abnormal operation
      • Example: If you write a 5Ah to something to set it on, you should verify that writing other values does not set it on
    • Requirements must be tested over a representative range of conditions
test thoroughness cont
Test Thoroughness (cont.)
  • A thorough test is complicated and time consuming
    • Requires some level of automation to be comprehensive
    • Requires some level of compromise to be practical
    • Requires team agreement to be acceptable
test thoroughness cont1
Test Thoroughness (cont.)
  • Ensuring thorough, yet practical, tests
    • Define which tests require manual interaction and which can be automated
      • Output impedance or analog fine precision may need to be manual
      • Signal level or analog coarse precision may be automatable
    • Define which tests must be performed over environmental conditions and which can be performed at room temperature only
      • Make decision based on character of measurement
      • Do not decide based purely on practicality
    • Define level of complexity for all measurement sets
      • Example: 24 analog inputs (3 eight channel muxes)
      • Test full range of values for 24, or limited range for 7 of 8 mux inputs and full range for remaining
    • Define need for repetition of measurement at higher level of integration (tiered verification)
      • Example: voltage level / signal quality / clock jitter at board level
      • Repeat at box level?
test thoroughness cont2
Test Thoroughness (cont.)
  • Test thoroughness issues affect everything
    • Customer relationship
    • Test set design (board, box, system)
    • Schedule / cost
  • An adequate test program is not trivial
    • Cannot wait until system level
    • Must incorporate lowest level of design
    • Must be a constant background activity during design process
  • All programs must balance perfect and good enough
  • A good test program
    • Will ensure that the specification is met
    • Will verify everything that can be affected by fabrication
    • Will build on the analysis and inspection effort
    • Will maximize automation without sacrificing test accuracy
tiered verification
Tiered Verification
  • Tiered verification is the process of matching the correct type of verification to the appropriate level of integration
  • Tiered verification incorporates all types of verification (before and during testing)
  • Tiered verification applied to the test regime attempts to have thoroughness with practicality
    • Test a requirement at the lowest conceivable level
    • Determine which tests must be repeated at higher levels by
      • Establishing the purpose of a test at a particular level
      • Determining whether the next level has the potential to change previous measured quantities
      • Determining what test set capabilities are
  • Tiered verification cannot be retrofitted into the process
    • Must be planned up front
    • Must be implemented throughout the development process
tiered verification example
Tiered Verification - Example
  • Need: A set of 16 high-level pulsed discretes to control latching relays used to change the spacecraft power configuration
  • Basic Requirements
    • Quantity: 16
    • Level: +30V +/- 2 V
    • Load: 2 kOhm, 4 mH
    • Duration of pulse: 50 ms
    • Rise / Fall time max: 1 ms
    • Command capability: Unique bit pattern, only one at a time, arm first
    • S/W requirements: Various (beyond scope of example)
tiered verification example cont
Tiered Verification – Example (cont.)
  • Tier 1 – Basic Verification (functional simulation)
    • Verify each channel fires for a specific bit pattern / address
    • Verify cannot be fired without an arm command
    • Verify that other complementary patterns don’t accidentally fire the pulse
    • Analyze drive capability of FPGA and input / output requirement / capability of translation chip
    • Evaluate glitch hardness of logic selected
tiered verification example cont1
Tiered Verification – Example (cont.)
  • Tier 2 – Board level
    • Test to confirm signal level between FPGA and driver chips is compatible
    • Test rise, fall, duration, level with dummy load
    • Repeat test for all 16 outputs
    • Replace dummy load with relay, verify that the relay switches for all 16 outputs
    • Repeat over temperature if necessary
tiered verification example cont2
Tiered Verification – Example (cont.)
  • Tier 3 – Box level
    • Attach GSE (verifies with relays)
    • Functionally pulse all outputs
      • Verify relay switches
      • Verify gross drive duration
  • Tier 4 – Box and operational software level
    • Verify functionality commanded through software
    • GSE verifies correct relay switch
  • Tier 5 – System
    • Verify gross functionality as needed for system operation
tiered verification notes
Tiered Verification - Notes
  • Most complex and detailed functionality is verified at lower levels
    • Performance
    • Error cases
    • Predictive of future operation
  • Higher levels focus on functionality / operation
  • Lower level verification requires more manual touch
  • Higher level verification is more automated
  • STE and test procedure developed as appropriate
    • Purpose is maximum test completeness
    • As well as “bang for the buck”
concluding the process
Concluding the Process
  • (“… through the provision of objective evidence…”)
  • The importance of traceability
    • Something is only verified (formally) when it is documented
    • The tie between verification item and specification must be made (verification matrix or database)
  • The importance of planning
    • Establish the links between requirement, method, level, test case early
    • Use the established structure to develop verification items
  • The importance of buy in
    • All responsible must complete verification and report results
    • Systems engineers must track and close
    • People must keep up with the process
  • Only personal commitment from all involved will ensure that the process is successful
  • Verification is to prove success, not avoid failure
  • Verification planning is an integral part of the project lifecycle
    • Must be initiated with specification (including customer buy-in)
    • Must be included in design effort
    • Must be used to develop documentation and appropriate test hardware
  • Verification is more than final test
    • Incorporates analysis and inspection throughout design process
    • Begins at lowest possible level
    • Develops proof of compliance at all levels of operation
  • A thorough test is complicated
    • Requires extensive work
    • Must trade perfect for practical (a tiered test approach can help)
  • Verification processes must be tracked and documented
    • The whole team must buy in