Using Natural Language Parsing (NLP) for Automated Requirements Quality Analysis

1. Using Natural Language Parsing (NLP) for Automated Requirements Quality Analysis Chris Ritter, Daniel Hettema, Steven H. Dam, Ph.D., ESEP, SPEC Innovations April 2014

2. Requirement Levels The Requirements Hierarchy A capability or feature identified by a User as being needed to perform his mission User Needs A high-level requirement generated during the concept development phase. Contained in ORD, CONOPS Conceptual Requirements Number of Requirements, Level of Detail Increases System Requirements A requirement that describes in technical language the desired capabilities of a system. Requirement that is at the level of detail needed for actually designing a new capability. Contained in System Requirements Specification (SRD) Application/Component Specifications

3. Characteristics of Good Requirements • Each individual requirement should be: • Correct: Describes the user’s true intent and is legally possible • Complete: Express a whole idea • Clear: Unambiguous and not confusing • Consistent: Not in conflict with other requirements • Verifiable: Provable (within realistic cost and schedule) that the system meets the requirement • Traceable: Uniquely identified, and able to be tracked to predecessor and successor lifecycle items/objects • Feasible: Able to be implemented with existing technology, and within cost and schedule • Modular: Can be changed without excessive impact on other requirements • Design: Does not impose a specific solution on design; says “what”, Independent not “how”

4. Avoid these Pitfalls • DON’T use ambiguous language • DON’T use bullet lists; use numbered lists instead • DON’T use jargon • DON’T use language that provides an escape clause • Ex: “The user shall be able to access the Internet as often as is practicable” • DON’T write long, rambling sentences • DON’T put two requirements in one sentence; e.g., “The system shall … and …” • DON’T use vague terms -- Ex: “user-friendly” • DON’T include suggestions or possibilities – Ex: “may”, “should”, “ought” • DON’T include wishful thinking – Ex: “The system shall be 100% reliable”

5. How Do We Ensure Quality Requirements? • Follow the “rules” for what makes a good requirement • Enforce the rule requires significant training and discipline • Most tools help “manage” the requirements, but do not help you track the quality of the requirements

6. Solution 1: Identify Requirements on Import • Innoslate’s one button Import Analyzer identifies requirements vs. statements (context for a requirements) by using a key word search (“will”, “shall”, “should”, or “requirement”) • Requirements entities have quality attributes associated with them • The user can transform a statement to a requirement and vice versa

7. Requirement Entities • Each quality factor has a Yes/No value to determine if it meets the requirement • Uncertain what attributes means? • Hover over name and definitions appear

8. Solution 2: Innoslate’s Quality Check • Run quality checker to automatically analyze the quality attributes of the requirements

9. Quality Check Uses NLP Technology • Natural Language Processing: • “a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages” • Innoslate uses this technology to break down sentences into nouns, verbs and adjectives to identify when conjunctions are used (clear), specific types of hardware/software specified (design), and other parameters that affect the requirement’s quality Wikipedia

10. Solution 3: Roll-up Quality Score • Change values from No to Yes to adjust score upward

11. Solution 4: Add Comments • Authors and reviewers can add comments and describe changes in more detail

12. Summary • We need to do more than just “manage” requirements • Innoslate brings new technology (NLP) into the automation of requirements analysis • However, it does not substitute for good engineering judgment – it is meant as a way to cue the analyst to potential problems