Michael Kass National Institute of Standards and Technology http://samate.nist.gov/ [email protected] Software Assurance Metrics and Tool Evaluation (SAMATE). Outline . Overview of Software Assurance (SwA) tool testing at NIST Description of SAMATE project Follow-on.
Overview of Software Assurance (SwA) tool testing at NIST
Description of SAMATE project
Do software assurance tools work as they should?
Do they really find vulnerabilities and catch bugs? How much assurance does running the tool provide?
Software Assurance tools should be:
Tested (accurate and reliable)
Develop metrics for the effectiveness of SwA tools and to identify deficiencies in software assurance methods and tools
Perform SwA R&D to assess current methods and tools in order to identify deficiencies which can lead to software product failures and vulnerabilities
Identify gaps in methods and tools and suggest areas of research
Researchers and companies
Host workshops & conference sessions
Taxonomy of SwA functions and techniques
Order of importance (cost/benefit, criticalities, …)
Gaps and research agendas
Studies to develop tool effectiveness metrics
Host reference dataset library
What constitutes a tool’s effectiveness metric?
SAMATE Project Timeline