Examining A Semantic Metrics Suite for Object-Oriented Design

1. Examining A Semantic Metrics Suite for Object-Oriented Design Dr. Letha Etzkorn (PI) Ms. Cara Stein Dr. Glenn Cox Dr. Sampson Gholston Dr. Dawn Utley Dr. Phil Farrington The University of Alabama in Huntsville

2. A Semantic Metrics Suite for Object-Oriented Design –Problem Standard software metrics have some problems! • They are implementation dependent since they are calculated strictly from the code • They count code items; sometimes it is arguable whether the items counted accurately reflect the qualities the metrics are supposed to measure • Does # of lines of code always reflect complexity? The University of Alabama in Huntsville

3. A Semantic Metrics Suite for Object-Oriented Design –Problem Segment #1 and Segment #2 do the same thing but have very different LOC metrics: Code Segment #1: test[--cnt]-> test = val1[input_count++].counter + val2[tmp_count--]->mycount; Code Segment #2: temp = val1[input_count].counter + val2[tmp_count]->mycount; input_count++; tmp_count--; --cnt; test[cnt]->test = temp; The University of Alabama in Huntsville

4. A Semantic Metrics Suite for Object-Oriented Design –Problem • These problems with software metrics all result due to the metrics being calculated on syntactic aspects of the code. • The solution is to define metrics based on semantic aspects of code (“what the code means”, the code design versus the code implementation) • Program Understanding!!! • Includes any activity that uses dynamic or static methods to reveal program properties The University of Alabama in Huntsville

5. A Semantic Metrics Suite for Object-Oriented Design –Approach Existing Etzkorn Informal Tokens approach: • Used natural language processing and information extraction techniques • Used informal tokens: comments and identifier names • Used a knowledge-base consisting of a hierarchical semantic network • Originally implemented in the PATRicia system (Program Analysis Tool for Reuse) The University of Alabama in Huntsville

6. A Semantic Metrics Suite for Object-Oriented Design –Approach Conceptual graphs, concepts in conceptual graphs infer CURSOR INSERT TEXT LOC OBJ infer Interface layer—consists of keywords tagged with the part of speech (noun, adjective, verb, etc.) The University of Alabama in Huntsville

7. A Semantic Metrics Suite for Object-Oriented Design –Approach Class Domain Complexity (CDC): • CDC = Σi=1m |concept + conceptual relations| X weight • 1 + number of conceptual relations linking the current concept to another concept recognized by the class. Concepts linking to concepts in another class are not included in the count. Only outgoing conceptual relations are included in the count (to prevent counting the same conceptual relation twice) The University of Alabama in Huntsville

8. A Semantic Metrics Suite for Object-Oriented Design -- Approach • Under the NASA grant, extended the informal tokens portion of the PATRicia system to analyze Semantic Metrics • Extended the PATRicia system to analyze software design documents • New tool is called semMet The University of Alabama in Huntsville

9. A Semantic Metrics Suite for Object-Oriented Design –Importance/Benefits Advantages of semantic metrics • Provide a seamless set of metrics from Software Design Document creation through implementation and maintenance • Provide metrics early in the software development cycle • Metrics do not vary based on code syntax alone • Metrics may provide additional insight based on measuring domain complexity rather than implementation complexity. The University of Alabama in Huntsville

10. A Semantic Metrics Suite for Object-Oriented Design –Relevance to NASA • Can be used anywhere traditional metrics are currently used, but have potential advantages compared to existing metrics • We have received data from MSFC • Plan is to examine this data using both semantic metrics and traditional metrics • Mike Chapman will analyze data from MDP using semMet The University of Alabama in Huntsville

11. A Semantic Metrics Suite for Object-Oriented Design –Accomplishments • Completed initial feature of semMet that measures semantic metrics on source code, both on class headers alone as well as entire source code • Defined several new semantic metrics in addition to those originally defined • Completed initial feature of semMet that analyzes Software Design Documents in IEEE Design Document format • Have performed some initial statistical analyses on code and Software Design Documents The University of Alabama in Huntsville

12. A Semantic Metrics Suite for Object-Oriented Design –Accomplishments – Current Publications Etzkorn, L.H., Gholston, S.E., Fortune, J.L., Stein, C.E., Utley, D., Farrington, P.A., Cox, G.W., “A Comparison of Cohesion Metrics for Object-Oriented Systems,” Information and Software Technology, Vol. 46, August, 2004, pp. 677-687 Stein,Cara Etzkorn,Letha, Cox,Glenn, Farrington,Philip, Gholston,Sampson, Utley,Dawn, and Fortune,Julie, "A New Suite of Semantic Metrics for Object-Oriented Software," 1st International Workshop on Software Audit and Metrics, Porto, Portugal, April 13-14, 2004. Stein, C. Etzkorn, L.,Utley,D., Farrington,P. Cox,G., Fortune, J. and Gholston,S. "Computing Software Metrics from Design Documents," 42nd Annual ACM Southeast Conference, Huntsville, AL, April 2-3, 2004. Stein, Cara, “Fine-Grained Semantic Metrics for Object-Oriented Software,” International Conference on Software Engineering Research and Practice (SERP ’04), Las Vegas, NV, June 21-24, 2004. (student paper) Stein, Cara, Semantic Metrics for Source Code and Design, doctoral dissertation, (successfully defended June 9, 2004). Additionally, currently have other papers submitted The University of Alabama in Huntsville

13. A Semantic Metrics Suite for Object-Oriented Design –Accomplishments – Next Steps • Continue statistical tests: • Analyze semantic metrics over additional code drawn from several different areas • Analyze data from MSFC • Analyze data from MDP (Mike Chapman) • Analyze other datasets • Analyze semantic metrics over design documents • Requires building multiple additional knowledge-bases • Examine additional new or improved semantic metrics The University of Alabama in Huntsville