GQM and data analysis Example from a Norwegian company. Tor Stålhane IDI / NTNU. The Problem. The company in question develops hardware and software . They have two software groups, each with circa 15 developers.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
GQM and data analysisExample from a Norwegian company
IDI / NTNU
The company in question develops hardware and software. They have two software groups, each with circa 15 developers.
Part of the system is developed in SDL. In order to focus their V&V work better they needed to know which SDL module characteristics that caused errors.
Possible candidates were number of module states, number of input signals etc.
In order to get a consistent and efficient data collection and analysis, we started with a GQM process.
Some of the metrics collected for eachSDL module:
When we defined the Qc question, we decided to use a Kiviat diagram to display the metrics included in this question.
We show the following data:
The large value of M6b reduces everything else to “noise”
Be ware different scales on the axis in the two diagrams
Be ware differentscales on the axisin the three diagrams
We went through all the hypothesis put forward by the developers during the GQM session. We will look at three of them:
The data for the three metrics M5, M9 and M10
were sorted according to the complexity scores
(High, Medium and Low).
An ANOVA analyses was then performed for eachdata set. We decided to require a p-value better than10%
Number of states – M5:
Source of VariationSSdfMS P-value
Between Group 1190,042595,02 0,25
Within Groups 1631,835326,37
Number of states does not contributesignificantly to the complexity asperceived by the developers.
Number of signals out – M9:
Source of VariationSSdfMS P-value
Between Group 2779,0421389,521 0,098
Within Groups 1813,835362,77
Number of signals outcontribute significantlyto the complexity as perceived by thedevelopers
Number of pages in the SDL description – M10:
Source of VariationSS df MS P-value
Between Groups 5586,04 22793,02 0,04
Within Groups 2133,83 5 426,77
Number of pages in the SDL descriptioncontribute significantly to the complexity asperceived by the developers
SDL module complexity as perceived by the
developers depends on two factors:
The other suspected factors identified during
the GQM process did not give a significant
We now have some ideas on what makes amodule look complex to the developers.
Thenext step is to see if there is anyconnectionbetween module complexity and the number oferrors in the modules.
The ANOVA can give us an answer.
Errors and complexity
Source of VariationSS dfMS P-value
Between Groups 1646,832823,42 0,06
Within Groups 770,67 5154,13
It is reasonable to assume that complexmodules have more errors.
If we look at the ANOVA summary table, we see that the differences are quite large:
GroupsCount Sum AverageVariance
Column 13102 34 343
Column 23 22 7,33 41,33
Column 32 2 1 2
Due to few observations for each complexity level, the variances are large. Thus, we should not be too categorical in our conclusions.
With all the necessary caveats in mind the companydecided as follows:
In order to reduce the number of errors we need tosingle out modules with :
The limiting values are the average values from theANOVA summary tables.