Cost optimisation of analytical software quality assurance [Elektronische Ressource] / Stefan Wagner

Sujets

• Informatik

Informations

• Publié par: technische_universitat_munchen
• Publié le: 01 janvier 2007
• Nombre de lectures: 21
• Langue: English

Extrait

Cost Optimisation
of
AnalyticalSoftwareQualityAssurance
StefanWagnerInstitutfur¨ Informatik
¨ ¨derTechnischenUniversitatMunchen
Cost Optimisation
of
AnalyticalSoftwareQualityAssurance
Stefan Wagner
Vollstandiger¨ Abdruck der von der Fakultat¨ fur¨ Informatik der Technischen Universitat¨
Munchen¨ zurErlangungdesakademischenGradeseines
DoktorsderNaturwissenschaften(Dr.rer.nat.)
genehmigtenDissertation.
Vorsitzender: Univ. Prof.Dr.HelmutVeith
Prufer¨ derDissertation:
1. Univ. Prof.Dr.Dr.h.c.ManfredBroy
2. Univ. Prof.Dr.PeterLiggesmeyer,
TechnischeUniversitat¨ Kaiserslautern
Die Dissertation wurde am 01.08.2006 bei der Technischen Universitat¨ Munchen¨
eingereichtunddurchdieFakultat¨ fur¨ Informatikam14.02.2007angenommen.iiAbstract
Analytical software quality assurance (SQA) constitutes a significant part of the to
tal development costs of a software system. Most estimates say that about 50% of the
costscanbeattributedtodefect detectionandremoval. Hence,thisisapromisingarea
for cost optimisation. Various techniques can be used to improve the
quality of software. Those techniques differ in many respects and it is consolidated
knowledgethatthebestresultintermsofdefect detectioncanbeobtainedbycombin
ingseveraldiversetechniques.
The main question in that context is then how to use those different techniques in
a cost optimal way. In detail, this boils down to the questions (1) which
to use, (2) in which order, (3) with how much effort, and (4) with which distribution
overthecomponents. Themajorproblemstatedbyseveralauthorsisthatthecostsand
benefitsofSQAandtheinterrelationshipsoftheinfluencingfactorsarestillnottotally
understood.
Themaincontributionofthisdissertationisananalyticalandstochasticmodelofthe
economics of analytical SQA, based on expected values. The model is more general
thanexistingmodels,asitisabletohandledifferenttypesoftechniques,i.e.,
staticanddynamicones. Nevertheless,theproposedmodelismoreconcretethancom
parableeconomicmodelsofQAbecauseitincludesthetechnicalfactorsthatinfluence
defect detection. The model can be used (1) to analyse different techniques theoreti
callyand(2)tooptimisetheSQAinacompanyusinghistoricalprojectdata. Relevant
data for the model is collected from the literature to give average values for different
techniques and defects. This allows an easier application of the model in practice.
Based on the empirical data the model is subject to sensitivity analysis resulting in a
quantitative order of importance of the input factors. This allows to determine which
factorsaremostbeneficialtobeanalysedinmoredetail.
Finally,anapproachtoidentifydefect pronecomponentsofasystembasedonmea
sures of detailed design models allows to concentrate the SQA on these components.
Ittherebyimprovestheefficiencyofdefect detection.
The approach for cost optimisation of SQA is calibrated using several case stud
ies. A complete validation was not feasible because the model needs calibration over
several projects in a company to yield reasonable predictions. The largest case study
is an evaluation of model based testing in an industrial project in which the network
controller of an automotive network was tested. Further studies were done in vari
ous domains and evaluating different techniques in different phases. All these case
studies provided valuable feedback on the model and also contributed to the body of
knowledgeontheefficiencyofthetechniques.
iiiAbstract
ivPreface
Iwouldliketothankallthepeoplewhohelpedinmakingthisprojectasuccess. FirstI
wanttoexpressmygratitudetomysupervisorManfredBroywhohasalwaysbeenvery
supportive although my topics are not very close to his main research interests. Only
the close connections to industry and the multitude of opportunities that he offered
me made this dissertation possible. I also would like to thank my co supervisor Peter
Liggesmeyerwhogavevaluablefeedback.
Inaddition,mythanksalsogoestoallthepeopleoftheresearchgroupofProf.Broy
whichIhadthepleasuretoworkinoverthelastthreeandahalfyears. Theatmosphere
has always been friendly and supportive. Special thanks goes to Sebastian Winter,
FrankMarschall,SilkeMuller¨ ,FlorianDeißenbock¨ andCeline´ Laurent. Manydiscus
sions–notonlyaboutinformatics–havebeenveryvaluabletomeandIhopethatthis
willcontinue.
AlotofmyresearchwasinfluencedbytheworkofanddiscussionswithAlexander
Pretschner and Wolfgang Prenninger. Especially the MOST case study was mainly
their effort. Thank you! I am also grateful to Jan Romberg who has been a great
colleague. It was a pleasure to share the office with him. Further thanks go to Ul
rikeHammerschall,JanJurjens,¨ LeonidKof,ChristianPfaller,MarkusPister,Markus
Pizka,HerbertReiter,andTilmanSeifertforvaluablediscussionsandjoinedwork.
Also the work of the secretary and the IT administration at the chair made working
thereapleasure. Thankyou! IalsoliketothankallthestudentsthatIsupervised. Ithas
been an interesting experience that I nearly always enjoyed. I am especially grateful
toClaudiaKollerwhodidexcellentworkontheevaluationofbugfindingtools.
I am am very grateful to Sandro Morasca, Bev Littlewood, and Bernd Freimut who
gave valuable feedback on various stages of my work although they have not been
directly involved. Thanks goes also to Helmut Fischer and Peter Trischberger who
allowedmetodointerestingcasestudieswithindustrialdata.
My friends and family have always supported what I do and I cannot thank them
enough for that. Especially my mother and late father made me the way I am. I am
sure he would be very proud of me. I am especially grateful to Josef Stadlmaier who
improvedmyEnglishinsomechaptersofthisdissertation.
Last but not least, I would like to thank Patricia de Crignis for the wonderful time
we had together, the experiences we shared, and the encouragement she often gave to
me.
vPreface
viContents
Abstract iii
Preface v
1 Introduction 1
1.1 BackgroundandMotivation . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 SoftwareQualityAssurance 5
2.1 SoftwareQuality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 ViewsandStandards . . . . . . . . . . . . . . . . . . . . . . 5
2.1.2 Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Defects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.2 DefectTypes . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3.1 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3.2 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.3 Inspection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.4 StaticAnalysisTools . . . . . . . . . . . . . . . . . . . . . . 15
2.4 QualityModels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4.1 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.2 SoftwareReliabilityModels . . . . . . . . . . . . . . . . . . 18
2.4.3 SoftwareQualityCosts . . . . . . . . . . . . . . . . . . . . . 20
3 StateoftheArt 23
3.1 DefectIntroductionandRemoval . . . . . . . . . . . . . . . . . . . . 23
3.2 SoftwareQualityEconomics . . . . . . . . . . . . . . . . . . . . . . 24
3.3 AnalyticalModelsofQualityAssurance . . . . . . . . . . . . . . . . 25
3.3.1 Inspection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.3.2 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.3.3 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4 LiteratureReviews . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.5 Defect Proneness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
viiContents
4 QualityEconomics 31
4.1 BasicFactors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.1.1 SoftwareQualityCostsRevisited . . . . . . . . . . . . . . . 31
4.1.2 InfluencingFactors . . . . . . . . . . . . . . . . . . . . . . . 32
4.2 AnAnalyticalModel . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.2.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.2.2 ModelComponents . . . . . . . . . . . . . . . . . . . . . . . 37
4.2.3 FormsoftheDifficultyFunctions . . . . . . . . . . . . . . . 41
4.2.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.2.5 EmpiricalKnowledge . . . . . . . . . . . . . . . . . . . . . 44
4.2.6 SensitivityAnalysis . . . . . . . . . . . . . . . . . . . . . . 58
4.2.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.3 PracticalModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.3.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.3.2 Components . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3.3 EmpiricalKnowledge . . . . . . . . . . . . . . . . . . . . . 67
4.3.4 Sensitivit