Managing shared resource pools for enterprise applications [Elektronische Ressource] / Daniel Jürgen Gmach

Daniel Gmach

Sujets

• Informatik
• Management

Informations

• Publié par: technische_universitat_munchen
• Publié le: 01 janvier 2009
• Nombre de lectures: 129
• Langue: English
• Poids de l'ouvrage: 1 Mo

Extrait

TechnischeUniversitätMünchen
FakultätfürInformatik
LehrstuhlIII-Datenbanksysteme
ManagingSharedResourcePoolsforEnterpriseApplications
Diplom-InformatikerUniv.
DanielJürgenGmach
VollständigerAbdruckdervonderFakultätfürInformatikderTechnischenUniversität
MünchenzurErlangungdesakademischenGradeseines
DoktorsderNaturwissenschaften(Dr.rer.nat.)
genehmigtenDissertation.
Vorsitzender: Univ.-Prof.Dr.M.Bichler
PrüferderDissertation:
1. Univ.-Prof. A.Kemper,Ph.D.
2. Adj.Prof. J.A.Rolia,Ph.D.,
CarletonUniversity,Ottawa/Kanada
DieDissertationwurdeam17.09.2008beiderTechnischenUniversitätMüncheneingereicht
unddurchdieFakultätfürInformatikam11.03.2009angenommen.Abstract
Inthepast,datacenterstypicallyconsistedofasmallnumberoflargemainframes,eachhosting
severalapplicationworkloads. Withtheadventofdistributedcomputing,newapplicationswere
assigned to their own smaller servers. Capacity planners often anticipated application demands
years in advance and equipped each new server with sufficient capacity so that the workload
could grow into it. However, the explosive growth in enterprise computing has led to server
sprawlandtoday’sdatacentersaretypicallyfulloflightlyutilizedserversthatincurgreaterman-
agement,facility,andpowercoststhannecessary. Servervirtualizationtechnologynowsupports
the creation of shared resource pools of servers. Applications can be dynamically consolidated
onto servers in such pools thereby enabling a more efficient use of resources. However, there
are challenges that must be overcome before such efficiencies can be achieved. There may be
many workloads with differing demands; each physical server can only host a finite number of
workloads; and, each workload has capacity requirements that may frequently change based on
businessneeds. Thisthesispresentsacomprehensivemanagementapproachforsharedresource
poolsthathelpstoenabletheefficientuseofserverresourceswhilemanagingtheresourceaccess
qualityofserviceofferedtoworkloads.
Themanagementapproachincludescontrollersthatoperateatthreedifferentlevels. Awork-
load placement controller globally allocates and consolidates workloads based on historical,
time-varying demands. However, chosen workload placements are based on past demands that
may not perfectly predict future demands. To improve efficiency and resource access quality of
service, a migration controller continuously observes current behavior and migrates workloads
between servers to decrease the duration of quality violations and adjust the number of servers
beingused. Finally,eachserverhasaworkloadmanagementservicethatusestime-varyingper-
workload metrics to locally optimize resource allocation on the server. Their aim is to manage
theresourceaccessqualityofserviceonserversbetweenmigrations.
A new resource pool simulator is developed that simulates application resource demands by
replaying historical resource demand traces captured from real enterprise environments. The
simulator enables the study of different management strategies. It models the placement of
workloads, simulates the competition for resources, causes controllers to execute according to
managementpolicies,anddynamicallyadjustsplacementsandconfigurations.
AcomprehensivecasestudyusingdataderivedfromarealSAPhostingcenterconcludesthe
thesis. Itaddressesseveralimportantquestions: Whatcapacitysavingsaretheoreticallypossible
from workload placement? What quality and capacity trade-off can be achieved from local and
global optimization? What quality can be achieved with per workload management? What are
theadvantagesofintegratedcontrollers? Howdoestheknowledgeofworkloaddemandpatterns
improve the effectiveness of resource pool management? To answer these questions, the case
study evaluates management policies with regard to required capacity, power usage, resource
accessqualityofservice,andmanagementefficiency. Thecasestudyshowsthatfullyintegrated
controllersoutperformtheseparateuseofcontrollersandlooseintegrations.Acknowledgments
I am most grateful to Prof. Alfons Kemper, Ph.D., for providing me the opportunity to conduct
research at the chair for Database Systems and constantly supported my work with invaluable
advice and comments. Very special thanks to Prof. Jerry Rolia, Ph.D., who always supported
and inspired me. Without his invaluable ideas and helpful comments, this work would not have
beenpossible. HealsogavemetheopportunitytovisithiminCanadaandprovidedanextremely
pleasantandconstructiveworkenvironmentduringthattime.
I would like to thank my colleagues Martina Albutiu, Stefan Aulbach, Veneta Dobreva,
Benjamin Gufler, Prof. Dr. Torsten Grust, Markus Keidl, Stefan Krompaß, Richard Kuntschke,
ManuelMayr,JessicaMüller,AngelikaReiser,TobiasScholl,AndreasScholz,MichaelSeibold,
Stefan Seltzsam, Bernhard Stegmaier, Jens Teubner, and Martin Wimmer for a very pleasant
workenvironment. SpecialthankstomycolleagueJanRittingerforsupplyinguswithicecream
andStefanAulbachwithwhomIsharedoffice. MartinaAlbutiu,StefanAulbach,SonjaGmach,
Richard Kuntschke, and Andreas Scholz supported me in proofreading parts of my thesis and
gave me valuable comments on my work. I also thank our secretary Evi Kollmann for always
supportingme.
I wish to thank John Manley, Nigel Edwards, Guillaume Belrose, and Lawrence Wilcock at
HP Labs Bristol for a great work environment. I absolutely enjoyed working with you. I also
thank Klaus Brandt, Lucy Cherkasova, and Sven Graupner. Thanks to Johannes Kirschnick and
IlangoSriramforthegreattimeinBristolandinspiringmetorunmyfirsthalfmarathon.
Many thanks to my housemates Ellie Waters and Dan Neale in Bristol and Asham El Rayes
inKanata. TheymademytimeinBristolandKanataanunforgettableexperience. Iwishyouall
thebest.
I would also like to personally thank all my friends. This includes Josef Bauer, Daniela
Blessing, Michael Dankerl, Josef Ederer, Roman Gilch, Florian Girgis, Sonja Gmach, Andrea
Gmach, Kathrin Feichtinger, Paul Holleis, Eva Holler, Frank Imsel, Verena Imsel, Benjamin
Kaiser,IrisKösters,TanjaKoller,RichardKuntschke,AiméLeonhardt,KathrinMeier,Michaeliv
Meindl, Yvonne Menzel, Elke Michlmayr, Julia Prenzyna, Mario Prenzyna, Vivien Procher,
PeterReitinger,StefanieScherzinger,CorinnaSeitz,OlliSummerer,MarcosTorresGran,Tobias
Vogl, Alexander Werni, and Thomas Zimmermann. Especially, I thank Paul Holleis, Richard
KuntschkeandThomasZimmermannforwonderfulmovieandgamenights. Veryspecialthanks
toVerenaImsel. SheandColtwerealwaystherewhenIneededhelp. Ialsowouldliketothank
myteammatesinthebadmintonclubTSVNeufahrn.
Finallyandmost deeply, Ithank mybeloved parents Marianne andFranzGmachfor always
supporting me throughout my whole life. I also sincerely thank my brother Markus and his
girlfriendTanja.
Munich,March2009 DanielGmachContents
1 Introduction 1
1.1 Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.1 CPUandMemoryAllocationtoVirtualMachines. . . . . . . . . . . . . 4
1.1.2 LiveMigrationofVirtualMachines . . . . . . . . . . . . . . . . . . . . 6
1.2 WorkloadManagementatDifferentTimeScales . . . . . . . . . . . . . . . . . . 7
1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2 RelatedWork 13
2.1 DemandPredictionandWorkloadAnalysis . . . . . . . . . . . . . . . . . . . . 13
2.2 ControlLoopsandSystemManagement . . . . . . . . . . . . . . . . . . . . . . 14
2.3 WorkloadPlacementManagement . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4 MigrationManagement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.5 WorkloadManagement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.6 IntegratedResourcePoolManagement . . . . . . . . . . . . . . . . . . . . . . . 19
3 MetricsforCapacityManagement 21
3.1 FixedandSlidingWindowTraces . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 ServiceLevelAgreements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3 FiguresofMerit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3.1 ResourceAccessQuality: ViolationPenalties . . . . . . . . . . . . . . . 23
3.3.2 ResourceAccessQuality: ResourceAccessProbability . . . . . . . . . . 25
3.3.3 ResourceAccessQuality: ResourceComplianceRatio . . . . . . . . . . 25
3.3.4 PowerEfficiency: Minimum,Maximum,andAverageWattsperInterval . 26
3.3.5 Utilization: Minimum,Maximum,andAverageNumberofServers . . . 26vi Contents
3.3.6 Utilization: ServerCPUHoursUsedandServerCPUHoursIdle . . . . . 26
3.3.7 ManagementEfficiency: NumberofMigrations . . . . . . . . . . . . . . 27
3.4 RequiredCapacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4.1 FixedWindowsandProbabilities . . . . . . . . . . . . . . . . . . . . . 27
3.4.2 SimpleSlidingWindow . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.4.3 QualityandRequiredCapacity . . . . . . . . . . . . . . . . . . . . . . . 28
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4 WorkloadDemandPrediction 31
4.1 ExtractionofWorkloadDemandPatterns . . . . . . . . . . . . . . . . . . . . . 32
4.1.1 PatternAnalysis . . . . . . . . .