Modern Optimization Methods , livre ebook
168
pages
English
Ebooks
2023
Obtenez un accès à la bibliothèque pour le consulter en ligne En savoir plus
Découvre YouScribe en t'inscrivant gratuitement
Découvre YouScribe en t'inscrivant gratuitement
168
pages
English
Ebook
2023
Obtenez un accès à la bibliothèque pour le consulter en ligne En savoir plus
Publié par
Date de parution
13 novembre 2023
Nombre de lectures
0
EAN13
9782759831753
Langue
English
Poids de l'ouvrage
6 Mo
With the fast development of big data and artificial intelligence, a natural question is how do we analyze data more efficiently? One of the efficient ways is to use optimization. What is optimization? Optimization exists everywhere. People optimize. As long as you have choices, you do optimization. Optimization is the key of operations research. This book introduces the basic definitions and theory about numerical optimization, including optimality conditions for unconstrained and constrained optimization, as well as algorithms for unconstrained and constrained problems. Moreover, it also includes the nonsmooth Newton’s method, which plays an important role in large-scale numerical optimization. Finally, based on the author’s research experiences, several latest applications about optimization are introduced, including optimization algorithms for hypergraph matching, support vector machine and bilevel optimization approach for hyperparameter selection in machine learning. With these optimization tools, one can deal with data more efficiently.
Preface..................................................... III
CHAPTER 1
Introduction................................................. 1
1.1 About Optimization ...................................... 1
1.2 Classification of Optimization ............................... 4
1.3 Preliminaries in Convex Analysis ............................ 10
1.4 Exercises............................................... 15
CHAPTER 2
Fundamentals of Optimization ................................... 17
2.1 Unconstrained Optimization Problem ......................... 17
2.2 What isa Solution? ...................................... 18
2.2.1 Definitions of Different Solutions ....................... 18
2.2.2 Recognizing a Local Minimum ......................... 20
2.2.3 Nonsmooth Problems ................................ 23
2.3 Overview of Algorithms ................................... 25
2.3.1 Line Search Strategy ................................ 26
2.3.2 Trust Region Strategy ............................... 30
2.4 Convergence ............................................ 31
2.5 Scaling................................................ 32
2.6 Exercises............................................... 33
CHAPTER 3
Line Search Methods .......................................... 35
3.1 Step Length ............................................ 35
3.1.1 The Wolfe Conditions ............................... 37
3.1.2 The Goldstein Conditions ............................ 40
3.1.3 Sufficient Decrease and Backtracking .................... 41
3.2 Convergence of Line Search Methods ......................... 42
3.3 Rate of Convergence ...................................... 44
3.3.1 Steepest Descent Method ............................. 44
3.3.2 Newton’s Method................................... 46
3.3.3 Quasi-Newton Methods .............................. 48
3.4 Exercises............................................... 50
CHAPTER 4
Trust Region Methods ......................................... 51
4.1 Outline of the Trust Region Approach ........................ 52
4.2 Algorithms Based on the Cauchy Point ........................ 54
4.2.1 The Cauchy Point .................................. 54
4.2.2 The Dogleg Method ................................. 56
4.2.3 Two-Dimensional Subspace Minimization ................. 58
4.3 Global Convergence ...................................... 59
4.3.1 Reduction Obtained by the Cauchy Point ................ 59
4.3.2 Convergence to Stationary Points....................... 61
4.4 Local Convergence ....................................... 65
4.5 Other Enhancements......................................65
4.6 Exercises...............................................68
CHAPTER 5
Conjugate Gradient Methods.................................... 69
5.1 LinearConjugate Gradient Method........................... 69
5.1.1Conjugate Direction Method .......................... 69
5.1.2 Conjugate Gradient Method........................... 72
5.1.3 A Practical Form of the Conjugate Gradient Method ........ 75
5.1.4 Rate of Convergence ................................ 76
5.1.5 Preconditioning .................................... 77
5.2 Nonlinear Conjugate Gradient Methods ....................... 78
5.2.1 The Polak-Ribiere Method and Variants.................. 80
5.2.2Global Convergence ................................. 81
5.3 Exercises............................................... 83
CHAPTER 6
Semismooth Newton’s Method ................................... 85
6.1 Semismoothness ......................................... 85
6.2 Nonsmooth Version of Newton’s Method....................... 87
6.3 Support Vector Machine ................................... 89
6.4 Semismooth Newton’s Method for SVM ....................... 91
6.5 Exercises............................................... 96
CHAPTER 7
Theory of Constrained Optimization ............................... 97
7.1 Local and Global Solutions ................................. 97
7.1.1 Smoothness ....................................... 98
7.2 Examples .............................................. 99
7.3 Tangent Cone and Constraint Qualifications .................... 103
7.4 First-Order Optimality Conditions ........................... 105
7.5 Second-Order Conditions .................................. 106
7.6 Duality................................................ 109
7.7 KKT Condition ......................................... 112
7.8 Dual Problem ........................................... 114
7.9 Exercises............................................... 118
CHAPTER 8
Penalty and Augmented Lagrangian Methods ........................ 119
8.1 The Quadratic Penalty Method.............................. 119
8.2 Exact Penalty Method .................................... 122
8.3 Augmented Lagrangian Method ............................. 123
8.4 Quadratic Penalty Method for Hypergraph Matching ............. 125
8.4.1 Hypergraph Matching ............................... 126
8.4.2 Mathematical Formulation ............................ 126
8.4.3 Relaxation Problem ................................. 128
8.4.4 Quadratic Penalty Method for (8.21) .................... 129
8.4.5 Numerical Results .................................. 130
8.5 Augmented Lagrangian Method for SVM ...................... 132
8.5.1 Support Vecotr Machine ............................. 132
8.5.2 Mathematical Formulation ............................ 133
8.5.3 Augmented Lagrangian Method (ALM) .................. 133
8.5.4 Semismooth Newton’s Method for the Subproblem.......... 136
8.5.5 Reducing the Computational Cost ...................... 137
8.5.6 Convergence Result of ALM ........................... 138
8.5.7 Numerical Results on LIBLINEAR...................... 139
8.6 Exercises............................................... 141
CHAPTER 9
Bilevel Optimization and Its Applications ........................... 143
9.1 Introduction ............................................ 143
9.2 Bilevel Model for a Case of Hyperparameter Selection in SVC ....... 145
9.2.1 An MPEC Formulation .............................. 147
9.3 The Global Relaxation Method (GRM)........................ 148
9.4 MPEC-MFCQ: A Hidden Property ........................... 149
9.5 Numerical Results........................................ 150
Bibliography................................................. 153
Publié par
Date de parution
13 novembre 2023
Nombre de lectures
0
EAN13
9782759831753
Langue
English
Poids de l'ouvrage
6 Mo
Ebooks
The Exoplanets Revolution
Ebooks
Linkage Analysis and Gene Mapping
Ebooks
A Monograph of the Genus Ulota s.l.
Ebooks
Ultra-cold atoms, ions, molecules and quantum technologies
Ebooks
Sciences formelles
Ultra-cold atoms, ions, molecules and quantum technologies
194 pages
English
Lire
Infos
Ebooks
Global Well-Posedness for Some Fluid Models
Ebooks
The basics of electron transport in spintronics
Ebooks
Sciences formelles
The basics of electron transport in spintronics
170 pages
English
Lire
Infos
Ebooks
Car following Dynamics: Experiments and Models
Ebooks
A Monograph of the genus Microtoena (Lamiaceae)
Ebooks
Science de la nature
A Monograph of the genus Microtoena (Lamiaceae)
150 pages
English
Lire
Infos
Ebooks
What is Space-Time Made of ?
Ebooks
AMETIS
Ebooks
Sign Pattern for Generalized Inverses
Ebooks
Practical PID Handbook
Ebooks
The Chinese botanical gardens
Ebooks
Synthesis and biomedical applications of magnetic nanomaterials
Ebooks
Autres
Synthesis and biomedical applications of magnetic nanomaterials
396 pages
English
Lire
Infos
Ebooks
1D Radiative Fluid and Liquid Crystal Equations
Ebooks
Sciences formelles
1D Radiative Fluid and Liquid Crystal Equations
154 pages
English
Lire
Infos
