# Modern Optimization Methods , livre ebook

168

pages

English

Ebooks

2023

Écrit par

Publié par

Obtenez un accès à la bibliothèque pour le consulter en ligne

Découvre YouScribe en t'inscrivant gratuitement

Découvre YouScribe en t'inscrivant gratuitement

168

pages

English

Ebook

2023

Obtenez un accès à la bibliothèque pour le consulter en ligne

With the fast development of big data and artificial intelligence, a natural question is how do we analyze data more efficiently? One of the efficient ways is to use optimization. What is optimization? Optimization exists everywhere. People optimize. As long as you have choices, you do optimization. Optimization is the key of operations research. This book introduces the basic definitions and theory about numerical optimization, including optimality conditions for unconstrained and constrained optimization, as well as algorithms for unconstrained and constrained problems. Moreover, it also includes the nonsmooth Newton’s method, which plays an important role in large-scale numerical optimization. Finally, based on the author’s research experiences, several latest applications about optimization are introduced, including optimization algorithms for hypergraph matching, support vector machine and bilevel optimization approach for hyperparameter selection in machine learning. With these optimization tools, one can deal with data more efficiently.

Preface..................................................... III

CHAPTER 1

Introduction................................................. 1

1.2 Classification of Optimization ............................... 4

1.3 Preliminaries in Convex Analysis ............................ 10

1.4 Exercises............................................... 15

CHAPTER 2

Fundamentals of Optimization ................................... 17

2.1 Unconstrained Optimization Problem ......................... 17

2.2 What isa Solution? ...................................... 18

2.2.1 Definitions of Different Solutions ....................... 18

2.2.2 Recognizing a Local Minimum ......................... 20

2.2.3 Nonsmooth Problems ................................ 23

2.3 Overview of Algorithms ................................... 25

2.3.1 Line Search Strategy ................................ 26

2.3.2 Trust Region Strategy ............................... 30

2.4 Convergence ............................................ 31

2.5 Scaling................................................ 32

2.6 Exercises............................................... 33

CHAPTER 3

Line Search Methods .......................................... 35

3.1 Step Length ............................................ 35

3.1.1 The Wolfe Conditions ............................... 37

3.1.2 The Goldstein Conditions ............................ 40

3.1.3 Sufficient Decrease and Backtracking .................... 41

3.2 Convergence of Line Search Methods ......................... 42

3.3 Rate of Convergence ...................................... 44

3.3.1 Steepest Descent Method ............................. 44

3.3.2 Newton’s Method................................... 46

3.3.3 Quasi-Newton Methods .............................. 48

3.4 Exercises............................................... 50

CHAPTER 4

Trust Region Methods ......................................... 51

4.1 Outline of the Trust Region Approach ........................ 52

4.2 Algorithms Based on the Cauchy Point ........................ 54

4.2.1 The Cauchy Point .................................. 54

4.2.2 The Dogleg Method ................................. 56

4.2.3 Two-Dimensional Subspace Minimization ................. 58

4.3 Global Convergence ...................................... 59

4.3.1 Reduction Obtained by the Cauchy Point ................ 59

4.3.2 Convergence to Stationary Points....................... 61

4.4 Local Convergence ....................................... 65

4.5 Other Enhancements......................................65

4.6 Exercises...............................................68

CHAPTER 5

5.1.1Conjugate Direction Method .......................... 69

5.1.3 A Practical Form of the Conjugate Gradient Method ........ 75

5.1.4 Rate of Convergence ................................ 76

5.1.5 Preconditioning .................................... 77

5.2 Nonlinear Conjugate Gradient Methods ....................... 78

5.2.1 The Polak-Ribiere Method and Variants.................. 80

5.2.2Global Convergence ................................. 81

5.3 Exercises............................................... 83

CHAPTER 6

Semismooth Newton’s Method ................................... 85

6.1 Semismoothness ......................................... 85

6.2 Nonsmooth Version of Newton’s Method....................... 87

6.3 Support Vector Machine ................................... 89

6.4 Semismooth Newton’s Method for SVM ....................... 91

6.5 Exercises............................................... 96

CHAPTER 7

Theory of Constrained Optimization ............................... 97

7.1 Local and Global Solutions ................................. 97

7.1.1 Smoothness ....................................... 98

7.2 Examples .............................................. 99

7.3 Tangent Cone and Constraint Qualifications .................... 103

7.4 First-Order Optimality Conditions ........................... 105

7.5 Second-Order Conditions .................................. 106

7.6 Duality................................................ 109

7.7 KKT Condition ......................................... 112

7.8 Dual Problem ........................................... 114

7.9 Exercises............................................... 118

CHAPTER 8

Penalty and Augmented Lagrangian Methods ........................ 119

8.1 The Quadratic Penalty Method.............................. 119

8.2 Exact Penalty Method .................................... 122

8.3 Augmented Lagrangian Method ............................. 123

8.4 Quadratic Penalty Method for Hypergraph Matching ............. 125

8.4.1 Hypergraph Matching ............................... 126

8.4.2 Mathematical Formulation ............................ 126

8.4.3 Relaxation Problem ................................. 128

8.4.4 Quadratic Penalty Method for (8.21) .................... 129

8.4.5 Numerical Results .................................. 130

8.5 Augmented Lagrangian Method for SVM ...................... 132

8.5.1 Support Vecotr Machine ............................. 132

8.5.2 Mathematical Formulation ............................ 133

8.5.3 Augmented Lagrangian Method (ALM) .................. 133

8.5.4 Semismooth Newton’s Method for the Subproblem.......... 136

8.5.5 Reducing the Computational Cost ...................... 137

8.5.6 Convergence Result of ALM ........................... 138

8.5.7 Numerical Results on LIBLINEAR...................... 139

8.6 Exercises............................................... 141

CHAPTER 9

Bilevel Optimization and Its Applications ........................... 143

9.1 Introduction ............................................ 143

9.2 Bilevel Model for a Case of Hyperparameter Selection in SVC ....... 145

9.2.1 An MPEC Formulation .............................. 147

9.3 The Global Relaxation Method (GRM)........................ 148

9.4 MPEC-MFCQ: A Hidden Property ........................... 149

9.5 Numerical Results........................................ 150

Bibliography................................................. 153

Publié par

Date de parution

13 novembre 2023

Nombre de lectures

0

EAN13

9782759831753

Langue

English

Poids de l'ouvrage

6 Mo

Ebooks

The Exoplanets Revolution

Lequeux James, Thérèse Encrenaz, Casoli Fabienne

Ebooks

Sciences formelles

The Exoplanets Revolution

Lequeux James, Thérèse Encrenaz, Casoli Fabienne

215 pages

English

Ebooks

Jiankang WANG, Huihui LI, Luyan ZHANG

Ebooks

Science de la nature

Jiankang WANG, Huihui LI, Luyan ZHANG

532 pages

English

Ebooks

A Monograph of the Genus Ulota s.l.

Qing-Hua WANG, Yu Jia

Ebooks

Science de la nature

A Monograph of the Genus Ulota s.l.

Qing-Hua WANG, Yu Jia

324 pages

English

Ebooks

Ultra-cold atoms, ions, molecules and quantum technologies

Héléne Perrin, Robin Kaiser, Michèle Leduc

Ebooks

Sciences formelles

Ultra-cold atoms, ions, molecules and quantum technologies

Héléne Perrin, Robin Kaiser, Michèle Leduc

194 pages

English

Ebooks

Global Well-Posedness for Some Fluid Models

Qin Yuming, Jianlin ZHANG

Ebooks

Sciences formelles

Global Well-Posedness for Some Fluid Models

Qin Yuming, Jianlin ZHANG

294 pages

English

Ebooks

The basics of electron transport in spintronics

Vincent Baltz

Ebooks

Sciences formelles

The basics of electron transport in spintronics

Vincent Baltz

170 pages

English

Ebooks

Car following Dynamics: Experiments and Models

Junfang TIAN, Jiang Rui

Ebooks

Techniques

Car following Dynamics: Experiments and Models

Junfang TIAN, Jiang Rui

160 pages

English

Ebooks

A Monograph of the genus Microtoena (Lamiaceae)

Wang Qiang

Ebooks

Science de la nature

A Monograph of the genus Microtoena (Lamiaceae)

Wang Qiang

150 pages

English

Ebooks

What is Space-Time Made of ?

David IZABEL

Ebooks

Sciences formelles

What is Space-Time Made of ?

David IZABEL

366 pages

English

Ebooks

AMETIS

Nihed CHAÂBANE, Frédéric SCHUSTER

Ebooks

Sciences formelles

AMETIS

Nihed CHAÂBANE, Frédéric SCHUSTER

382 pages

English

Ebooks

Sign Pattern for Generalized Inverses

Changjiang BU, Lizhu SUN, Yimin Wei

Ebooks

Sciences formelles

Sign Pattern for Generalized Inverses

Changjiang BU, Lizhu SUN, Yimin Wei

234 pages

English

Ebooks

Practical PID Handbook

Daniel Lequesne

Ebooks

Techniques

Practical PID Handbook

Daniel Lequesne

420 pages

English

Ebooks

The Chinese botanical gardens

Hongwen HUANG

Ebooks

Science de la nature

The Chinese botanical gardens

Hongwen HUANG

442 pages

English

Ebooks

Synthesis and biomedical applications of magnetic nanomaterials

Yanglong Hou, Yu Jing, Gao Song

Ebooks

Autres

Synthesis and biomedical applications of magnetic nanomaterials

Yanglong Hou, Yu Jing, Gao Song

396 pages

English

Ebooks

1D Radiative Fluid and Liquid Crystal Equations

Qin Yuming

Ebooks

Sciences formelles

1D Radiative Fluid and Liquid Crystal Equations

Qin Yuming

154 pages

English

• Ebooks
• Livres audio
• Presse
• Podcasts
• BD
• Documents