Mixed models based on likelihood boosting [Elektronische Ressource] / vorgelegt von Florian Reithinger

Sujets

• Mathematics

Informations

• Publié par: ludwig-maximilians-universitat_munchen
• Publié le: 01 janvier 2006
• Nombre de lectures: 14
• Langue: English
• Poids de l'ouvrage: 7 Mo

Extrait

Mixed Models based on
Likelihood Boosting
Florian Reithinger
München 2006Mixed Models based on
Likelihood Boosting
Florian Reithinger
Dissertation
an der Fakultät für Mathematik, Informatik und Statistik
der Ludwig–Maximilians–Universität
München
vorgelegt von
Florian Reithinger
aus München
München, den 28. September 2006Erstgutachter: Prof. Dr. Gerhard Tutz
Zweitgutachter: PD. Dr. Christian Heumann
Tag der mündlichen Prüfung: 20. Dezember 2006Contents
1 Introduction 1
1.1 Mixed Models and Boosting . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Guideline trough This Thesis . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Linear Mixed Models 6
2.1 Motivation: CD4 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3 The Restricted Log-Likelihood . . . . . . . . . . . . . . . . . . . . . . . 11
2.3.1 The Maximum Likelihood Method . . . . . . . . . . . . . . . . . 12
2.4 Estimation with Iteratively Weighted Least Squares . . . . . . . . . . . . 13
2.5 Estimation with EM-Algorithm - The Laird-Ware Method . . . . . . . . . 14
2.6 Robust Linear Mixed Models . . . . . . . . . . . . . . . . . . . . . . . . 16
3 Semi-Parametric Mixed Models 21
3.1 Short Review on Splines in Semi-Parametric Mixed Models . . . . . . . . 22
3.1.1 Motivation: The Interpolation Problem . . . . . . . . . . . . . . 22
3.1.2 Popular Basis Functions . . . . . . . . . . . . . . . . . . . . . . 23
3.1.3 Motivation: Splines and the Concept of Penalization - Smoothing
Splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.1.4 Motivation: The Concept of Regression Splines . . . . . . . . . . 26CONTENTS
3.1.5 Identification Problems: The Need of a Semi-Parametric Repre-
sentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.1.6 Singularities: The Need of a Regularization of Basis Functions . . 31
3.2 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3 Penalized Maximum Likelihood Approach . . . . . . . . . . . . . . . . . 33
3.4 Mixed Model Approach to Smoothing . . . . . . . . . . . . . . . . . . . 36
3.5 Boosting Approach to Additive Mixed
Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.5.1 Short Review of Likelihood Boosting . . . . . . . . . . . . . . . 37
3.5.2 The Boosting Algorithm for Mixed Models . . . . . . . . . . . . 40
3.5.3 Stopping Criteria and Selection in BoostMixed . . . . . . . . . . 42
3.5.4 Standard Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.5.5 Visualizing Variable Selection in Penalty Based Approaches . . . 45
3.6 Simulation Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.6.1 BoostMixed vs. Mixed Model Approach . . . . . . . . . . . . . 47
3.6.2 Pointwise Confidence Band for BoostMixed . . . . . . . . . . . . 55
3.6.3 Choosing an Appropriate Smoothing Parameter and an Appropri-
ate Selection Criterion . . . . . . . . . . . . . . . . . . . . . . . 59
3.6.4 Surface-Smoothing . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.7 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.7.1 CD4 data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4 Extending Semi-Structured Mixed Models to incorporate Cluster-Specific
Splines 70
4.1 General Model with Cluster-Specific Splines . . . . . . . . . . . . . . . . 72
4.1.1 The Boosting Algorithm for Models with Cluster-Specific Splines 73
4.2 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3 Application of Cluster-Specific Splines . . . . . . . . . . . . . . . . . . . 79CONTENTS
4.3.1 Jimma Data: Description . . . . . . . . . . . . . . . . . . . . . . 79
4.3.2 Jimma Data: Analysis with Cluster-Specific Splines . . . . . . . 79
4.3.3 Jimma Data: Visualizing Variable Selection . . . . . . . . . . . . 83
4.3.4 Ebay-Auctions: Description . . . . . . . . . . . . . . . . . . . . 85
4.3.5 Ebay-Data: Mixed Model Approach vs. Penalized Splines: Prog-
nostic Performance . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.3.6 Ebay Data: Final model . . . . . . . . . . . . . . . . . . . . . . 90
4.3.7 Canadian Weather Stations: Description and Model . . . . . . . . 92
5 Generalized Linear Mixed Models 95
5.1 Motivation: The European patent data . . . . . . . . . . . . . . . . . . . 95
5.2 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.3 Numerical Integration Tools . . . . . . . . . . . . . . . . . . . . . . . . 99
5.4 Methods for Crossed Random Effects . . . . . . . . . . . . . . . . . . . 104
5.4.1 Penalized Quasi Likelihood Concept . . . . . . . . . . . . . . . . 104
5.4.2 Bias Correction in Penalized Quasi Likelihood . . . . . . . . . . 109
5.4.3 Alternative Direct Maximization Methods . . . . . . . . . . . . . 110
5.4.4 Indirect Maximization using EM-Algorithm . . . . . . . . . . . . 112
5.5 Methods for Clustered Data . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.5.1 Gauss-Hermite-Quadrature . . . . . . . . . . . . . . . . . . . . . 116
5.5.2 Adaptive Gauss-Hermite Quadrature . . . . . . . . . . . . . . . . 119
5.5.3 Gauss-Hermite-Quadrature using EM-Algorithm . . . . . . . . . 122
6 Generalized Semi-Structured Mixed Models 126
6.1 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
6.1.1 The Penalized Likelihood Approach . . . . . . . . . . . . . . . . 128
6.2 Boosted Generalized Additive Mixed Models - bGAMM . . . . . . . . . 130
6.2.1 Stopping Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . 131CONTENTS
6.2.2 Simulation Study . . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.3 Application of the European Patent Data . . . . . . . . . . . . . . . . . 139
7 Summary and Perspectives 143
Appendix A:Splines 147
A.1 Solving Singularities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
A.1.1 Truncated Power Series for Semi-Parametric Models . . . . . . . 147
A.1.2 Parametrization ofα andΦ Using Restrictions . . . . . . . . . . 147
A.1.3 Parametrization ofα andΦ Using Mixed Models . . . . . . . . . 148
A.2 Smoothing with Mixed Models . . . . . . . . . . . . . . . . . . . . . . . 149
Appendix B:Parametrization of covariance structures 151
B.1 Independent Identical . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
B.2 Independent but Not Identical . . . . . . . . . . . . . . . . . . . . . . . 151
B.3 Unstructured . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Appendix C:Simulation Studies 153
C.1 Mixed Model Approach vs. BoostMixed . . . . . . . . . . . . . . . . . 153
C.2 Choosing an Appropriate Smoothing Parameter and an Appropriate Se-
lection Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
C.2.1 BIC as Selection/Stopping Criterion . . . . . . . . . . . . . . . . 166
C.2.2 AIC as Selection/Stopping Criterion . . . . . . . . . . . . . . . . 169
C.3 Linear BoostMixed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
C.4 Boosted GAMM - Poisson . . . . . . . . . . . . . . . . . . . . . . . . . 181
C.5 Boosted GLMM - Binomial . . . . . . . . . . . . . . . . . . . . . . . . 187
Bibliography 193List of Tables
3.1 Study 1: Comparison between additive mixed model fit and BoostMixed
(ρ = 0.1). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.2 Study 2: Comparison between additive mixed model fit and BoostMixed
(ρ = 0.5). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.3 Simulation study 8: Mixed Model approach vs BoostMixed . . . . . . . . 64
3.4 MACS: Estimates computed with mixed model approach and BoostMixed 69
ˆ4.1 Simulation study: estimated covariance matricesQ :=Q(ρˆ) . . . . . . . 78
4.2 Simulation study: MSE for BoostMixed vs. cluster-specific splines . . 78η
4.3 Jimma study: Effects of categorical covariates in Jimma study . . . . . . 82
4.4 Jimma study: Covariance matrix for random intercept and slope for
Jimma data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.5 Ebay study: Estimated covariance matrix . . . . . . . . . . . . . . . . . 92
4.6 Weather study: estimated covariance matrix . . . . . . . . . . . . . . . . 92
6.1 Simulation study: Generalized additive model and poisson data . . . . . . 135
6.2 Simulation stuy: Generalized linear mixed model and binomial data . . . 138
6.3 Summary statistics for the response considering small campanies . . . . . 140
6.4 Patent study: Summary statistics . . . . . . . . . . . . . . . . . . . . . . 140
6.5 Patent study: Estimated effects and variance . . . . . . . . . . . . . . . . 141
C.1 Study 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154LIST OF TABLES
C.2 Study 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
C.3 Study 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
C.4 Study 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
C.5 Study 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
C.6 Study 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
C.7 Study 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .