Testing for a Global Maximum of the Likelihood

Christophe Biernacki

Description

Testing for a Global Maximum of the Likelihood Christophe Biernacki When several roots to the likelihood equation exist, the root corresponding to the global max- imizer of the likelihood is generally retained but this procedure supposes that all possible roots are identified. Since, in many cases, the global maximizer is the only consistent root, we propose a test to detect if a given solution is consistent. This test relies on some necessary and su?cient conditions for consistency of a root and simply consists of comparing the di?erence between two expected log-likelihood expressions. Monte-Carlo studies and a real life exam- ple show that the proposed procedure leads to encouraging results. In particular, it clearly outperforms another available test of this kind, especially for relatively small sample sizes. KEY WORDS: Consistency; Maximum likelihood; Local and global maximizers; Test power. 1. INTRODUCTION In many applications where the maximum likelihood principle is involved, statisticians know that there may be multiple roots to the likelihood equation. Under standard regularity conditions, theory tells us that there is a unique consistent root to the likelihood equation (see Cramér 1946 and its multidimensional generalization in Tarone and Gruenhage 1975), but generally gives poor indication on which root is consistent in case of several roots. The review paper of Small et al. (2000) discusses various approaches for selecting among the roots (see also a discussion in Lehmann 1983, chap.

• parameter space


• global maximizer


• space ?


• likelihood


• ?n


• gj99's test


• conditions hold


• maximum likelihood

Sujets

• Ferguson
• Université de Franche-Comté
• Parameter space
• Likelihood function
• Maximum likelihood

Informations

• Publié par: profil-urra-2012
• Nombre de lectures: 29
• Langue: English

Extrait

oTCedex,estinginfomrtaotsGlobaldMaximumtheoryof1975),the(2000)LikaelihoMathematics,oapplicationsdknoChristopheoBiernaconsistenckimWhenindicationsevTheeralamongroinstanceotsthetokithe16likINTRODUCTIONelihoumovdmaequationtoexist,standardthethererotootCram?rcorrespTondingestorothesevglobalofmax-approacimizer(seeofhap.thetlikmethoelihoformoailable.dprofessor,[e-mail]-fcombutmanthistheproelihocedureissupped,osesthatthatballroplikossibleequation.royotsusareaidenrotied.likSince,equationinandmangeneralizationyandcases,generallytheoglobalwhicmaximizerisiscasetheroonlypapconsistenettvroforot,rowae1983,propincludingosefromaemplotestotoordetectwhenifforaagivChristopheenAssistansolutiontisersit?consistenhe-Comt.deThis25030testrancerelies1onInsomeynecessarywhereandmaximsucienliktoconditionsprincipleforinconsistencyolvofstatisticiansawrothereotyandesimplyultipleconsistsotsofthecomparingelihotheddierenceUnderbregularitetconditions,wtellseenthattiswuniqueotexpotectedthelog-likelihoelihodo(seed1946expressions.itsMonultidimensionalte-CarloinstudiesaroneandGruenhageabutrealgivlifepexam-orpleonshohwotthatconsistentheinpropofosederalproots.cedurereviewleadsertoSmallencouragingal.results.discussesInariousparticular,hesitselectingclearlytheoutpotserformsalsoanotherdiscussionaLehmannvcailable6),testforofiteratingthisconsistenkind,estimators,espyingeciallybforotstraprelativdelyexaminingsmallasymptoticssampleexplicitsizes.ulasKEYroWareORDS:vConsistency;AnotherMaximBiernacumisliktelihoDepartmenoofd;UnivLodecalrancandt?,globalroutemaximizers;GraT,estBesan?onpFo(E-mail:wte.fr).er.1.p
estossibilitty.isastothesimplywhenselectrothecurrenroaluatedotinleadingthistoultidimensional.theAnothermaximmethoumossiblelikeelihotheoindmatrixvvalueVsinceofWisald(1998)(1949)toestablishedaconsistencyequation.ofgivtheoglobalotmaximizer(1997)ofgotherolikeselihotheostatisticdductunderform.somehighlighconditionsextreme-v(ttedypicallyantheecomesglobalbmaximizerofisparameteraMarkrorandomotonalthoughreasonableityisofnotlikalwords,atoysshouldtrue,ofinus,particularforforitsomeisGaussianMortonmixturestoastonoticedtorsthbdyitsKieferotandandWw)olfohwitzw1956).FisherNoteparameteralsote-CarlothatofWeryald'sbasedpropasymptoticertiespofbthe(1991)maximconumhlikbelihonoofdsuppestimatorparameter(MLE)and/orareisgeneralizedconbetyoseWhitep(1982)dinotheautomaticallymorehrealistichcaseinwhereforthegivprobabilittoyomootherdelhisallomisspifecied.roSo,eexceptglobalinlikthefunction.rareiscaseslowherenewthetoMLEtilmaroyrejected.bHeydeehainconsistenosedty(seedness-of-texamplestheinotNeymankandforScottHessian1948elihoorehamorelikrecenectationtlytheinhand.Fspirit,erguson(1999)1982borhosealsodecisioninbasedStefanskibandtheCarrollof1987ectedamongoutothers),itsthethestrategyerimenwhicrestrictedhunidimensionalconsistsaofdselectingaluetheonglobalaluemaximizertheoryseemsAstooinbouteyaeallstraighintforweconometricardtext,proapproaccedurebtoimpracticalretainecauseantheadequateumroerot.computationsHothewortevtheer,spacesomelargepracticalthedicultiesspaceomccurInandtrast,watoueal.aimproptoaaddressstartingthemoininmethothebasedpresenbtotstrappapconstructer.aIndeed,searcinregion.practice,approacamasearcconsisthconstructingfortestallconsistencyroaotsencorrespotondingthetoeliholodcalInmaximizerswmasucyatakdewsconsid-decideerableatimeenandotnobguaranadoptedteeaismaximizergivtheenelihothatdallThloitcalpmaximizerstowillokhaavroeandbtesteenunfoundtheintaotnitenottime,Heydeevandenandif(1998)thevnpropumeitherbemploeraoforocriterionotsselectisbbrooundedor(seepicBarnettthe1966otforwhicantheexampleofoflog-likanounbbvoundedasymptoticallyneumexpbeveratofroroatots).InBeysameondGanthisJiangbasic(GJ99strategyshortofelosearcching,afewofpreviouswhicstudiesisareonadierencevetailable.eenFproorforminstance,theDeexpHaaninformation(1981)abproptheosedandaHessianpUnfortunately,-condenceMoninexptervtsaltheofcasetheamaximparameterumtlikveliho2o‘(?) ? ?0
? ?0
E r‘(?)=0:?0
ˆ?n
£ ⁄
r‘(?) =0:ˆ?=?n
ˆ? ‘(?)n
£ ⁄ £ ⁄
‘(?) ¡ E ‘(?) …0;ˆ ? ˆ?=? ?=?n n
£ ⁄
ˆE ‘(?) ?? n?=?0
ybulationethighlywnaturaleenularlytfollowconcludeodierenexpvectederimenlog-likparameterselihoSectionotodsomethingexpressions.handDenotingtermboyImplementoparameterorderoind.testfarthetolog-likGJ99'selihosetonewdloofnot.atermsparameterareGJ99'sestimatorswhosehtrueretainsvaalueEquations(unknomethown)yistforwtoearssimilarthe,oneittomabystudyexistandsomearevshortaluesacedureand,videddierenptcomparisonsfromotpromaximizeraw,itwhicbhthesatisfyofalsowtconsistenpresentheenowstatistic,er,basedpapathisroInwhicestimation.bparameterandultidimensionalthe(1)isThandus,mtheisproblemThroughisitthatthethereermaosedyerformsbGJ99'seammultiplemaronootsmeaningless.toorganizedtheData,liktoelihotheoteddwhereequationtation(globalismaximizer,ailable.loerimencalrealmaximizer,thenstationarySectionpaluateoinoftInandwsoroon),(aitcalmeansormelse)ultipleouldroIndeed,otsseemsmthatofothsituationinsuclefthsidethat(3)usualttheointeciallytespoftest,samethiswsimplyisiswhict.edtheondierenceemploinifelythetheofconsistenuseSo,thetuitivrecommend,totestdicultasisgoitd"(2)otInparameterorderhtoeriesdetectoththe(2)ro(3).ottingconsequence,newadAspartic-correspeasyondingapplicabilittotoaultidimensionalglobalcasesmaximizerstraighofard.sizes.expsamplets,smallappelythat,pthewideaofofpropthistestpapoutperthisisofvmethoeryAssimple:consequence,AconsiderglobalultidimensionalmaximizersituationswyouldesatisfywnotfromonlyThe(2),isbutasalsow.relativassumptionswiththeoreticalerolswbuiltotestppresenininlies2primarilyadierencepresenTheofelihotestwalsolikvt.Simtheexptotsotarodataloareconsistenproisinequation3devothewitherformanceofthetesttest.givparticular,InlastresultsaGJ99'sifaredecideen.thistheersection,ae3tpap(3)withwhereasdiscussion.aninconsistenX ;X ;:::;X n1 2 n
X f (x) f(x;?)t
? ? ?0
R
E lnf(X;?)= lnf(x;?)f (x)dx ?t t 0
Θ ?0
f(x;?)=0 ?2Θ
R
f(x;?) ? f(x;?)d·
’ (x;?) = rlnf(x;?) ’ (x;?) = lnf(x;?)¡ E lnf(X;?)1 2 ?
Pn
` (?) = ’ (X ;?)=n d (?) = E ’ (X;?) = E ` (?) j = 1;2 ’(x;?) = (’ (x;?);’ (x;?))j j i j t j t j 1 2i=1
Pn
`(?) = (` (?);` (?)) d(?) = (d (?);d (?)) ‘(?) = lnf(X ;?)1 2 1 2 ii=1
ˆ? X ;:::;X ? r‘(?)1 n n
` (?) v(?) = Var lnf(X;?) j¢j L1 ? 1
max E sup j’ (X;?)j<1 max B <1 B =E sup jr’ (X;?)jj=1;2 t j j=1;2 j j t j?2Θ ?2Θ
0<Var lnf(X;? )<1t 0
jrv(? )j<10
ˆ? ?n 0
ˆ` (? )2 n
pandtiableparameteretoesect2.1.haLethrespardswithandtiableLetdierensamewice(ftnoiswvvalueesisandanCONSTRUCTIONinoterior((c)with.sequel,pInoinalsobWeotheinlog-likTheelihotoodThefunctionofofthet.isbased2.onTHEobservwations),alleSettforwa.e..(b)as.theadensitm,ultidimensional.andmaximized.letparametric6presenparameter.tDeneonbtesteessenayrooneotandoffortheofvtoaluefamilyofonewhereasymptotic,inthewingsofolloequivthatalenuniquetlythataandroassumeotOFofTESTtheTtegralwithexptheoectedremslog-likbbindeptheoremsendentherandomundervhectorsothesisthecon(e)ergence.distributionofsahvwhicariableofvingspaceycompactfunctiona.is)isConsidertanandidenwiceisdieren-.tiableeandtdenotingwbwytheoremsunderwhictheourinreliestheantegraltialsign.anorm,.wrstegivassumenecessarythesucienfolloconditionswingconconditions:ergence(d)dFirst,densitletwusydene.spacesecondparametergivThethe(a)distributionhold:where,isconditionselihoyregularitgivthisinypendixof4varianceProalsoofossiblyoth.areFinallyen,AppdeningA.thevd(?)=(0;d (? )))?=? :2 0 0
Pˆ ˆ? ¡!? P(? 2Θ)!1n 0 n
Pˆ` (? )¡!d (? ):2 n 2 0
Pˆ? ¡!?n 0
? ¶
Var lnf(X;? )D t 0ˆ` (? )¡!N d (? ); :2 n 2 0
n
ˆ? ?n 0
f (x) f(x;?) f (x)=f(x;? )t t 0
ˆd (? ) = 0 ?2 0 n
Pˆ` (? )¡!02 n
ˆ` (? )2 n
v(? ) = Var lnf(X;? )0 ? 00
ˆ ˆ ˆv(? )=Var lnf(X;? ) ? ?n ˆ n 0 n?n
Pn 2ˆ ˆ ˆV (? ) = (lnf(X ;? )¡‘(? )=n) =nn n i n ni=1
ˆv(? )n
ˆv(? )n
ˆV (? )n n
ˆ ˆv(? ) V (? )n n n
ˆv(? ) v(? )n 0
ˆp ` (? )2 n D
nq ¡!N(0;1):
ˆv(? )n
ˆ ˆ ˆE lnf(X;? ) ` (? ) v(? )ˆ n 2 n n?n
ˆY Y f(x;? )1 m n
P Pm m 2ˆ ˆA = lnf(Y ;? )=m B = (lnf(Y ;? )¡A ) =mm i n m i n mi=1 i=1
tivt,and,alenothThenothesis.(4)evthattest.oseinequivformssuppelongsaddition,consequenInnisaresatised.proareto(a)-(d)estimatorconditionsythatinosegeneratedothesisectivSuppSupp1.fromemtestorothesis,Theonewhereoexptoectationsthat,areconestimatedWinarianceant,empiricalewmethoavya.,ApptheendixothBydiscussesdistributionproposeerties.ofnotheseintunderwhoeestimators:tobaothwingaretconsistenerformtisbut(7)somepractice,empiricalforobservergenceationsardsshoconsiderwthethatthetheandrstpone,yypestimatedhMonullifnnotasThand,w,samplehas.ahloullwwhereerandmeanaresquaredestimatederrororvemaluetheand2leads.alsotlytotestaTheorembwettererythingpourerformanceand,ofthetheulltest.ypItwishareasonableetobuildconjectureparametricthatnatural:so,estimatorstestingfolloofwconsistencyTheatheforpusesrequiredmoreaecienceduretlyNotetheininformationbandterms.testingthevmoofdelwthan.,ethat(usedconditionsthe(a)-(e)situationyvbofreplacedfunction)doanesoinbutthisitmaseemsbdiculteasilytobderivaete-Carlosomedgeneralclosedpropareertiesatoailable.compareus,morethisaccuratelyork,simplyi.i.d.isAare,.satised..whereypIfisandfrom,nthenunder(6)b2.2toAbtesttermsforesp.elyThbus,thewfamilyeofc2.hoasymptoticosevidesrproconvergenceTheoremandThenInandsituation,one(5),immediatesemi-parametricTheWthiseithaisvthatasforantoestimator1,ofeab5outd (?)2
f(x;?)
? >0 ? >