Cet ouvrage et des milliers d'autres font partie de la bibliothèque YouScribe
Obtenez un accès à la bibliothèque pour les lire en ligne
En savoir plus

Partagez cette publication

BacktestingValue-at-Risk:FromDynamicQuantileto

DynamicBinaryTests

Elena-IvonaDumitrescu,

ChristopheHurlin,

andVinsonPham

February2012

Abstract

InthispaperweproposeanewtoolforbacktestingthatexaminesthequalityofValue-at-
Risk(VaR)forecasts.Todate,themostdistinguishedregression-basedbacktest,proposed
byEngleandManganelli(2004),reliesonalinearmodel.However,inviewofthedi-
chotomiccharacteroftheseriesofviolations,anon-linearmodelseemsmoreappropriate.
Inthispaperwethusproposeanewtoolforbacktesting(denoted
DB
)basedonady-
namicbinaryregressionmodel.Ourdiscrete-choicemodel,
e.g.
Probit,Logit,linksthe
sequenceofviolationstoasetofexplanatoryvariablesincludingthelaggedVaRandthe
laggedviolationsinparticular.Itallowsustoseparatelytesttheunconditionalcoverage,
theindependenceandtheconditionalcoveragehypothesesanditiseasytoimplement.
Monte-Carloexperimentsshowthatthe
DB
testexhibitsgoodsmallsampleproperties
inrealisticsamplesettings(5%coverageratewithestimationrisk).Anapplicationona
portfoliocomposedofthreeassetsincludedintheCAC40marketindexisfinallyproposed.


Keywords
:Value-at-Risk;RiskManagement;DynamicBinaryChoiceModels


J.E.LClassification
:C22,C25,C52,G28


Correspondingauthor:MaastrichtUniversityandUniversityofOrle´ans(LEO,UMRCNRS7322),Ruede
Blois,BP6739,45067Orle´ansCedex2,France.Email:elena.dumitrescu@univ-orleans.fr

UniversityofOrle´ans,(LEO,UMRCNRS7322).Email:christophe.hurlin@univ-orleans.fr.

UniversityofCaliforniaatSantaCruz(UCSA).VinsonPhambenefitedfromagrantfromtheEuropean
Program
Atlantis
AIME”ExcellenceinMobility”forhisvisitattheUniversityofOrle´ans.

1Introduction

Thereisanintenseacademicdebateonthevalidityofriskmeasuresingeneralandonthe

validityoftheValue-at-Risk(hereafterVaR)inparticular.Indeed,thisisaparticularproblem,

sincetheVaRisnotobservable,andthereforewehavetorelyupontheanalysisofthebehaviour

oftheviolationssoastotestitsvalidity.Aviolationisactuallydefinedasasituationwhere

thelossobservedex-postgoesbeyondtheex-antevalueoftheVaRinabsolutevalue.Amodel

ishencevalidiftheviolationprocesssatisfiesthemartingaledifferencehypothesis.

TherearethreemainissuesgenerallyemphasizedwhenonecomestoevaluatingVaRse-

quences.First,thepowerofthebacktestingtest,
theprobabilityofrejectingamodelthatisnot

valid
,especiallyinsmallsamples(250to500observations,or,toputitdifferently,1-2yearsof

VaRforecasts)playsakeyrole.Ithasbeenshownthatgenerallythesetestshavelowpower,as

thebacktestingprocedureistoooptimisticinthesensethatitdoesnotrejectthevalidityofa

modelasoftenasitshould(seeHurlinandTokpavi,2008).

Second,thebacktestingmethodologyhastobemodel-free.Indeed,theevaluationprocedure

mustbeimplementablewhateverthemodelusedtogeneratethesequenceofVaR,soasto

reachadiagnosticregardingthevalidityoftheVaR.Third,estimationriskmustbetakeninto

account.VaRseriescanbeestimatedusingvariousmodels,somemore,otherslesscomplicated,

withafewornumerousparameters,accordingtothespecificmethodologyofacertainfinancial

institution.TestingprocedurescanthussuccessfullyanswerthequestionofVaRvalidityonly

bytakingintoaccountestimationerror,astheriskofestimationerrorpresentintheestimates

oftheparameterspollutesVaRforecasts.Conditionalonallowingfortheseerrors,weshould

observenoparticularorientationofthediagnosticofthebacktestinthesenseofunder-rejecting

orover-rejectingtoooften.

Variousbacktestshavebeenproposedsoastosatisfythesethreerequirements(highpower,

model-free,introduceestimationrisk).Theycanbeclassifiedintofourcategories.First,in

thepioneerworksofChristoffersen(1998)thevalidityofVaRforecastsistestedthroughpa-

2
rameterrestrictionsonthetransitionprobabilitymatrixassociatedwithatwo-statesMarkov

chainmodel(violation/noviolation).Tobemoreprecise,twoassumptionsarederivedfrom

themartingaledifferencehypothesis,namelytheunconditionalcoverageandtheindependence

hypotheses.Second,testsrelyingonthedurationbetweentwoconsecutiveviolationsareput

forwardbyChristoffersenandPelletier(2004),Haas(2005)andCandelonetal.(2008)ina

likelihood-ratioframework.Atthesametime,themartingaledifferenceassumptionistested

directlybyBerkowitzetal.(2011),HurlinandTokpavi(2007)orPerignonandSmith(2008).

Lastbutnotleast,sometestsarebasedonregressionmodels(seeEngleandManganelli,2004).

ThegeneralideaistoprojectVaRviolationsontoasetofexplanatoryvariablesandsubse-

quentlytestdifferentrestrictionsontheparametersoftheregressionmodel,thatcorrespondto

theconsequencesofthemartingaledifferenceassumption.Insuchacontext,bothlinearand

non-linearregressionmodelscanbeconsidered.Forexample,therecentpaperofGaglianoneet

al.(2011)proposestoevaluatethevalidityoftheVaRbyrelyingonquantileregression,which

allowsthemtoidentifywhyandwhenaVaRmodelismisspecified.

Nevertheless,themostpopulartestofthiscategoryisEngleandManganelli’sDynamic

Quantiletest(2004),hereafter
DQ
.
1
Itconsistsintestingsomelinearrestrictionsinalinear

modelthatlinkstheviolationstoasetofexplanatoryvariables.However,thedependentvariable

isbynatureabinaryone.Itfollowsthatlinearregressionmodelsarenotthemostappropriate

choiceallowingtoinferontheparametersandconsequentlyonthehypothesisofvalidityofthe

VaR.Thelinearmodelhasseveralshortcomingsinthiscontext.Theinnovationsofthelatent

modelareassumedtofollowadiscretedistribution.Theyarealsoheteroscedasticinaway

thatdependsontheestimatedparameters.Atthesametime,constrainingtherightpartof

theregressiontothe0-1intervalimpliesnegativevariancesandnonsenseprobabilities.Still,

itistechnicallypossibletotestthesignificanceoftheslopeparametersinthecaseofabinary

dependentvariablebyrelyingonlinearmodels(seeGourieroux,2000).

InthispaperweproposeanewtoolforbacktestingVaRforecasts.LikeEngleandMan-

ganelli,weconsideraregressionmodelthatlinkstheviolationstoasetofexplanatoryvariables.
1
Notethatthe
DQ
backtestisnotrelatedtothequantileregressionmethodusedintheCAViaRmethodto
forecasttheVaR(EngleandManganelli,2004).

3
However,giventhedichotomiccharacteroftheseriesofviolations,weuseanon-linearmodel

and,morespecifically,aDynamicBinary(hereafter
DB
)regressionmodel.Theissueaddressed

inthispaperishencetheimprovementofthefinitesamplepropertiesofthebacktests,particu-

larlythepowerofthesetests,whenusingalinkfunctionthatismoreappropriateforthebinary

dimensionoftheregressand.Besides,thesenewtestsareexpectedtoberobusttoestimation

.ksir

Byproposingdynamicbinarymodels,whichrelyonrecentextensionsadvocatedinthe

EarlyWarningSystem
literature,thepotentialcorrelationbetweentheviolations(clusters)is

takenintoaccountintheestimation.Consequently,thetestsusedtoassesstheindependence

assumptionfortheviolationsandimplicitlytheonestestingtheconditionalcoveragehypothesis

areexpectedtoexhibithigherpowerthantheonespreviouslyproposedintheliterature.To

bemoreprecise,weproposesevendifferentspecifications,denotedby
DB
1
to
DB
7
,inspired
fromtheCAViaRspecificationsputforwardbyEngleandManganelli(2004).Thesubspaceof

explanatoryvariablesincludesseverallagsoftheviolationsseriesandoftheVaR,towhichthe

laggedindexisaddedinviewofthedynamicnatureofthemodels.Totesttheaccuracyofthe

VaRsequence,atwo-stepframeworkisthusimplemented.First,theseven
DB
specifications

areestimatedbyconstraintmaximum-likelihood(KauppiandSaikonnen,2008).Subsequently,

likelihood-ratiostatisticsareusedtoassessthejointsignificanceoftheparametersandthusthe

validityoftheVaR.

Notethatthistesthasseveraladvantages.First,itcanbeeasilyimplemented.Second,it

allowsustoseparatelytesttheunconditionalcoverage,theindependenceandtheconditional

coveragehypotheses.Third,Monte-Carloexperimentsshowthatbytakingintoaccountesti-

mationrisk,ourconditionalcoveragetestexhibitsgoodfinitesamplepropertiesinverysmall

samples(250observations)fora5%coveragerate.

AmainissueinVaRliteratureregardstheconsequencesofthepotentialcorrelationamongst

assetsontheconstructionofriskmeasures.WethusproposetotestthevalidityoftheVaRob-

tainedbyestimatingbothmultivariatemodels,
i.e.
modelsthattakeintoaccountthecorrelation

amongassetsandunivariatemodels,
i.e.
modelsthatdonotcareforthepossiblecorrelation

4
amongassets.Toachievethisaim,weconsideraportfolioconstitutedfromthreeassetsincluded
intheCAC40marketindexfortheperiodJune1,2007-June1,2009.Ourbacktestshows
thatthetwoapproachesleadustoriskmeasuresthatarevalidfromtheconditionalcoverage
hypothesisviewpoint.ThesefindingsgoalongthelinesofBerkowitzandO’Brien’sdiagnostic
(2002).
Therestofthispaperisorganizedasfollows.Section2presentsthetestingframework.
Insection3thebinaryregression-basedbacktestsarepresentedwhileinsection4theirsmall-
samplepropertiesaregauged.Section5revealsthemainresultsofanempiricalapplicationon
athree-assetillustrativeportfolio.

2Environmentandtestablehypotheses

Letusdenoteby
r
t
thereturnofanassetorofaportfolioattime
t
andby
VaR
t
|
t

1
(
α
)the
ex-
ante
VaRforan
α
%coveragerateforecastconditionallyonaninformationset
F
t

1
.Following
theactuarialconventionofapositivesignforaloss(seeGourierouxetal.,2000andScaillet,
2003,
interalii
),theconditionalVaRisactuallydefinedasfollows:

Pr[
r
t
<

VaR
t
|
t

1
(
α
)]=
α
,

t

Z
.

()1

Let
I
t
(
α
)bethebinaryvariableassociatedwiththe
ex-post
observationofan
α
%VaRviolation
attime
t
,
i.e.
:



1if
r
t
<

VaR
t
|
t

1
(
α
)
I
t
(
α
)=.


0otherwise

)2(

AsstressedbyChristoffersen(1998),VaRforecastsarevalidifandonlyiftheviolationprocess
I
t
(
α
)satisfiesthefollowingtwohypotheses:


Theunconditionalcoverage(UCthereafter)hypothesis:theprobabilityofan
ex-post
return
exceedingtheVaRforecastmustbeequaltothe
α
coveragerate,

Pr[
I
t
(
α
)=1]=
E
[
I
t
(
α
)]=
α
.

5(3)


Theindependence(INDthereafter)hypothesis:VaRviolationsobservedattwodifferent

datesforthesamecoverageratemustbedistributedindependently.Formally,thevari-

able
I
t
(
α
)associatedwithaVaRviolationattime
t
foran
α
%coveragerateshouldbe

independentofthevariable
I
t

k
(
α
),

k
6
=0.Inotherwords,pastVaRviolationsshould

notbeinformativeaboutcurrentandfutureviolations.

TheUChypothesisisastraightforwardone.Indeed,ifthefrequencyofviolationsobserved

over
T
periodsissignificantlylower(respectivelyhigher)thanthecoveragerate
α
,thenthe

VaRmodeloverestimates(respectivelyunderestimates)thetruelevelofrisk.However,the

UChypothesisshadesnolightonthepossibledependenceofVaRviolations.Therefore,the

independencepropertyofviolationsisanessentialone,becauseitisrelatedtotheabilityofa

VaRmodeltoaccuratelymodelthehigher-orderdynamicsofreturns.Infact,amodelwhich

doesnotsatisfytheindependencepropertycanleadtoclusteringsofviolations(foragiven

period)evenifithasthecorrectaveragenumberofviolations.Consequently,theremustbeno

dependenceintheviolationsvariable,whateverthecoveragerateconsidered.

WhentheUCandINDhypothesesaresimultaneouslyvalid,VaRforecastsaresaidtohave

acorrectconditionalcoverage(CCthereafter),andtheVaRviolationprocessisamartingale

difference:

E
[
I
t
(
α
)
|F
t

1
]=
α.

(4)

ThislastpropertyisatthecoreofmostofthebacktestsforVaRmodelsavailableintheliterature

(Christoffersen,1998;EngleandManganelli,2004;Berkowitzetal.,2011;etc.).

Actually,thisbacktestingapproach,basedontheUC,INDandCCassumptions,canbecon-

sideredasaspecialcaseoftheeventprobabilityforecastevaluationandcanbeimplemented

throughfourmaintypesoftests.Tobemoreexact,thefirstbacktestingtests(Kupiec,1995;

Christoffersen,1998)werebasedonaMarkovchaintypemodelwithtwostates(violation/no

violation).Inthisframework,theUC,INDandCCassumptionsaresimplytestedthrough

parameterrestrictionsonthetransitionprobabilitymatrixassociatedwiththeMarkovrepre-

sentation.Othertests(ChristoffersenandPelletier,2004;Haas,2005;Candelonetal.,2008)

relyonthedurationbetweentwoconsecutiveviolations.UndertheCChypothesis,thedura-

6
tionvariablefollowsageometricdistributionwithparameter
α
.Exploitingthisproperty,itis

straightforwardtodevelopalikelihoodratio(
LR
)testforthenullhypothesisofconditional

coverage.Thegeneralideaconsistsinspecifyingalifetimedistributionthatneststhegeometric,

sothatthememorylesspropertycanbetestedbymeansof
LR
tests.Third,someothertests

checkthemartingaledifferenceassumptiondirectly,throughitsmaincorrelationbasedimpli-

cations.Itis,forinstance,thecaseofBerkowitzetal.(2011),HurlinandTokpavi(2007)or

PerignonandSmith(2008).Finally,sometestsarebasedonregressionmodels.Inthiscontext,

themostpopularbacktestingtest,oftencalled
DQ
(DynamicQuantile)test,wasproposedby

EngleandManganelli(2004).Itisbasedonalinearregressionmodelofthehitsvariableona

setofexplanatoryvariablesincludingaconstant,thelaggedvaluesofthehitvariable,andany

functionofthepastinformationsetsuspectedofbeinginformative.
Moreformally,letusdenoteby
Hit
t
(
α
)=
I
t
(
α
)

α
thedemeanedprocessofviolation,
thattakesthevalue1

α
everytimes
r
t
islessthanthe
ex-ante
VaRand

α
otherwise.From
thedefinitionoftheVaR,theconditionalexpectationof
Hit
t
(
α,
)giventheinformationknown
at
t

1mustbezero.Inparticular,undertheCCassumption,thevariable
Hit
t
(
α
)must
beuncorrelatedwithitsownlaggedvaluesandwithanyotherlaggedvariable(includingpast

returns,pastVaR,etc.),anditsexpectedvaluemustbeequaltozero.Toputitanotherway,

theCCassumptioncanbetestedinthefollowinglinearregressionmodel:

(5)

KXHit
t
(
α
)=
δ
+
β
k
Hit
t

k
(
α
)
1=kKX+
γ
k
g
[
Hit
t

k
(
α
)
,Hit
t

k

1
(
α
)
,...,z
t

k
,z
t

k

1
,...
]+
ε
t
,
1=k

where
ε
t
isadiscrete
i.i.d.
processandwhere
g
(
.
)isafunctionofpastviolationsandofthe
variables
z
t

k
belongingtotheentireinformationalsetavailable
F
t

1
.Tobemoreexact,we
canconsidervariableslikepastreturns
r
t

k
,thesquareofpastreturns
r
t
2

k
,thepredicted
Value-at-Risk
VaR
t

k
|
t

k

1
(
α
)orevenimplicitvolatilitydata.Nevertheless,testingforthenull
hypothesisofconditionalefficiencyisequivalenttotestingthejointnullityofthecoefficients
β
k
,
γ
k
,

k
=1
,..,K,
andthatoftheintercept
δ
,independentoftheVaRmodelspecification

7
considered:
H
0
:
δ
=
β
1
=
...
=
β
k
=
γ
1
=
...
=
γ
k
=0
,

k
=1
,..,K

(6)

Atthesametime,presentviolationsoftheVaRarenotcorrelatedwithpastviolationsif
β
1
=
...
=
β
k
=
γ
1
=
...
=
γ
k
=0(asanimplicationoftheindependencehypothesis),whilethe
unconditionalcoveragehypothesisisfulfillediftheconstant
δ
isnull.Indeed,underthenull
hypothesis,
E
[
Hit
t
(
α
)]=
E
(
ε
t
)=0
,
whichmeansthatbydefinitionPr[
I
t
(
α
)=1]=
E
[
I
t
(
α
)]=
α.
Allinall,itresultsthatthetestofjointnullityofallcoefficientscorrespondstoaconditional
efficiencytest.
0Therefore,ifwedenotebyΨ=(
δ,β
1
,..,β
K

1
,..,γ
K
)thevectorofthe2
K
+1parameters
ofthemodelandby
Z
thematrixofexplanatoryvariablesofmodel(5),theteststatistic
designatedby
DQ
CC
andassociatedwiththeconditionalcoveragetest
2
satisfiesthefollowing
relation:

Ψ
b
0
Z
0
Z
Ψ
b
L
2
DQ
CC
=
α
(1

α
)
T




χ
(2
K
+1)

)7(

SimilartoChristoffersen’stest,the
DQ
testcanbedecomposedsothatwecanverify,for
example,onlythehypothesisofindependenceofviolations.Theteststatistic
DQ
IND
associated
withtheindependencehypothesis
H
0
:
β
k
=
γ
k
=0satisfies:

Ψ
b
0
R
0

R
(
Z
0
Z
)

1
R
0


1
R
Ψ
b
L
2DQ
IND
=
α
(1

α
)
T




χ
(2
K
)

)8(

0where
R
=[0:
I
2
K
]isa(2
K
+1
,
2
K
)matrixsothat
R
Ψ=
β,
where
β
=(
β
1
,..,β
K

1
,..,γ
K
)
.

3ADynamicBinaryResponsemodel

Asaforementioned,EngleandManganelli(2004)implementatestbasedonasimplelinearre-
gressionwhichlinksthecurrenthit(violation)tothelaggedviolations.Nevertheless,itiswell
knownthatthistypeofmodelisnotsuitablefordichotomousdependentvariables(Gourieroux,
2000)sincetheresidualsfollowadiscretedistributionandtheyareheteroscedasticbyconstruc-
2
Underthenullhypothesis,theresiduals
ε
t
correspondtotheviolationprocess
Hit
t
(
α
),whichfollowsa
Bernouillidistributionofparameter
α
andvariance
α
(1

α
)
.

8
tion.Additionally,thescatterplotoftheobservationscannotbecorrectlyadjustedwithasingle
lineasinthecaseofcontinuousvariables.
Asolutionconsistsinusingnon-linearregressionmodelssuitedforlimited-dependentvari-
ables.Indeed,weproposeadichotomicmodel(logit,probit,cloglog,...)toestablishalink
betweenthecurrentviolationsofVaRandthesetofexplanatoryvariables.Weactuallyexpect
thattheuseofthisappropriatenon-linearmethodologyforbinarydependentvariablesinthe
backtestingregressionwillimprovethefinitesamplepropertiesofthebacktestingtests.
ItshouldbenotedthatthebacktestingVaRtopicresemblesatdifferentlevelsthe
Early
WarningSystems
methodologywhichaimsatforecastingfinancialcrisesorrecessions.First,
theobjectofthestudyhasthesameform,sinceinbothcasesthedependentvariableisbinary,
e.g.
thelossisgreaterthentheVaRattime
t
ornot;thereisacrisisinacertainperiodor
not.Itfollowsthataspecialeconometricmethodologymustbeimplementedsoastoobtain
consistentestimators.Furthermore,bothevents(lossorcrisis)canbepersistentasitisnot
alwayspossibletoshiftfromlosstoprofitorfromcrisistocalminjustoneperiod.Thisdynamics
shouldbetakenintoaccountinaVaRbacktestregressionmodelsasithasbeenrecentlydonein
EarlyWarningSystems
byKauppiandSaikonnen(2008).Therefore,weexplicitlyimplementa
dynamicmodelwhosemainadvantageisthatitallowsustotestbothhypothesesofindependence
andconditionalcoverage.
Inthispartofthepaperournewmethodologybasedonadynamicbinaryresponsemodel
isintroduced.First,thedifferentmodelspecificationsconsideredarepresented.Then,the
estimationmethodisdevelopedandfinallythebacktestingteststatisticsareintroduced.

3.1ModelSpecification

Letusconsiderthefollowingdynamicbinaryresponsemodel,inwhichtheconditionalprobability
ofviolationattime
t
isgivenby:

Pr[
I
t
(
α
)=1
|F
t

1
]=
E
[
I
t
(
α
)
|F
t

1
]=
F
(
π
t
)
.

9(9)

where
F
(
.
)denotesac.d.f.(whichdoesnotnecessarilycorrespondtothedistributionofthe
returns)and
F
t

1
isasetofinformationavailableat
t

1.Weassumethattheindex
π
t
satisfies
thefollowingautoregressiverepresentation:

q
X
1
q
X
2
q
X
3
q
X
4
π
t
=
c
+
β
j
π
t

j
+
δ
j
I
t

j
(
α
)+
ψ
j
l
(
x
t

j

)+
γ
j
l
(
x
t

j

)
I
t

j
,
(10)
j
=1
j
=1
j
=1
j
=1

where
l
(
.
)isafunctionofafinitenumberoflaggedvaluesofobservables,and
x
t
isavectorof
explicativevariables.Theroleof
l
(
.
),istolinktheindex
π
t
totheobservablevariablesthat
belongtotheinformationset.Anaturalchoicefor
x
t

j
isgivenbythelaggedreturnsorthe
laggedVaR.
Consequently,weproposesevenspecifications,denotedby
DB
1
to
DB
7
,(where
DB
stands
for
DynamicBinaryModel
):

DB
1:
π
t
=
c
+
β
1
π
t

1
,
DB
2:
π
t
=
c
+
β
1
π
t

1
+
δ
1
I
t

1
(
α
)
,
DB
3:
π
t
=
c
+
β
1
π
t

1
+
δ
1
I
t

1
+
δ
2
I
t

2
(
α
)
,
DB
4:
π
t
=
c
+
β
1
π
t

1
+
δ
1
I
t

1
+
δ
2
I
t

2
+
δ
3
I
t

3
(
α
)
,
DB
5:
π
t
=
c
+
β
1
π
t

1
++
ψ
1
VaR
t

1
,
DB
6:
π
t
=
c
+
β
1
π
t

1
+
δ
1
I
t

1
(
α
)+
ψ
1
VaR
t

1
,
DB
7:
π
t
=
c
+
β
1
π
t

1
+
δ
1
I
t

1
(
α
)+
ψ
1
VaR
t

1
+
γ
1
VaR
t

1
I
t

1
.

)11()21()31(()41)51()61()71(

Thefirstfourspecificationscorrespondtoadynamicbinaryresponsemodelincludingthelagged
indexasanexplanatoryvariableandsomeadditionalinformationthroughtheobservedpast
valuesoftheviolationprocess.Thefifthandsixthmodelarederivedfromtheautoregressive
quantilespecificationsCAViaRusedbyEngleandMangenelli(2004),andmorespecificallyfrom
theirsymmetricabsolutevaluespecification.Tobemoreprecise,theindexisassumedtorespond
symmetricallytolaggedVaRvalues.Thelastspecification
DB
7
introducesanasymmetryin
theresponseoftheindextopastVaRaccordingtotheappearanceofaVaRviolation.

01

Un pour Un
Permettre à tous d'accéder à la lecture
Pour chaque accès à la bibliothèque, YouScribe donne un accès à une personne dans le besoin