Welcome to the Dichotomous BMD models section of the on-line benchmark dose software training

Epa/Ord/Ncea

Informations

• Publié par: Shean
• Nombre de lectures: 23
• Langue: English

Extrait

Transcript for the Dichotomous Module of the online BMD Models Training nd Slide 1. Welcome to the Benchmark Dose models training session. This is the 2 module and focuses on dichotomous models. Slide 2. The purpose of the benchmark dose or (BMD) program is to derive a point of departure for calculating a reference dose or a reference concentration abbreviated RfD and RfC respectively. These values are calculated by dividing the BMDL, which stands for the lower confidence limit on the benchmark dose estimate, or the BMCL which stands for the lower confidence limit on the benchmark concentration estimate by uncertainty factors. For additional background on using the BMD in developing a reference dose or reference concentration, please refer to the BMD Introduction training module. Slide 3. Dichotomous data are quantal data where an effect for an individual may be classified by one of two possibilities. Examples include: dead or alive, present or absent as in tissue pathology or cancer incidence. Slide 4. The first step in BMD modeling is to determine whether or not the data are worth modeling. To do this, evaluate the database as for the NOAEL approach. You need good quality studies that are of the appropriate duration and route of exposure and that measure the endpoints of concern. You should model all potentially adverse endpoints, especially if different uncertainty factors may be used for different endpoints. Slide 5. You also want to look for a significant doserelated trend in the data set you are going to model, at least two doses with responses in excess of the control and responses near the benchmark response or BMR. Slide 6. You should model all biologically, statistically significant responses if feasible. As a range to consider, model all endpoints with a LOAEL that is less than 10 fold above the lowest LOAEL of the database. In some cases, it may be difficult to obtain a curve that provides a good fit to the entire doseresponse for an endpoint, particularly if the response plateaus at high doses. In such cases, the high dose group may be dropped if that improves the data fit at the response close to the BMR. Because we are generally interested in the response at the low dose, it is usually more important for the modeled curve to fit the data in the low dose region than to fit the overall doseresponse. However, plateaus or other changes in the dose response at the high dose may reflect changes in toxicokinetics (such as saturation of metabolism) or changes in toxic response that the assessor may wish to model. At the expense of estimating an extra “asymptote” parameter some models such as the Hill model are capable of modeling data that “flatten out” in the high dose region. Version 1.4.1 of EPA’s benchmark dose software contains a Hill model for the evaluation of continuous data and the soontobe released version 2.0 of BMDS will contain a Hill model for the evaluation of dichotomous data.

Slide 7. The benchmark dose analysis process is presented here in 6 steps. The first step is to select a benchmark response level to evaluate. Slide 8. The benchmark response or BMR in dichotomous analysis corresponds to the percent of animals or subjects that will show the given endpoint at a particular dose. How should we select the BMR? Slide 9. When selecting a BMR, it should be near the low end of the range of increased risks that can be detected by a bioassay. Selecting a BMR that is too far below the observed data causes one to rely more heavily on the shape that the model takes below the data, thus leading to high model dependence. This means that BMD modeling should not be used to extrapolate from the range of the data to doses or responses that are too far below the experimental doses, such as might occur with extrapolation to environmental exposure levels. In addition, great care should be used if the lowest nonzero dose in a study gives a response well above the standard benchmark dose, such as if the low dose gives a 50% response. In such cases, extrapolation below the data involves considerable uncertainty. Slide 10. . In selecting a BMR, you must also decide on the measurement of increased risk you will use. The BMD software provides two options for quantal data: Added risk and extra risk. Added risk is the increase in the probability of an adverse health effect at a treatment dose, here we call dose d, above the background probability in the control group, here we call dose 0. The equation is shown here. Therefore, the added risk refers to the absolute increase in the percentage response from the background in the control group.  Extra risk is the relative increase in the probability of an adverse health effect above the background probability given the condition that the individual would not have had the adverse health effect at the background dose (dose 0). The equation for extra risk takes the added risk equation and divides it by the term, 1 the probability at dose 0.  Extra risk is typically used by the EPA in risk assessments. Slide 11. Example calculations for both added and extra risk are presented here. The difference between them is that with extra risk, you take into account the background risk and end up calculating a lower dose for the same BMR. The extra risk and the added risk are the same if the background term is zero because the denominator in extra risk becomes 1. Slide 12. The EPA BMD guidance on selecting a BMR for dichotomous data says that an extra risk of 10% is the default since the 10% response is at or near the limit of sensitivity in most cancer bioassays and in some noncancer bioassays. There are examples in risk assessment where one uses a lower BMR, typically if a study has greater than usual sensitivity. Response levels between 1 and 10% can be used if the study has sufficient sensitivity to detect a response in the range of the chosen BMR. An example where this has been found is in the case of developmental toxicity studies where there are larger than average sample sizes. However, the BMD and BMDL 10 should always be presented for comparison purposes.

Slide 13. After you have chosen the BMR to evaluate, the next step in the BMD analysis flowchart is to select a model, set the parameters and run the model. Slide 14. In selecting a specific model, remember that the nonlinear models do not necessarily have a biological interpretation. These are flexible mathematical equations or models that are used to optimize the fit of the curve to the actual data by changing the parameters of the model. The mathematical equations do not have any information in them associated with actual biological processes. Criteria for final model selection will be based mainly on whether various models describe the data. Slide 15. The models currently available in the EPA Benchmark dose software to analyze dichotomous data are shown on this slide. There is a gamma model, a logistic and probit model, both of which give the option of selecting the log of the dose, the multistage model and the Weibull model. The quantallinear model is a special case of the Weibull model where the power term has been set to 1. Slide 16. This slide shows a screen shot of what you will actually see in the software. After entering the data, and selecting dichotomous as the model type, a pull down menu next to the word model allows you to select the model you want to run. Slide 17. After you have selected a model, you will next set the parameters. Slide 18. For this example we will look at a couple of the models. The first is the gamma model. The form of the model is shown on this slide. There are several terms in this model. The gamma term is a background term for this equation. The alpha term is a power term and the beta term represents the slope. Slide 19. On the window in BMDS titled “Type Model Run” you will be given the option to check the box that says “Restrict Power”. This is where you select the model parameters. We will refer to this window repeatedly during the course of this presentation. The technical guidance document recommends that you restrict the power term to be greater than or equal to one, because if the alpha term take on a value that is less than one, the doseresponse curve could become infinite at the control dose. Slide 20. This slide illustrates that point by showing the graphical output for the gamma model when it was run without restricting the power. The infinite slope at zero dose is not biologically plausible. Slide 21. Rerunning this model while restricting the power provides a more gradual increase in the slope at zero dose and is more biologically plausible. Slide 22. The next model we will discuss is the logistic model. You will recall that this is one of the models that has the option of taking the log of the dose, giving you the log logistic model. The form of the model is shown on this slide.

Slide 23. There are several terms in this model. The gamma term is a background term that only appears when you select the log of the dose. The alpha term is an intercept term and the beta term represents the slope. Slide 24. On the window in BMDS titled “Type Model Run” you will be given the option to check the box that says “Restrict Betas”. The technical guidance document recommends that you restrict the slope or beta term to be greater than or equal to one because if the beta term takes on values less than one, the doseresponse curve could become infinite at the control dose. Slide 25. Now let’s look at the form of the multistage model. The form of the model is shown on this slide. It has a gamma term for the background and beta terms to represent the power. Slide 26. On the window in BMDS titled “Type Model Run” you will be given the option to check the box that says “Restrict Betas”. The technical guidance document recommends that you restrict the beta term to be greater than or equal to zero because if the beta term takes on values less than zero, the doseresponse curve could become wavy. Slide 27. Looking at our first example, after entering the data into the Create/Edit dataset screen, and selecting dichotomous for “Model type”, you can select the type of model you want to run. In this case, the multistage model is selected. Slide 28. When you click the “Proceed” button, another window will appear titled “Dichotomous Type Model Run”. Circled in green is the “Restrict Betas” check box. Slide 29. If this box is not checked, this graphic output provides an example of the wavy character that the multistage model can take on. This is not biologically plausible. Slide 30. Checking the “Restrict Betas” check box provides a more linear fit to the data. Slide 31. The next model we will discuss is the probit model. You will recall that this is the other model that has the option of taking the log of the dose, giving you the logprobit model. The form of the model is shown on this slide. There are several terms in this model. The gamma term is a background term that only appears when you take the log of the dose. The alpha term is an intercept term and the beta term represents the slope. Slide 32. On the window in BMDS titled “Type Model Run” you will be given the option to check the box that says “Restrict Slope”. The technical guidance document recommends that you restrict the slope to be greater than or equal to one because if the slope takes on values less than one, the doseresponse curve could become infinite at the zero dose. Slide 33. Now let’s look at the Weibull model. The form of the model is shown on this slide. There are several terms in this model. The gamma term is a background term. The alpha term is a power term and the beta term represents the slope.

Slide 34. On the window in BMDS titled “Type Model Run” you will be given the option to check the box that says “Restrict power”. The technical guidance document recommends that you restrict the power to be greater than or equal to 1 because if the alpha term takes on values less than one, the slope of the doseresponse curve could become infinite at zero dose. To summarize, whenever there is a choice for restricting a parameter for the model, we recommend to restrict that parameter unless you have other reasons for not doing that. Slide 35. Looking at our flowchart, after you select a model and set the parameters it is time to run the model. Slide 36. EPA’s BMD software (BMDS) can be downloaded from EPA’s website at www.epa.gov/nceadownloaded file is a selfextractable software package, and you. The can easily install the software by double clicking on the downloadable file. After installing BMDS onto your system, an icon will appear like the one shown in this screen shot. Click on the BMDS 1.4 icon to launch the program. Slide 37. You will see the following screen as the program loads. Click “OK” to proceed to the next step. Slide 38. To open a dataset, you can click on the file open icon or select File and Open from the toolbar. Slide 39. A list of files will show up on a popup screen. Here, select DICHOTOMOUS.SET. Slide 40. Enter your data as you would an excel spreadsheet. Slide 41. To change the column headings right click on the box of the column label and select “Rename Column.” Slide 42. A popup box will appear. Enter your column headings and click “Ok.” Slide 43. You can also transform a column of data by right clicking on the column header, then clicking on “Transform Column” from the pull down menu. Slide 44. Using the transform column feature, a popup screen will appear, allowing you to perform a number of different transformations. Of course, the data transform can also be performed in other data spreadsheets, and the results then transferred into BMDS by using the function of copying and pasting. Please note that due to some software issues, if you are going to use copy and paste to transfer the data from Excel spreadsheet to BMDS, you need to copy the content, and then close the spreadsheet before you perform the pasting.

Slide 45. After you enter the data in the spreadsheet, you can select the model type and the model you’d like to use. Here, we are using dichotomous data, so we will select dichotomous under model type. Once you select a model type, the models that are available in the software become shown as options in the “model” window pulldown menu. Here, we are using the gamma model. Slide 46. Next, we will select the appropriate data columns into the boxes below. In the box marked “dose”, you will select the name for the column header that provides dose information. In this example, the appropriate header is “DOSE.” In the box marked “number of subjects in dose column”, you will select the “total” column. In the box marked “incidence” you will select the “effect 1column,” since we will be demonstrating modeling for the “effect 1” data. Now, you can select the proceed button to go to the next screen. Slide 47. The popup window that comes up is titled “Dichotomous type model run”. Here as you recall you will select your parameters. First, you will have the option of selecting the risk type. The choices are extra or added risk. We will use extra risk, as this is what is preferred by the EPA. Next, there is a box that allows you to input the BMR Level. We will use 0.1 for 10%. Next you have the option of selecting the BMD calculation. This box needs to be checked in order for the calculated BMD to appear on the output. You can also check the box to select the BMDL Curve calculation. This will appear on the graphic output. Finally, you will want to restrict the power by clicking the restrict power box. Again, this is the method preferred by the agency. Now click “run” to run the model. Slide 48. Several screens will pop up, following running the model. One is the text output screen shown in this slide. Notice the information at the top of the output provides you with type of model run, the location of the input file used, and the location of the graphic output. The other information informs you of the variables and the data that were used to generate this result. Slide 49. The second screen you will see is the graphical output, which is shown here for the gamma model. This shows you the visual fit of the model to the data. The red line represents the BMD estimate. The green points represent the actual data points. And the blue line represents the lower limit on the BMD estimate. Slide 50. The next step in our flow chart is to determine “Does the model fit the data?”

Slide 51. How will we determine whether the model fits the data? There are three areas we will look at; a global measurement, a local measurement, and a visual inspection of the model. First, we will look at the global measurement of fit, which is found in the goodness of fit p value. A value of p greater than 0.1 indicates that the model fits the data. Note that for the goodness of fit p value, a larger number indicates better fit, and a p value of 1.0 indicates a perfect fit. Slide 52. This global measurement is based upon a chi square calculation. The equation and the definition of the terms are shown. Based on the chi square and the degrees of freedom, a goodness of fit p value can be calculated. Slide 53.The goodness of fit p value can be found on the text output file under the goodness of fit table. The pvalue is shown here circled in green. Slide 54 Please note that the goal of BMD modeling is to fit a model to doseresponse data that describes the data set, especially at the lower end of the observable dose response range. Slide 55. The next area we can look at to determine if the model fits the data is the local measurements of fit called the scaled residuals. Scaled residuals should have an absolute value less than 2.0. Slide 56. How do we use the scaled residuals to assess how well the model fits the low dose data? Occasionally the software will converge on a less than optimal solution or will “fit’ the wrong region of the dose response curve, the high end for example. Scaled residuals are a measure of how closely the model fits the data at each point. They are calculated by taking the observed data point and subtracting the estimated data point, with 0 equaling an exact fit. It is more important to have small absolute values of the scaled residual near the BMR than at higher doses. The technical guidance says you should question values of scaled residuals that are greater than an absolute value of 2. Slide 57. Looking at our graph of the gamma model fit to our data, the scaled residuals show how close our model, the red line, comes to the data points shown in green. Here we can see the data point closest to the 10% BMR is the 50 mg/kg dose group. Slide 58. The original data and model estimated values appear on the text output in the column marked scaled residuals under the goodness of fit table. We are particularly interested in the scaled residual closest to the BMR which in this case is the dose of 50 as shown in the previous slide. The absolute scaled residual value at this data point is less than 2 as our guidance says it should be. Slide 59. Looking at the graphical output, the region we are interested in is circled in green.

Slide 60. Finally, a visual inspection of the model fit will allow you to detect obvious problems such as high dose bias in other words, not fitting the low dose information or, if the model takes on a shape that is not biologically possible, such as a wavy curve. Slide 61. As shown in this slide, visual inspection of the gamma model suggests that the model fits the data well and does not take on any forms that are biologically implausible. Slide 62. It is recommended to use all of the models in the BMDS suite to evaluate model dependence. After the data are modeled with all the available models, we need to compare the modeling results from various models to choose the one that gives the final BMD. Slide 63. The first step to do a model comparison is to evaluate the BMDLs from these models to determine whether or not they are within a 3fold range. Slide 64. If the answer is no, this indicates a significant model dependency in the data fitting. Therefore, the lowest BMDL will be used as the point of departure as a conservative estimate. Slide 65: If the answer is yes, we will proceed to determine if one model fits our data best. Slide 66: The AIC is used to determine whether one model fits the data best. The Akaike Information Criterion or AIC can be used to compare models from different families which use a similar fitting method (for example least squares or a binomial maximum likelihood) as well as within the same family. The smaller number represents a better fit. The equation for the AIC is shown here along with the definition of its terms. Within a family of models, fit will improve as parameters are added. However, there is a tradeoff between adding additional parameters and improving fit. For a similar degree of fit, the AIC rewards the less complex model. Slide 67. The AIC can be found on the text output under the Analysis of Deviance Table and is shown here circled in green. Slide 68. The next step in our flow chart says to use the BMDL from the model that provides the best fit. Again, the smallest AIC represents the best model. If several models fit the data equally well based on a comparison of the AICs, an average of the BMDLs from these models can be used as the point of departure. Otherwise, the BMDL from the model with the best fit should be used. Slide 69. Now, let’s look at our first example. You can also run the BMD software with this sample data as a handson exercise. Slide 70. A sample dichotomous data set is shown. There are two dose groups along with a control and there appears to be a doseresponse relationship, so we can model this data.

Slide 71. First, enter this dataset as shown in the Create/Edit Dataset Screen. To enter new data, you can click on the new/create icon or select File and New from the toolbar. Enter the data and label the columns as discussed previously. Then, we need to select dichotomous for the model type. We will select the multistage model as an example. When you have finished, you may click Proceed. Slide 72. On the Dichotomous Type Model Run Screen, please note that the red arrows point to default settings that you selected in the previous model run with the same model type. You need to double check these parameter selections before you proceed. In this case, the BMR is set to 0.1 or 10%, the BMD calculation is checked and we will restrict the betas for this example. The green arrow points to the Degree of the polynomial box st which is set to 2. Change this entry to 1 as we will begin with the 1 degree of the polynomial. If you click the “BMDL Curve Calc” box, the graph will also show the BMDL curve as shown in blue. When you have finished, click Run. Slide 73. After the model run is finished, several pop up windows will be shown on the screen. The graphical output screen should look like this. The title at the top of the graph allows you to make sure you ran the model you intended. Slide 74. You will also have a text output screen which should provide the values that are summarized for you here. Check to make sure you can locate these values and that they are the same as those shown in the text output window. Slide 75. Here we have provided a table which you can also fill in to keep track of particular values as you run other models. For this example, we will run the 1st and 2nd degree multistage models and the logprobit model. From these runs, we will record the BMD10, the BMDL10, the AIC, the p value and the scaled residual at the BMR for each model. Slide 76. Now you will go back to the Dichotomous Type Model Run Screen and change the degree of the polynomial to 2 as indicated by the green arrow. Click Run when you are ready to proceed. nd Slide 77. The graphical output for the 2 degree multistage model is shown in this slide. Again, you will need to go to the text output window to find corresponding statistics values and fill in those values required in the table. Comparing the graphic output from nd nd the first degree model to the 2 degree model, the 2 degree model fits the data a little better. nd Slide 78. The 2 degree multistage model values are shown here. You should make sure that your values look like this as well. Note that the scaled residuals are smaller with the nd nd 2 degree model and the p value has improved. However, the AIC using the 2 degree st nd model is larger than the AIC for the 1 degree model, meaning that the fit for the 2 degree model did not improve sufficiently to compensate for the additional complexity added by the additional parameter. We will discuss the evaluation of the AIC later in this presentation.

Slide 79. To run the last model in the table, you will need to close the Dichotomous Type Model Run Screen and go back to the first model run screen, the Create/Edit Dataset Screen. The only selection you need to change is the model. Now select Probit and click on Proceed. Slide 80. On the Dichotomous Type Model Run Screen for the Probit model, note that red arrows are next to the “Restrict Slope” box and the “Log of Dose” box. Make sure that both of these boxes are checked. No other changes need to be made. Click the Run button to proceed. Again a graphical output and a text output will come up. Record the appropriate values for the LogProbit model into your summary table. Slide 81. The values for the LogProbit model have been added to the table and you now have the results from all three model runs. How do we decide which is the best model? Slide 82. Going back to our comparison of model fit slide, remember that visual inspection of the model fit will allow you to detect obvious problems such as high dose bias or, if the model takes on a shape that is not biologically possible. Of course, the goodness of fit P value will provide measurement of overall model fit to the data. In addition, the AIC is also a very useful statistics used to compare across models. The smaller the AIC, the better the model fit. Slide 83. In our flowchart, step 5 asks whether or not the BMDLs are within a 3fold range. Often, more than one model or modeling option will result in an acceptable fit to the data, that is a p value greater than 0.1. If the BMDL estimates from acceptable models are widely divergent, that is, they are outside of a 3fold range, consider using the lowest BMDL. If the models result in similar BMDL estimates, that is they are within a 3fold range, consider all the models that adequately fit the data and select the model with the best fit. nd Slide 84. Based on the information summarized in this table, the 2 degree model provides a little better goodness of fit P value. However, this benefit could not outweigh nd the addition of an extra parameter. Thus, we will not consider the 2 degree multistage model further. st Slide 85. Here, the 1 degree multistage model is compared with the logprobit model. Looking at the AIC values for the 2 remaining models, the Log Probit model has the lower AIC, therefore we would select this model. Slide 86. As shown in the flowchart, we would use the BMDL form the model that provides the best fit and document the BMD analysis as outlined in reporting requirements of the Benchmark Dose Technical Guidance document. Slide 87. In summary, first, you will determine whether the data should be modeled. Then, you need to determine how a model fits the data by visual inspection and the goodness of fit p value where a p value greater than 0.1 indicates an acceptable fit. After

you obtain information from various models with acceptable P values, compare across models using visual inspection, the AIC and scaled residuals. nd Slide 88. Now let’s look at the 2 dichotomous sample problem. Slide 89. A sample dichotomous data set is shown in this slide. There are four dose groups along with a control and there appears to be a doseresponse relationship, so we can model this data. Slide 90. We will run the gamma model for this example. Shown in this slide is the graphical output for the fit of the gamma model to all of the sample data. There appears to be a poor fit at the high dose region upon visual inspection. Please pay attention to the scaled residuals in the region of the BMR. Slide 91. This slide shows the graphical output for the fit of the gamma model to the sample data after dropping the highest dose. The visual fit to all the data points appears to have been improved. Slide 92. A comparison of the values between the two runs shows an improvement in the p value, the scaled residuals and the AIC. After dropping the highest dose group, the global goodness of fit P value increased from 0.0864 to 0.9032, and the AIC value decreased from 427.3 to 295.6. In addition, the scaled residue at the data point close to the BMR decreased from 0.722 to 0.230. Please note that the smaller the absolute value of the scaled residuals, the closer the estimated value is to the data point. Visual comparison of the curves also indicates an overall improvement to the model fit. Overall, the model fitting statistics indicate a significant improvement in the model fit after removing the highest dose group from this data set. Slide 93. Based on this comparison, removing the highest dose group will result in a better model fit at the range of the BMR which is 10%. Slide 94. We have provided a summary of BMD results running all of the dichotomous models against this data set without the highest dose group data. First, looking at the P value column, we can see right away that the quantallinear model did not provide a good fit. The multistage model overestimated responses at the middlerange of the dose response curve. The quantallinear model resulted in poor data fit in the range of the BMR. Here we used the scaled residue at the low dose data point which gives about 5% response to indicate the model fit at the BMR range. The Logistic and Probit models overestimated the control response which may have resulted in an overestimate of the BMD and BMDL due to the nature of the calculation of the BMR. Looking at the models that we have left those whose BMDs and BMDLs are in range, we would next look at the AIC value, we can see that the model with the lowest AIC is the Weibull model. According to our guidance, this model best fits our data, and should be used for BMD analysis.