Appendix D Stages 1 and 2 Physico-Chemical and Habitat Audit Error Report

Department Of The Environment And Heritage

Description

WATER ECOscience: QA/QC Project – Year 2 Milestone and Final Report. Appendix D. Physico-Chemical and Habitat Audit Error Report Appendix D Stages 1 and 2 Physico-Chemical and Habitat Audit Error Report Appendix D Page 1 National River Health Program AusRivAS Quality Assurance and Quality Control Project Physico-Chemical and Habitat Audit Error Report (Stages 1 and 2) for Australian Government Department of the Environment and Heritage August 2004 Project Manager: Ross Bannister telephone [phone] email [e-mail] WATER ECOscience Pty Ltd ACN 064 477 989 Head Office 68 Ricketts Road Mt Waverley Victoria 3149 Australia Private Bag 1 Mt Waverley Victoria 3149 Australia telephone [phone] facsimile [phone] Wangaratta st1 floor NETC House 90-100 Ovens Street Wangaratta, Victoria, 3676 Carrum Eastern Treatment Plant Thompson Road Bangholme Victoria 3175 Gippsland 71a Argyle Street Traralgon Victoria 3844 Geelong 49 Carr Street Geelong, Victoria 3220 Hobart 20 St Johns Avenue New Town Tasmania 7008 Werribee Western Treatment Plant New Farm Road Werribee Victoria 3030 WATER ECOscience Report Number: 545 August 2004 Cover photo: Habitat assessment QA/QC audit, Queensland. Tasmania. © Commonwealth of Australia (courtesy of WATER ECOscience 2000) WATER ECOscience: QA/QC Project – Year 2 Milestone and Final Report. Appendix D. ...

Informations

• Publié par: Ipsa
• Nombre de lectures: 25
• Langue: English

Extrait

WATER ECOscience: QA/QC Project–Year 2 Milestone and Final Report. Appendix D. Physico-Chemical and Habitat Audit Error Report  

Appendix D

 

 

Appendix D  

Stages 1 and 2 Physico-Chemical and Habitat Audit Error Report

Page 1

 

         

 

National   Rive   htlaeH   rPmraog   AusRivAS Quality Assurance and Quality Control Project  

Physico-Chemical and Habitat Audit Error Report (Stages 1 and 2)

  for   Australian Government Department of the Environment and Heritage  

August 2004

 

 

 

 

Project Manager: Ross Bannister telephone [phone] email [e-mail]   WATER ECOscience Pty Ltd ACN 064 477 989   Head Office 68 Ricketts Road Mt Waverley Victoria 3149 Australia Private Bag 1 Mt Waverley Victoria 3149 Australia telephone [phone] facsimile [phone]   Wangaratta  1 st floor NETC House 90-100 Ovens Street Wangaratta, Victoria, 3676   Carrum Eastern Treatment Plant Thompson Road Bangholme Victoria 3175   Gippsland  71a Argyle Street Traralgon Victoria 3844   Geelong  49 Carr Street Geelong, Victoria 3220   Hobart   20 St Johns Avenue New Town Tasmania 7008   Werribee   Western Treatment Plant New Farm Road Werribee Victoria 3030   WATER ECOscience Report Number: 545     August 2004  

Cover photo: Habitat assessment QA/QC audit, Queensland. Tasmania. © Commonwealth of Australia (courtesy of WATER ECOscience 2000)

 

WATER ECOscience: QA/QC Project–Year 2 Milestone and Final Report. Appendix D. Physico-Chemical and Habitat Audit Error Report

Foreword

WATER ECOscience (formerly AWT Victoria) was engaged by Environment Australia to undertake a study relating to Quality Assurance/Quality Control of AusRivAS data collection methods. The study involved auditing state / territory lead agency physical, chemical and environmental data entered into the AusRivAS model and assessing lead agency QA/QC procedures for data collection and management. Audits of states / territories and analysis of results were conducted in a two-stage process. Stage 1 was conducted during the first year of the project and involved audits of Victoria, Queensland, Western Australia and the Northern Territory. The results of the Stage 1 audit were presented in the project’s Year 1 Milestone and Final Report (WATER ECOscience 2002). Stage 2 was conducted during the “second” year of the project and involved audits of New South Wales, the Australian Capital Territory, South Australia and Tasmania. This report presents results from both the Stage 1 and Stage 2 audits and, as such, supersedes the previously submitted Stage 1 audit report, particularly the findings for between state / territory comparisons, included as part of the project’s Year 1 Milestone and Final Report (WATER ECOscience 2002).      

Appendix D

Page i

WATER ECOscience: QA/QC Project–Year 2 Milestone and Final Report. Appendix D. Physico-Chemical and Habitat Audit Error Report

Contents Foreword  1 Introduction 1.1 Rapid Bioassessment 1.2 AusRivAS- National Context 1.3 Project Outline 1.4 Report Structure 2 Methods 2.1 Field Audits and Laboratory Visits Audit Plans Field Audits Laboratory Visits 2.2 Desktop-based Audits QA/QC Procedures Site Physico-chemical Data Audits Mapwork 2.3 Data Analysis Data Entry Map Errors Field validation 3 Results 3.1 QA/QC procedures 3.2 Data entry Summary of data entry errors Data analysis 3.3 Map errors 3.4 Field-validation error estimates Stream width Substratum variables 4 Discussion 4.1 Individual States and Territories 4.2 Interstate comparisons Summary - 4.3 Possible QA/QC Program 5 Conclusions and Recommendations 6 References

Appendix D

i 1 1 2 3 4 5 5 5 5 10 10 10 10 16 17 17 19 20 22 22 23 23 23 29 31 31 33 35 36 42 26 45 47

Page ii

WATER ECOscience: QA/QC Project–Year 2 Milestone and Final Report. Appendix D. Physico-Chemical and Habitat Audit Error Report

List of Figures

Figure 1 Diagram of transect method used to measure substrate heterogeneity. 9 Figure 2 Location of the 20 sites used for physical, chemical and habitat assessment audit for Queensland in relation to all sites sampled during the MRHI program. 12 Figure 3 Location of the 26 sites used for the physical, chemical and habitat audit (red dots) of Victoria in relation to all sites sampled for the MRHI program. 14 Figure 4 Location of the 26 sites used for physical, chemical and habitat assessment audit in Tasmania in relation to all sites sampled during the MRHI program. 14 Figure 5 Location of the 53 sites used for physical, chemical and habitat assessment audit in South Australia in relation to all sites sampled during the MRHI program. 15 Figure 6 Location of the 33 sites used for physical, chemical and habitat assessment audit (red dots) for Western Australia in relation to sites sampled overall for MRHI and FNARH. 15 Figure 7 Location of the 16 sites used for physical, chemical and habitat assessment audit for the Northern Territory in relation to all sites sampled during the MRHI program. 16

List of Tables

Table 1 The lead agency of each state / territory audited and the relevant field sampling methods/guidelines employed by each (as was current during the field audit). 6 Table 2 Auditing procedures conducted for each state / territory. Y indicates auditing was conducted. 8 Table 3 Different categories of transcription and data entry error 11 Table 4 Number of MRHI reference sites in each state / territory audited for errors in physical habitat data (sites generally involved more than one habitat and were sampled on more than one occasion (year and/or season). 12 Table 5 Variables used for analysis of data entry errors within (A) and between (B) states / territories. 21 Table 6 Occurrence and level of detail of documented field sheet and QA/QC procedures in audited states / territories. 22 Table 7 Summary of the total number of errors, the number of types of errors and the number of variables affected for data entry in each state / territory. 23 Table 8 Results of logistic regression to describe the effects of variable and error type on data entry error rates. 24

Appendix D

Page iii

WATER ECOscience: QA/QC Project–Year 2 Milestone and Final Report. Appendix D. Physico-Chemical and Habitat Audit Error Report

Table 9 Variables for which the error rate was significantly (P < 0.025) less than that for the variable ‘Stream width’. The odds-ratio (OR) for each variable is indicated 27 Table 10 Variables for which the error rate in comparisons between states and territories was significantly less than that for the variable ‘Mean Stream Width’. The odds-ratio (OR) for each variable is indicated 29 Table 11 Repeated-measures analyses of variance statistics for Victorian map-based data estimation. 30 Table 12 Among-operator variability in map-based measurements in Victoria: sums-of-squared residuals. 31 Table 13 The mean and one standard error of stream width estimation errors in Tasmania for each combination of operator (agency staff member) and site: 33

List of Plates

Plate 1 Field measurement of substrate heterogeneity using the transect method. 9  

Appendices

Appendix D:1 Description of Habitats Sampled During State and Territory Field Audits. 1 Appendix D:2 Lead Agency Internal QA/QC Methods 10    

Appendix D

Page iv

WATER ECOscience: QA/QC Project–Year 2 Milestone and Final Report. Appendix D. Physico-Chemical and Habitat Audit Error Report

Summary

This component of the AusRivAS Quality Assurance and Quality Control Project examined the performance of state and territory lead agencies in collecting habitat and physico-chemical data for inclusion in AusRivAS models. General findings were as follows: · procedures are poorly developed and, although informaldocumented QA/QC QA/QC procedures were often applied, they did not always show in agency performance;  · transcription errors were the most common followed by averaging or guess errors to make up for incomplete field data; · visual assessment of variables (mean width and substrate characteristics) result in high error rates and more objective practices should be applied to these and map variables.   The following suggestions are for the consideration of Environment Australia and the state / territory agencies.

General recommendations: · consistent documentation should be established for procedures associated with field work, desk-based assessments, laboratory work and QA/QC; · standard formats should be applied for entry of field data, desk assessments and laboratory work, to ensure that all required data is obtained and calculated correctly;    · labelling of samples should be adequate for correct identification, processing and storage; · calibration and maintenance of field equipment should be undertaken to meet accepted practices; · data entry and validation should be included in documented QA/QC procedures and should include scanning for extreme or outlier values; · targets for acceptable error limits should be set for each of the AusRivAS variables and monitoring undertaken to determine compliance with these targets;  · QA/QC training, and assessment of operator competencies, should be undertaken in addition to existing AusRivAS training; · a formal coordinated national QA/QC program should be established against which external auditing can be undertaken to review to assess its efficacy.    

Appendix D

Page v

WATER ECOscience: QA/QC Project–Year 2 Milestone and Final Report. Appendix D. Physico-Chemical and Habitat Audit Error Report

Habitat and map variables · more objective procedures should be developed for stream width and substrate size ranges, these would provide more accurate data that the current visual estimation practiced; map variables were also found to be subject to operator error, improved techniques and training would improve the accuracy of these variables.

·

 

 

Appendix D

Page vi

1

WATER ECOscience: QA/QC Project–Year 2 Milestone and Final Report. Appendix D. Physico-Chemical and Habitat Audit Error Report

Introduction

1.1 Rapid Bioassessment Rapid biological assessment (ie. rapid bioassessment - RBA) can be used to describe two very different types of biological monitoring (Norris and Norris 1995). The first is a continual monitoring situation to detect trigger or alarm levels of organisms or toxicants. The second, and the subject of this review, refers to expeditious sampling of biota with rapid delivery of assessment results (Norris and Norris 1995). Benthic macroinvertebrate rapid bioassessment techniques and procedures were developed in 1977 in conjunction with the commencement of the RIVPACS program in October of the same year (Wright 1997; Davies 2001pers comm United States Environmental Protection Authority further developed). The and expanded the RBA techniques to include fish and the work of Plafkinet al. (1989) was later used by several other countries (eg. Australia and Canada) to develop their own rapid bioassessment procedures (Norris and Norris 1995). Rapid bioassessment offers several advantages over the more traditional macroinvertebrate sampling methods, which involve a relatively large expenditure of time to collect, process and identify biological samples (Lenat and Eaton 1991). Rapid bioassessment reduces sampling effort, and therefore cost, by taking a relatively large sample instead of several individual replicates and reduces the number of organisms that must be processed by using a standardised sub-sampling procedure. In addition, rapid bioassessment programs often employ more efficient methods of data analysis than traditional biological assessment programs and produce results that are presented and summarised in a manner readily understood by non-specialists (Reshet al.1995). Although rapid bioassessment of freshwater systems is now used in a number of different countries, only the United States, the United Kingdom, Canada and Australia conduct integrated, large-scale programs using comprehensive models that integrate macroinvertebrate and physico-chemical data to compare test sites to a benchmark or reference condition. The bioassessment programs used by these four countries vary in the extent to which they are applied and in their base method. However, all are based on similar theory and all require established and documented quality assurance and quality control systems to ensure that the integrity and veracity of the models used, and the results they produce, are maintained.

Appendix D

Page 1

WATER ECOscience: QA/QC Project–Year 2 Milestone and Final Report. Appendix D. Physico-Chemical and Habitat Audit Error Report

1.2 AusRivAS- National Context The Australian River Assessment System (known as “AusRivAS”) was developed by the Cooperative Research Centre for Freshwater Ecology, in partnership with state and territory river management agencies, under the auspices of the National River Health Program (NRHP) funded by the Commonwealth Government. The NRHP was established in the Prime Minister’s Environment Statement in 1992 (O’Connor et al.1996). The objectives of National River Health Program are to: · provide a sound information base on which to establish environmental flows; · undertake a comprehensive assessment of the health of inland waters, identify key areas for the maintenance of aquatic and riparian health and biodiversity, and identify stressed inland waters; · consolidate and apply techniques for improving the health of inland waters, particularly those identified as stressed; · develop community, industry and management expertise in sustainable water resources management and raise awareness of environmental health issues and the needs of our rivers. The NRHP, initially called the National River Processes and Management Program commenced in December 1992 (Davies 1994; O’Connor et al. 1996). The Monitoring River Health Initiative (MHRI) – a key component of the NRHP – used aquatic invertebrates to assess on a national level the ecological condition of Australian rivers (Smith & Kay 1998). As part of the MHRI more than 1500 reference sites were sampled across all states and territories during 1994/96 to establish the predictive AusRivAS models. The second phase of the NRHP utilised the AusRivAS models to undertake the First National Assessment of River Health (FNARH) (Smith & Kay 1998), later referred to as the Australia-wide Assessment of River Health (AWARH). The FNARH - AWARH commenced in 1997 and nearly 6000 sites have been assessed nation- wide. Australia is the first country in the world to undertake such a continental-scale assessment of the ecological health of its rivers (PIE 1998). The bioassessment component of AusRivAS uses a series of models to predict the composition of the aquatic macroinvertebrate community expected at a specific site in the absence of environmental stress (expected taxa (E)). This is compared with the macroinvertebrate community composition actually found at the site (observed (O)). AusRivAS assessments are reported as the ratio of observed to expected (O/E) taxa for the site, which are then assigned to a band indicating the extent to which a site has been impacted. Importantly, the veracity and national consistency of the AusRivAS–based river health assessments are reliant on the collection and entry of accurate and precise data. The current project, theNational River Health Program - AusRivAS Quality Assurance and Quality Control Project, a national, external audit of data provides collected by the various state and territory government agencies using AusRivAS to assess river health in Australia. Information from this project augments previous quality assurance and quality control (QA/QC) work conducted under the NRHP and focuses on QA/QC during field operations, during subsequent laboratory sample

Appendix D

Page 2