Audit of USAID Cambodia’s Monitoring of the
Description
Audit of USAID/Cambodia’s Monitoring of thePerformance of Its HIV/AIDS ProgramAudit Report No. 5-442-02-002-PJune 21, 2002Philippines______________________________________________________________________________________________________________________________________________________________________________________________U.S. AGENCY FORINTERNATIONAL DevelopmentRIG/ManilaJune 21, 2002MEMORANDUMFOR: Mission Director, USAID/Cambodia, Lisa ChilesFROM: RIG/Manila, Bruce N. BoyerSUBJECT: Audit Report for the Audit of USAID/Cambodia's Monitoringof the Performance of Its HIV/AIDS Program, Report No. 5-442-02-002-PThis is our final report on the subject audit. We reviewed your comments tothe draft report, made some revisions based on them, and included thecomments in their entirety as Appendix II.This report includes six recommendations to be addressed byUSAID/Cambodia. Based on your comments to the draft report, we considermanagement decisions to have been made on all six recommendations.Please advise the Bureau for Management, Office of Management Planningand Innovation when final actions are complete. I appreciate the cooperation and courtesies extended to my staff during theaudit.1______________________________________________________________________________________________________________________________________________________________________________________________Summary of Results 3Table ofContentsBackground 3Audit Objectives ...
Informations
| Publié par | Shawyeg |
| Nombre de lectures | 16 |
| Langue | English |
Extrait
Audit of USAID/Cambodias Monitoring of the Performance of Its HIV/AIDS Program
Audit Report No. 5-442-02-002-P
June 21, 2002
Philippines
______________________________________________________________________________________________________________________________________________________________________________________________
U.S. AGENCY FOR INTERNATIONAL Development RIG/Manila June 21, 2002 MEMORANDUM FOR:Mission Director, USAID/Cambodia, Lisa Chiles FROM:RIG/Manila, Bruce N. Boyer SUBJECT:Audit Report for the Audit of USAID/Cambodia's Monitoring of the Performance of Its HIV/AIDS Program, Report No. 5-442-02-002-P This is our final report on the subject audit. We reviewed your comments to the draft report, made some revisions based on them, and included the comments in their entirety as Appendix II. This report includes six recommendations to be addressed by USAID/Cambodia. Based on your comments to the draft report, we consider management decisions to have been made on all six recommendations. Please advise the Bureau for Management, Office of Management Planning and Innovation when final actions are complete. I appreciate the cooperation and courtesies extended to my staff during the audit.
1
______________________________________________________________________________________________________________________________________________________________________________________________
Table of Contents
Summary of Results Background Audit Objectives Audit Findings Did USAID/Cambodia monitor performance of its HIV/AID program in accordance with Automated Directives System (ADS) guidance? Need to Strengthen Performance Monitoring Plan and Establish Performance Targets Mission Should Assess Data Quality PSI Needs to Implement Controls Over its Operations in Cambodia Need to Establish Performance Measures for Children Affected by AIDS Activities Is USAID/Cambodia achieving intended results from its HIV/AIDS program? Increase Availability of Condoms to Provinces and Military Bases What is the status of USAID/Cambodias efforts to meet anticipated HIV/AIDS reporting requirements? Management Comments and Our Evaluation Appendix I - Scope and Methodology Appendix II Management Comments Appendix III Rapid Scale-Up, Intensive Focus, and Basic Countries Appendix IV Summary of USAID/Cambodias Performance Monitoring Controls Reviewed by the Audit
3 3 6
6 10 13 14 19 20 24 27 28 30 32 41 42
2
______________________________________________________________________________________________________________________________________________________________________________________________
Summary of Results
Background
USAID/Cambodia complied with some of the Automated Directives System (ADS) guidance in monitoring the performance of its HIV/AIDS program. (See page 6.) However, the audit found that the Mission needs to strengthen its monitoring system by preparing a performance monitoring plan in accordance with USAID guidance for its recently approved strategic plan, by establishing performance targets, and by assessing data quality. (See page 10.) Moreover, USAID/Cambodia needs to address several areas requiring management attention. These areas, discussed on pages 14 to 20, include: (1) ensuring that its implementing partner, Population Services International, has implemented adequate controls over its operations in Cambodia, and (2) establishing performance measures for children affected by AIDS activities. As of year 2000, the Mission had exceeded the intended results for one of the three indicators reviewed (Condom Sales), but the audit cannot draw a conclusion on the other two (Condom Use and STD Care). While the reported results for Condom Use have shown an increasing trend over the last several years, the Mission did not set performance targets for the indicator. Similarly, the Mission did not establish performance targets for the STD Care indicator, and a baseline for this indicator has only recently been established. (See pages 20 to 24.) In addition, the Mission should ensure that Population Services International makes condoms available to provinces outside of Phnom Penh, especially rural areas where about 83 percent of Cambodians live, and to military bases in remote locations. (See pages 24 to 26.) Finally, in response to the increased USAID funding for HIV/AIDS, USAID has drafted new Monitoring and Evaluation Guidance. The guidance establishes several global targets that USAID is expected to achieve. The guidance also requires missions to monitor and report progress of HIV/AIDS programs using standard indicators. USAID/Cambodia has already established some indicators similar to those required by the guidance and is preparing to meet additional reporting requirements under its recently approved new strategic plan. (See pages 27 to 28.)
USAID funding for HIV/AIDS (Human Immunodeficiency Virus/Acquired Immune Deficiency Syndrome) has increased dramatically over the past three years: from $139 million in fiscal year 1999 to $330 million in fiscal year 2001 (see graph on next page).1 USAID is organizing its response to HIV/AIDS around three categories of countries: rapid scale-up countries, intensive focus countries, and basic countries. These categories describe the resources that USAID will apply and expectations as to when a measurable 1Information was provided by USAID and is unaudited.
3
______________________________________________________________________________________________________________________________________________________________________________________________
impact might be achieved. (See Appendix III for a description of these categories.) For example, rapid scale-up countries are defined as countries that will receive a significant increase in resources to achieve a measurable impact within one to two years. USAID plans to increase funding to rapid scale-up countries to reduce HIV prevalence rates, and to increase support services for people living with HIV/AIDS.
USAID's HIV/AIDS Funding
$350 $300 $250 $200 $150 $100 $50 $0
1999 2000 2001
This chart depicts USAIDs HIV/AIDS funding levels for fiscal years 1999 to 2001: $139 million for 1999, $190 million for 2000, and $330 million for 2001. Cambodia, with an estimated population of 12 million, has the highest HIV prevalence rate of any country in Asia and is one of four rapid scale-up countries. The Cambodian National Census has estimated that 169,000 Cambodians were living with HIV/AIDS in 2000, or 2.8 percent of the sexually active adult population. Factors driving the HIV/AIDS epidemic in Cambodia include: extensive solicitation of sex by Cambodian men; high prevalence of sexually transmitted diseases (STDs) that are drug resistant; inadequate national health facilities; mobility of the population, and the trafficking of children and women. Unprotected heterosexual sex is the 2 dominant mode of HIV transmission in Cambodia. USAID/Cambodia funding for HIV/AIDS activities increased from $2.5 million in fiscal year 1999 to $10 million in fiscal year 2001.3 The Mission has focused its efforts in combating the HIV/AIDS epidemic on effecting behavioral change and improving STD Care. Mission activities have targeted high-risk groups 2HIV/AIDS Briefs, Cambodia and HIV/AIDS Summary, United States Agency for International Development, May 2001. 3USAID/Cambodia's Resource Request and Results Review Report, April 2001.
4
______________________________________________________________________________________________________________________________________________________________________________________________
which drive the HIV/AIDS epidemic including brothel-based commercial sex workers and their clients (especially uniformed servicemen), freelance sex workers (beer promoters and bar girls), and mototaxi drivers. USAID/Cambodia has three major partners implementing its HIV/AIDS prevention activities: Family Health International (FHI), Population Services International (PSI), and the Khmer HIV/AIDS Non-Governmental Organization Alliance (KHANA). In addition, the Mission has funded a number of smaller prevention programs through other partners. A breakdown of the Mission's funding by implementing partner is shown in the following chart.
$3.5 $3.0 $2.5 $2.0 $1.5 $1.0 $0.5 $0.0
USAID/CambodiaObligations for HIV/AIDS Activities for Fiscal Years 1999-2001
PSI FHI KHANA Other Major Implementing Partners & Other
2001 2000 1999
This chart illustrates the breakdown of Mission funding by implementing partner for fiscal years 1999 to 2001. Funding levels for fiscal year 2001 are as follows: $ 1.8 million for PSI, $2.1 million for FHI, $0.8 million for KHANA and a total of $2.9 million for other partners and certain program positions.4
4 Information was provided by USAID/Cambodia and is unaudited.
5
______________________________________________________________________________________________________________________________________________________________________________________________
Audit Objectives
Audit Findings
This audit is one of a series of audits to be conducted worldwide of USAIDs monitoring of the performance of its HIV/AIDS program at the mission level. The Performance Audits Division of USAIDs Office of Inspector General (OIG) is leading the audits. The Regional Inspector General, Manila (RIG/Manila) conducted this audit. The audit objectives and the scope and methodology for the audit were developed in coordination with USAIDs HIV/AIDS Division in the Bureau for Global Programs, Field Support and Research (now the Bureau for Global Health). RIG/Manila performed this audit to review USAID/Cambodias HIV/AIDS program and specifically, to answer the following audit objectives:
•USAID/Cambodia monitor performance of its HIV/AIDS program inDid accordance with Automated Directives System guidance?
•Is USAID/Cambodia achieving intended results from its HIV/AIDS program?
•status of USAID/Cambodia efforts to meet anticipatedWhat is the HIV/AIDS reporting requirements? Appendix I describes the audits scope and methodology.
Did USAID/Cambodia monitor performance of its HIV/AIDS program in accordance with Automated Directives System (ADS) guidance? USAID/Cambodia complied with some of the Automated Directives System (ADS) guidance in monitoring performance of its HIV/AIDS program. However, USAID/Cambodia needs to strengthen its monitoring system by (1) preparing a performance monitoring plan that meets the requirements of USAID guidance; (2) establishing performance targets, and (3) assessing the quality of performance data. Moreover, USAID/Cambodia needs to increase the monitoring of their activities and to address several areas requiring management attention. These areas discussed on pages 14 to 20 include: (1) ensuring that its implementing partner, Population Services International, implements controls over its operations in Cambodia, and (2) establishing performance measures for activities for children affected by AIDS. ADS guidance outlines USAID policies and procedures for implementing a performance monitoring system which includes monitoring measures such as performance monitoring plans, establishing performance baselines and targets,
6
______________________________________________________________________________________________________________________________________________________________________________________________
evaluations, portfolio reviews, progress reports, and special studies. In accordance with the ADS, the Mission prepared a performance monitoring plan (PMP) which contained some of the required information, i.e., indicator definition, data source, data collection methods, data collection schedules, and disclosed data limitations. In addition, the Mission established baselines for some indicators in the plan. The Mission has also used other monitoring tools such as portfolio reviews and a mid-term evaluation of Family Health Internationals IMPACT (Implementing AIDS Prevention and Care) project.
USAID/Cambodia organizes its HIV/AIDS activities under the special objective: Reduced Transmission of STD/HIV among High-Risk Populations. This special objective has three intermediate results which focus on policy advocacy, condom use, and STD care. The Mission included twelve performance indicators in its PMP to report progress of its on-going activities. The schematic on the next page reflects the relationship between the HIV/AIDS special objective, the three intermediate result objectives, and the performance indicators under each objective. The shaded boxes represent the three indicators which the Mission officials and auditors selected for this review (these three indicators are also marked by asterisks.)
7
______________________________________________________________________________________________________________________________________________________________________________________________
Graphic Presentation of USAID/Cambodia's HIV/AIDS Performance Indicators
Special Objective: Reduced transmission of STD/HIV among high-risk populations
Prevalence of selected STDs among Percentage of men in target areas commercial sex workers in target areas reporting sex with two or more partners in the past month AIDS Program Effort Index
Intermediate Result 1: Policy makers are informed about the HIV/AIDS epidemic in Cambodia
Percentage of brothels in target areas participating in IMPACT project activities Policies, guidelines or materials used or adopted by other organizations
Intermediate Result 2: Intermediate Result 3: Reduced high-risk behaviors in Model STD and reproductive health service target areas delivery program for high-risk populations in target areas
* Percentage of men in target areas reporting always using condoms with commercial sex workers during previous three months Percentage of men in target areas reporting always using condoms with "mistresses" in previous three months
Percentage of female commercial sex workers in target areas reporting consistent condom use with clients over pastweek
* Number of Number One Condoms sold in project areas in past 12 months
Percentage of STD clients * in targeted facilities assessed and treated according to national standards Percentage of STD clients in targeted facilities provided information on condoms and partner notification
Percentage of clients reporting clinic as first source of treatment for STD-related complaint
This graphic shows USAID/Cambodia's performance indicators for its HIV/AIDS program, both at the "special objective" and at the "intermediate results" levels. The three indicators selected for review are marked by asterisks.
8
______________________________________________________________________________________________________________________________________________________________________________________________
At the start of the audit work, the auditors in collaboration with Mission officials decided to focus the review on the three highlighted performance indicators in the previous diagramnamely: (1) Condom Sales, (2) Condom Use, and (3) STD Care. All three indicators are at the intermediate results level for the activities the Mission is carrying out in targeted areas.5 officials Mission believed that these three indicators give the best overall sense of progress being made in its HIV/AIDS programs in year 2000. (See map below for location of the IMPACT projects target areas.) Cambodia
The highlighted locations on this map illustrate the IMPACT project target areas for years 1998 to 2000. The target areas consist of Phnom Penh, Kandal, and Kampong Cham. For the three indicators reviewed, the Mission's performance monitoring plan provided some of the information required for such a plan, such as indicator definition, data sources, data collection methods, schedules for data collection, and the disclosure of known data limitations. In addition, the results reported for one of the three indicators, Condom Use, agreed with source documents specified in the plan. Nonetheless, the audit found certain areas in which the performance monitoring system could be improved. For example, the Missions performance monitoring plan was not as complete as specified by 5According to the Missions performance monitoring plan, target areas for the IMPACT projectconsistofPhnomPenh,KampongCham,andKandalforresultsrelatedtocondom use and STD care indicators. The target area for condom sales is the entire country.
9
______________________________________________________________________________________________________________________________________________________________________________________________
ADS guidance. In addition, no performance targets were established for two of the three indicators, and no data quality assessments were conducted for any of the three indicators. These opportunities for improvement are discussed below and summarized in Appendix IV. Need to Strengthen Performance Monitoring Plan And Establish Performance Targets ADS 201.3.4.13 states that at a minimum, a performance monitoring plan (PMP) must provide a detailed and precise definition of performance indicators to be tracked; specify the source, method and schedule for data collection; and assign responsibility for data collection to a specific office, team, or individual. Similarly, a PMP must disclose known data limitations and describe procedures for data quality assessments. In addition, USAID guidance requires missions to review and update their respective performance monitoring plans annually and to establish performance targets. Nevertheless, our review of the Mission's PMP concluded that the Missions PMP for the HIV/AIDS special objective did not meet all the requirements of USAID guidance. In addition, the Mission did not establish performance targets for Condom Use and STD Care indicators. According to Mission officials, the principal causes for these shortcomings were (1) the Mission's Program Office being short-staffed, and (2) a change in the Missions plans and prioritiesand the development of a new strategic plan. Specifically, for the three indicators reviewed, the Missions PMP did not follow USAID guidance in four areas: roviding precise definitions of indicators; P Specifying the source and method for data collection; ssigning responsibility for data collection; and A Describing the procedures for data quality assessments. Without a complete and updated performance monitoring plan, the Mission does not have an effective monitoring and reporting system. Similarly, without establishing performance targets, the Mission can not judge whether its HIV/AIDS program is making progress as planned. The following section will discuss the first three of the four shortcomings in the Missions PMP, followed by a short discussion on the need to establish performance targets. The fourth shortcoming in the PMP (establishing procedures for data quality assessments) will be discussed in a separate finding entitled the Mission Should Assess Data Quality. Providing Precise Definition of Indicators -USAID policy requires a performance monitoring plan to provide consistent and precise definition of performance indicators to assure that comparable data is collected on a regular
10
© 2010-2024 YouScribe