Project Performance Audit Report
Description
ASIAN DEVELOPMENT BANK PPA:PAK 21124 PROJECT PERFORMANCE AUDIT REPORT ON THE PRIMARY EDUCATION (GIRLS) SECTOR PROJECT (Loan 977-PAK[SF]) IN PAKISTAN September 2000CURRENCY EQUIVALENTS Currency Unit – Pakistan Rupee/s (PRe/PRs) At Appraisal At Project Completion At Operations Evaluation (August 1989) (December 1997) (April 2000) PRe1.00 = $0.0474 $0.0285 $0.0184 $1.00 = PRs21.103 PRs35.09 PRs54.19 ABBREVIATIONS ADB − Asian Development Bank CMS − community model school DEO − district education officer FCU − federal coordinating unit FGD − focus group discussion GCET − government college for elementary teachers MOE − Ministry of Education NWFP − Northwest Frontier Province OEM − operations evaluation mission PCR − project completion report PED − provincial education department PIU − project implementation unit PPAR − project performance audit report PTA − parents-teachers association SAP − Social Action Program SMC − school management committee NOTES (i) The fiscal year (FY) of the Government ends on 30 June. (ii) The school year (SY) starts in April. (iii) In this report, “$” refers to US dollars. Operations Evaluation Office, PE-550 CONTENTS Page BASIC DATA ii EXECUTIVE SUMMARY iii MAP v I. BACKGROUND 1 A. Rationale 1 B.Formulation C. Purpose and Outputs 1 D. Cost, Financing, and Executing Arrangements ...
Sujets
Informations
| Publié par | Suen |
| Nombre de lectures | 158 |
| Langue | English |
Extrait
ASIAN DEVELOPMENT BANK
PROJECT PERFORMANCE AUDIT REPORT
ON THE
PPA:PAK 21124
PRIMARY EDUCATION (GIRLS) SECTOR PROJECT
(Loan 977-PAK[SF])
IN
PAKISTAN September 2000
CURRENCY EQUIVALENTS Currency Unit Pakistan Rupee/s (PRe/PRs) At Appraisal At Project Completion At Operations Evaluation (August 1989) 1997) (December 2000) (April PRe1.00 = $0.0474 $0.0285 $0.0184 $1.00 = PRs21.103 PRs35.09 PRs54.19 ABBREVIATIONS ADB− Development Bank Asian CMS− community model school DEO− education officer district FCU− coordinating unit federal FGD− focus group discussion GCET− government college for elementary teachers MOE− Ministry of Education NWFP− Frontier Province Northwest OEM− evaluation mission operations PCR− project completion report PED− education department provincial PIU− implementation unit project PPAR− project performance audit report PTA− parents-teachers association SAP− Action Program Social SMC− school management committee NOTES fiscal year (FY) of the Government ends on 30 June.(i) The school year (SY) starts in April.(ii) The (iii) In this report, “$ refers to US dollars. Operations Evaluation Office, PE-550
CONTENTS BASIC DATA EXECUTIVE SUMMARY MAP I. BACKGROUND A. Rationale B. Formulation C. Purpose and Outputs D. Cost, Financing, and Executing Arrangements E. Completion and Self-Evaluation F. Operations Evaluation II. PLANNING AND IMPLEMENTATION PERFORMANCE A. Formulation and Design B. Cost and Scheduling C. Consultants’ Performance, Procurement, and Construction D. Organization and Management III. ACHIEVEMENT OF PROJECT PURPOSE A. Operational Performance B. Performance of the Operating Entity IV. ACHIEVEMENT OF OTHER DEVELOPMENT IMPACTS A. Socioeconomic Impacts B. Environmental Impacts C. Impacts on Institutions and Policy V. OVERALL ASSESSMENT A. Relevance B. Efficacy C. Efficiency D. Sustainability E. Institutional Development and Other Impacts F. Overall Project Rating G. Assessment of ADB and Borrower Performance VI. ISSUES, LESSONS, AND FOLLOW-UP ACTIONS A. Key Issues for the Future B. Lessons Identified C. Follow-Up Actions APPENDIXES
Page ii iii v 1 1 1 1 2 2 3 3 3 4 5 6 7 7 10 10 10 11 11 12 12 12 12 13 13 13 14 15 15 16 17 18
BASIC DATA Primary Education (Girls) Sector Project (Loan 977-PAK [SF]) PROJECT PREPARATION/INSTITUTION BUILDING TA No. TA Project Name Type Person-Months Amount Approval Date 907-PAK Primary Education PPTA 15.5 $150,000 13 Oct 1987 (Girls) Project 907-PAK Supplementary PPTA $25,000 23 Aug 1988 KEY PROJECT DATA($ million) Actual per ADB Loan As Documents Total Project Cost 80.51 52.85 Foreign Exchange Cost 8.44 6.39 ADB Loan Amount: Utilization 64.20 42.61 ADB Loan Amount: Cancellation 0 21.59 Amount of Cofinancing 0 3.90a KEY DATES Expected Actual Fact-Finding 27 Jan-23 Feb 1989 Appraisal 22 Apr-4 May 1989 Loan Negotiations 28-30 Sep 1989 Board Approval 26 Oct 1989 Loan Agreement 11 Dec 1989 Loan Effectiveness 24 Jan 1990 14 May 1990 First Disbursement 30 Sep 1991 Project Completion 31 Dec 1994 30 Sep 1996 Loan Closing 31 Jul 1995 21 Mar 1997 Months (Effectiveness to Completion) 59 76 KEY PERFORMANCE INDICATORS(%) Appraisal PCR PPAR Financial Internal Rate of Return nc nc nc Economic Internal Rate of Return nc nc nc BORROWERIslamic Republic of Pakistan EXECUTING AGENCYMinistry of Education MISSION DATA Type of Mission No. of Missions Person-Days Fact-Finding 1 72 Appraisal 1 75 Follow-Up 1 3 Project Administration Inception 1 8 Review 5 39 Disbursement 1 2 Special Loan Administration 3 36 Project Completion 2 30 Operations Evaluation 1 34 ADB = Asian Development Bank, nc = not calculated, PAK = Pakistan, PCR = project completion report, PPAR = project performance audit report, PPTA = project preparatory technical assistance, TA = technical assistance. ain November 1990 to finance the innovative activitiesThe Norwegian Government provided a grant of $4.5 million component of the Project of which $3.9 million equivalent was expended.
EXECUTIVE SUMMARY
Enrolment Goals Stymied as Quality of Education Lags Behind primary education supported by the Asian Development BankThe Project was the first in (ADB). At the time of appraisal, the literacy rate among women in Pakistan was 14 percent (5 percent in rural areas), one of the lowest in the world. Girls’ participation rate1was only 33 percent (compared with about 70 percent for boys), and only half of the girls who started school finished class 5. The Government’s National Education Policy (1979) targeted the attainment of universal primary education for girls by 1992. The objectives of the Project were to increase access to schooling for girls in rural areas, improve the quality of instruction, increase the retention rate of female students, and upgrade the management and supervision of the primary education system. The project design was based on four concepts: (i) five-room community model schools (CMSs), one teacher per grade rather than single-room multigrade, single-teacher schools; (ii) a CMS acting as a miniresource center to the cluster of small schools around it; (iii) a learning coordinator at the district level to provide assistance to CMS and cluster school teachers to improve subject content and instructional methodology; and (iv) involvement of parents and the local community through the establishment of school management committees (SMCs) and parents-teachers associations (PTAs). The Project was the first phase of an envisaged series of projects in support of the Government’s policy of promoting universal primary education, and established 832 CMSs in 20 percent of all union councils2 in the country. The actual cost of the Project at completion was $52.85 million equivalent (66 percent of the original estimate). Of this amount, the Government of Pakistan financed $6.34 million equivalent (12 percent) of the local currency cost, and ADB contributed $42.61 million (80.6 percent). The Norwegian Government provided a grant amounting to $3.9 million equivalent (7.4 percent) for the innovative activities component. Total loan cancellations amounted to $21.59 million (34 percent of the loan amount). The loan was approved in October 1989 and the Project was completed in September 1996. A delay of 17 months in completing the Project was caused by difficulties in the organization of subproject (provincial) appraisal teams, the appointment of staff to the federal coordinating unit and the project implementation units, and the inadequate monitoring and information system. While providing a number of well-provided model schools, disparities were generated in terms of facilities, learning materials, and human resources among schools and communities in the villages. The formulation and design of the Project would have been more relevant if they had been directed toward the improvement of smaller (2-3 room) schools or the many shelterless schools that comprised 40 percent of all schools at the time of appraisal. While accomplishing its target of increased access (35,000 new student places), the Project did not raise the quality of education significantly. In spite of enrolments increasing four-fold in project schools in the last 10 years, the schools are operating below capacity. This underutilization is due mainly to the high dropout rate (more than 50 percent) over the five-year schooling period, and the large number of school-aged girls who stay out of school (62 percent of girls 5-9 years do not go to school). Parents must be convinced that education is of sufficient quality because of the cost of education and the opportunity cost of children’s work. Outcome indicators measuring quality remain poor: 20 percent of students repeat a grade level, the student-teacher 1 Participation rate is defined as the percentage of 5-9 year old girls enrolled in primary school. 2 A union council is a village with an average population of about 20,000.
iv ratio has worsened from 40:1 to 60:1, and class sizes have grown to as many as 55-80students per teacher. The elimination of multigrade teaching, one of the objectives of the Project, has not been attained. Considering the 20 percent underutilization of facilities financed under the Project and the higher construction cost per student place in comparison with other similar projects, the achievement of project purpose in relation to the use of inputs under the Project for the hardware component is considered inefficient. The use of project inputs for the software component was minimal. This highlights the need for a longer gestation period and continuing inputs over the long term. Internal and external efficiencies are low. The need for a comprehensive database and information system was not addressed. There is no monitoring system for educational improvement for the CMSs aside from the minimal supervision of the district educational office. The Project benefited mainly lower income families, the large majority of whom gain their livelihood in farmingfamilies that survive on around PRs30 ($0.50) per person per day. The major contribution of the Project lies in its sociocultural impact, i.e., its contribution to the behavioral change of the mostly illiterate rural people so that girls’ education is more readily accepted now. However, parents’ involvement, community participation, and links between school and local community are still weak. The establishment of SMCs and PTAs is a significant improvement in strengthening the school-community relationship that will ensure quality and sustained improvement in the long term. The quality of instruction has not improved much as evidenced by the poor education indicators in the project schools. Project design was too optimistic in its expectations of change in social attitudes and did not pay enough heed to the need for a much longer capacity-building and development period. It did not achieve its purpose with regard to: (i) establishing CMSs as the resource center of the cluster schools; (ii) establishing learning coordinators as catalysts for quality improvement; and (iii) elimination of multigrade teaching. The inclusion of a kindergarten class in the project design would have made a major impact on increasing enrolment. There are sustainability problems due to the perennial lack of budgetary allocations for operation and maintenance. Sectorwide issues that are systemic in nature have constrained the performance of the Project. Systemic problems include the lack of teachers (caused mainly by the ban on government hiring), inadequate budgetary allocations for operation and maintenance, weak management, and the absence of an effective monitoring and information system. Nonetheless, the Project has laid a foundation for efforts to build on. The physical facilities, teachers, and administrators are in place. PTAs and SMCs have made a good, albeit a tentative start. Taking into account the less than satisfactory state of project outcomes since the Project was 3 completed, the Project is rated less than successful.However, with the groundwork laid by the Project and the lessons identified from its design, implementation, and outcomes, it is expected that a follow-up project, already in operation, will be better positioned to produce the desired results.
3 ADB’s Projects Division feels strongly that the Project should be rated successful for the following reasons: (i) the Project was highly relevant to Pakistan's needs and ADB's strategy; (ii) the Project achieved its main aim of increasing school access for girls; (iii) the lack of enrolment is attributable more to socioeconomic constraints than to poor educational quality; (iv) underutilization of facilities is temporary and will improve with better social mobilization efforts to recruit girls; (v) the improvements in classroom learning environments have greatly improved quality in the project schools; and (vi) the project achievements are sustainable in spite of current budgetary shortage.
v
I. BACKGROUND
A. Rationale 1. The Project4was the first financed by the Asian Development Bank (ADB) in support of primary education. Prior to this, the emphases in the sector had been on technical education, vocational training, engineering, and science education. At the time of appraisal, the literacy rate among women in Pakistan was only 14 percent (5 percent in the rural areas), one of the lowest in the world. Low school attendance was a major reason for this, a consequence of the unavailability of primary schools for girls, particularly in rural areas. There were about 90,000 primary schools of which 24,000 were girls’ schools (in Pakistan there are separate schools for boys and girls). Girls’ participation rate5at that time was only 33 percent (compared with about 70 percent for boys) and only half of the girls who started school finished class 5. The Government’s National Education Policy (1979) targeted the attainment of universal primary education for girls by 1992, and for boys by 1986. The Project supported the Government’s education program and complemented those of other aid agencies, particularly the World Bank, the United States Agency for International Development, and the United Nations Children’s Fund. B. Formulation 2. In March 1987, the Government requested ADB to provide technical assistance for the development of a project to support primary education for girls in the rural areas. A sector study on education6 and an education sector report7 identified assistance to primary education (classes 1-5, ages 5-9 years), with the emphasis on girls in rural areas, as one of the priority areas for ADB’s involvement in Pakistan. A project preparatory technical assistance8 was provided in 1987 and was carried out by international consultants from May to November 1988. The Project was appraised in April-May 1989 and the loan became effective on 14 May 1990. A project framework has been constructed for the purpose of this report and is shown in Appendix 1. A follow-up project became effective on 19 January 1998.9 C. Purpose and Outputs 3. The specific objectives of the Project were to (i) increase the participation rates of girls; (ii) improve the quality of instruction; (iii) increase the retention rate for girls; and (iv) upgrade the management and supervision of the primary education subsector. The purpose of the 4 977-PAK(SF): LoanPrimary Education (Girls) Sector Project, for SDR51.711 million ($64.2 million), approved on 26 October 1989. 6578 PPETaraAktiisctiapna:ti oSne crtaotre iSst uddeyfirnto,en AdEs adsiau tnchaeDtieopvne 9-5 fo egatnecr elsir gld oaryenip iramrnloel dl. ry schooaB tnempirpA ,kniaAs, loveDen 88. l 19nempaB t ,knrbeFloeuary 1989. ducation Sector Repo 907-PAK:Primary Education (Girls), for $150,000, approved on 13 October 1987, with supplementary funding 9f $orL oan 1454pP-arKpAvSo(d)eF 5:20,00 ,Soenc o2n3dAugust 1988.ject, for SDR31.178 million ($45 million), approved Girls Primary School Sector Pro on 15 August 1996.
7
Project was to make primary education more accessible to girls in rural areas, and increase the quality and efficiency of the primary education system for girls in Pakistan. This was to be achieved through the establishment of community model schools (CMSs) in selected rural areas covering 20 percent of all union councils10 in the country. The Project consisted of three parts: (i) an institutional development component that comprised teacher training and staff development, benefit monitoring and evaluation, provision of instructional materials and library books, sector research, and building maintenance; (ii) an infrastructure development component that comprised the construction of about 800 CMSs, and the provision of furniture and equipment; and (iii) an innovative activities component to overcome sociocultural constraints to the education of girls. 4. The bulk of project resources went to the first two components. The innovative activities component was intended to introduce a number of educational innovations on a pilot basis. These activities were to include (i) provision of transport for teachers and students; (ii) construction of teachers’ hostels in strategic locations; (iii) recruitment of qualified women in their communities as teachers; (iv) establishment of incentive packages for students, teachers, headmistresses, and high-performing school management committees (SMCs); (v) establishment of teachers’ resource centers; (vi) recruitment of outstanding primary school graduates as teaching assistants; and (vii) orientation of agencies connected with the project implementation procedures. In November 1990, the Government obtained a grant of $4.5 million from the Government of Norway to finance the innovative activities component. D. Cost, Financing, and Executing Arrangements 5. At the time of appraisal, the total project cost was estimated at $80.51 million equivalent, with a foreign exchange component of $8.44 million (about 10 percent of the total cost, including $1.08 million inservice charges) and a local currency component of $72.1 million equivalent (about 90 percent of the total cost). ADB provided a loan of $64.2 million from its Special Funds resources (80 percent of the total project cost). The Government was to finance part of the local currency cost estimated at $16.34 million equivalent. The grant from the Government of Norway reduced ADB’s financing to $59.7 million. Overall management of the Project was under the Ministry of Education (MOE), whose joint educational adviser on primary and nonformal education was appointed the director of the federal coordinating unit (FCU) on a part-time basis. FCU project and deputy project managers were appointed on a full-time basis. The directors of the four project implementing units (PIUs) were given full responsibility for implementing components at the provincial and district levels in their respective provinces. The management employed by MOE comprised multiple levels of steering and coordinating committees, i.e., girls primary education development steering committee, provincial coordinating committees, district coordinating committees, and SMCs. E. Completion and Self-Evaluation 6. The project completion report (PCR) was circulated to the Government and MOE, and in ADB in June 1998. The report noted the significant delays at the outset of the Project due to late identification of schools to be assisted and sites for new schools caused in turn by the delay in appointing project staff in the four provinces and at the FCU in Islamabad. The PCR rated the 10 A union council is a village with an average population of about 20,000.
8
Project’s infrastructure development component highly successful, noting that the CMS targets had been achieved and the quality of construction was above average. On the other hand, the institutional development component had mixed success. Targets were not met due to poor coordination of the inservice teacher training and the government colleges for elementary teachers (GCETs) for women. The Project’s innovative activities component comprised pilot programs intended to improve the teaching-learning process and the social environment. Some of these activities worked well initially and were considered successful, but many, such as promotional campaigns, have not been sustained. The PCR concluded that the Project made a significant contribution to primary education for girls, and rated the Project generally successful. F. Operations Evaluation 7. This project performance audit report (PPAR) presents an assessment of the Project’s effectiveness in terms of achieving its objectives, generating benefits, and ensuring the sustainability of the operations. It discusses issues of current relevance to the sector, identifies lessons, and presents follow-up actions that need to be taken by the Government and/or ADB. The discussion and results are based on the findings of the Operations Evaluation Mission (OEM) which was fielded on 28 March-13 April 2000; a review of the appraisal report, PCR, and project files; and discussions with ADB staff and Government officials. The PPAR draws data from three sources: (i) survey questionnaires, (ii) focus group discussions (FGDs), and (iii) secondary statistical data from the concerned ministries and provincial education departments (PEDs). A survey of headmistresses, teachers, students, and school leavers was conducted in 86 CMSs and 88 nonproject schools in the four provinces (Appendix 2). To complement the quantitative survey, FGDs were held in the CMSs visited. Separate discussions were held with teachers, class 5 students, school leavers, parents, and union council members. The findings of the FGDs, interviews, and school visits were used as a crosscheck on the consistency and reliability of the survey data. 8. This evaluation will provide feedback through the midterm review to improve the implementation of the ongoing follow-up project (footnote 6). The follow-up project is the second phase of the overall plan to increase girls’ enrolment and literacy rates, and targets another 25 percent of the 4,000 union councils in Pakistan. The views of the concerned ADB departments and offices are reflected in the report. Copies of the draft PPAR were provided to the Borrower and the Executing Agency for comments in June 2000. Although the request was subsequently followed up, no comments were received. It is, therefore, assumed that neither the Borrower nor the Executing Agency wishes to comment on the PPAR. II. PLANNING AND IMPLEMENTATION PERFORMANCE
A. Formulation and Design 9. The project design was supportive of the Government’s policy of universal primary education, especially promoting primary education for girls, as embodied in the Seventh Five-Year Development Plan (1988-1993) and the National Education Policy (1979). The design was based on four concepts: (i) five-room, one-teacher-per-grade model schools instead of single-room, multigrade, single-teacher schools; (ii) the CMS as a miniresource center for the cluster of small schools around it in the rural union council; (iii) a learning coordinator at the district level to
9
help CMS and cluster school teachers improve subject content and instructional methodology; and (iv) involvement of parents and the local community through the establishment of SMCs and parents-teachers associations (PTAs). While the CMS concept was considered appropriate at the time, it eventually created an unwelcome disparity in terms of facilities, learning materials, and human resources between the CMSs and other schools in the villages. It is to be noted that at the time of project formulation about 40 percent of the 90,000 primary schools were shelterless,11 and in Punjab Province, only 20 percent of all schools had more than two separate classrooms to accommodate five grades. 10. The construction cost of $279 per student place as appraised was about 10 percent higher than that of the World Bank’s primary education projects in Pakistan, estimated at $254 per student place. The furniture unit cost was 28 percent higher$19 per student place against the World Bank’s $15. Given the sector requirements, the design of the Project was more hardware oriented (64 percent of the base cost in the appraisal report was for civil works, furniture, and equipment).12The cost effectiveness of the infrastructure development component of the Project was undermined by the underutilization of the CMSs (paras. 22-24). In spite of the rigid site selection criteria agreed upon during appraisal, the significant number of CMSs with the less than expected 200-student enrolment points to difficulties in the selection of sites by the provincial appraisal teams. The World Bank considers the selection and approval of sites as one of the most intractable problems in school construction. Political interference in the process delayed construction programs by as much as two years. 11. Institutional and human resource development and innovative activities, were integral parts of the Project, addressing quality improvement aspects in the school and community environment. Only half of the targeted 16,000 female teachers to be trained through inservice refresher courses were actually trained. Where SMCs and PTAs have been established, they are making an impact on the learning achievements of the students and the overall school operation. 12. The complex project design required multilayered management and coordination involving MOE and PEDs, as well as various committees at the national, provincial, and district levels. Project implementation highlighted the difficulty of implementing an umbrella-type national project that spans individually implemented provincial components. According to the PCR (paras. 6, 12, 25), delays at the start of the Project, especially those associated with the organization of subproject appraisal teams and the appointment of staff to the FCU and PIUs, occurred because of the inexperience of MOE, PEDs, and ADB (as this was the first primary education project supported by ADB). On the other hand, one reason cited in the appraisal report for the Project’s sector approach was the experience of MOE and PEDs in implementing such a project. 13. While more classrooms have been provided, there is a 20 percent underutilization of project school buildings. The Project was too ambitious in its goal of changing the sociocultural environment and did not pay enough attention to the need for a much longer capacity-building and development period. It did not achieve its purpose with regard to: (i) establishing CMSs as the resource center of the cluster schools, (ii) establishing learning coordinators as catalysts for quality improvement, and (iii) the elimination of multigrade teaching. The Project lacked a realistic policy framework, resources and logistic support, and leadership at the district level. Subsidiary objectives that were not met include the database and information system, and most 11 contrast, the World Bank provided $513.5 million in Infour primary education projects between 1987 and 1995 to assist in improving facilities of the poorer (1-2 room) schools and shelterless schools. The projects provided more than 50,000 new classrooms, 6,000 new schools, and built roofs over more than 10,000 shelterless schools. 12 Actualfor infrastructure development increased to 79 percent of total project cost. project expenditure
© 2010-2024 YouScribe