Mid-term evaluation of the second Marine science and technology programme (MAST II)

Directorate-General For Research And Innovation European Commission

Description

Research policy and organisation

Informations

• Publié par: DIRECTORATE-GENERAL-FOR-RESEARCH-AND-INNOVATION-EUROPEAN-COMMISSION
• Nombre de lectures: 9
• Langue: English
• Poids de l'ouvrage: 2 Mo

Extrait

ISSN 1018-5593
■ü ύ -ü
i:
■d A
ΑΛώ
European Commission
Mid-term evaluation
of the second Marine Science
and Technology Programme
(MAST II)
Report
EUR 16610 EN Research evaluation - Report No 69 European Commission
l-term evaluation
of the second Marine Science
and Technology Programme
(MAST II)
Authors: T. Audunson (Chairman)
G.E. Fogg
S. Guerzoni
X. Niell
P. Ré
G. Siedler
The translation of the Executive Summary in other official languages
of the European Union will be published separately.
1995 EUR 16610 EN Published by the
EUROPEAN COMMISSION
DIRECTORATE GENERAL XIII
Telecommunications, Information Market and Exploitation of Research
L-2920 Luxembourg
LEGAL NOTICE
Neither the European Commission nor any person acting on behalf of the Commission
is responsible for the use which might be made of the following information
Cataloguing data can be found at the end of this publication
Luxembourg: Office for Official Publications of the European Communities, 1995
ISBN 92-827-4964-9
© ECSC-EC-EAEC Brussels · Luxembourg, 1995
Printed in Belgium CONTENTS
PAGE
EXECUTIVE SUMMARY ν
The FutureContinuationorAlteration of the Programme.Vi
The ManagementoftheProgramme.vi­¡
The Use ofResearchResultsbythe Organizations. vi­¡­j
carrying outthework.
The transferoftechnologytoother organizations,
by movementsofpersonnel,bylicensing,andbyothermeans.ix
1. GENERAL INTRODUCTION1
1.1 Mandate. 1
2. THEMASTIIPROGRAMME: BRIEF DETAILS
OFOBJECTIVES.
Ç
3. GENERALSUMMARY OF PROGRAMME
STATISTICS9
3.1BreakdownBy Area.TO
3.2ParticipationBy Country.12
3.3 n By Institution Type. 13
3.4 Large TargetedProjects.13
3.5 Supporting Initiatives.Η
4. THE EVALUATIONPROCEDURE15
4.1 The Work Plan andTimeTable15
4.1 Strategy. 1b
5. PRESENTATIONOF QUESTIONNAIRE FINDINGS 17
5.1GeneralObservations17
5.1Administrative Matters.18
5.2 Programme Impacts.19
5.3 e Deliverables.20
6. PROGRAMME EVALUATION ¿3
6.1 Criteria and proceduresusedfortheappraisal
and selection ofprojectproposals.23
6.2 The proceduresfortheauthorisationincluding
the respective rolesoftheCommission and the CAN. 25
6.3 The monitoringofthemidtermscientific and
technical achievementsoftheProgramme, takinginto
account its originalobjectivesandmilestonesand
whenever relevant ofchangedcircumstances. 26
6.4 The efficiency and effectivenessofmanagementand
the use of resources. 27
6.5 TheWorkProgrammeand its RelationtotheTechnical
ObjectivesoftheMASTII Programme.29
m PAGE
6.6 The Programme contribution to the development of
community policies and to the social and economic
development of EU. 29
6.7 Thebenefitsresultingfrom the implementation of
theProgrammeatCommunity level (Community added value)30
6.8 Overlapsorsynergiesbetween the Programme
andotherrelatedprogrammes and activities. 31
ANNEX
1 MASTIIMid­TermEvaluationPanelMembership List 35
2 Listofpersonsinterviewed by theMASTΠ
Mid­Term Evaluation Panel37
3 Questionnaire Survey of M AST Π Participants 41
4Statistical Summary of QuestionnaireSurvey53
5. List of Acronyms and Abbreviations¿3
6 Rating of Proposals pr. Area 65
7List of AdvancedStudyCourses67
TV EXECUTIVE SUMMARY
In accordance with the legal requirements under the Statutes governing R&D programmes
financed by the CEC, MAST II must be evaluated [Council Decision 90/221/Euratom,
EEC:Article 2(4) and 5]. The administration for doing this has been provided by the
Evaluation Unit of the Directorate General for Science, Research and Development of the
CEC, which is entirely independent of programme management within the CEC. The
evaluation itself was carried out by a panel of six independent experts in the field (see
Annex), appointed by the Director General of Directorate General XII with advice from the
Marine Science Advisory Committee (CAN-MAST) and the Evaluation Unit, which also
provided one of its members as Secretary.
Information has been obtained by examination of documents relating to the Programme and
in interview with Commission Programme Officers (CPO),with members of CAN MAST,
with project co-ordinators and participants. The confidential nature of these interviews was
always emphasized. To gain understanding in depth of the working of MAST II each panel
member was allocated two projects, one of his own choice and another chosen at random, to
examine in detail by interviews with both the responsible CPO and co-ordinators. The
intention here was to identify strengths and weaknesses in procedure and management, as
well as to assess the process of evaluation of application for funding itself. The projects
selected for such examination covered a broad spectrum of science, size, application and
countries.
An overview of the motives, experiences and reactions of participants was obtained by means
of a questionnaire prepared under the guidance of the Panel Chairman and sent to all (i.e.,
-400) MAST II participants. It was designed to obtain, in quantitative form as far as possible,
information about their motivations and expectations, the management of projects and
impressions of EC procedures. About 305 responses were received of which about a quarter
came from project co-ordinators and the rest from other partners. The return rate was about
80 per cent, which is considered quite satisfactory. These data have been subjected to detailed
statistical analysis. The individual completed questionnaires have been treated in strict
confidence and only the aggregated data will be presented in this Report. In parallel with this
gathering of information the Panel held discussions and identified topics for special
consideration.
The main recommendation and findings included in the report are summarised below. The Future Continuation or Alteration of the Programme
The Panel recommends intensified Programme information action, including sectors
¡fe
outside the academic marine sciences community. (15)
The Panel recommends strong emphasis in the evaluation process on well defined and
suitable project scope and deliverables. Some projects were obviously too broad in their
approach and would gain in scientific strength by stronger focusing with well defined
project objectives and clearly identified scientific and technological deliverables,
bearing in mind the limited project lifetime. (15&21)
The Panel, while emphasising the requirement of scientific excellence to be the prime
criterion with the largest weight in the selection of projects, recommends a clearer
specification of the weight of other projectn criteria. (21)
The Panel recommends that the transparency of the project selection evaluation process
should be improved.('22J)
The Panel recommends strengthening requirements to define project deliverables and
end user interaction which would also strengthen the technology aspects of M AST.( 16)
The Panel recommends, based on the noted discrepancy in funding reduction compared
to reduction in objectives early consideration of proper action in order to reach the
goals of the projects within their funding Iimits.fi6)
The Panel recommends consideration should be given to more frequent reporting on big
projects follwing a simplified standard formats that would take little time to complete.
(17)
The Panel, while welcoming the possibility of funding some shiptime in MAST III,
recommends that procedures should be introduced which ensure that funding of
projects based on a provision of ships from national institutions will depend on a firm
commitment of ship time being provided.f25)
(The number in parentheses refers to the page in the report where the respective recommendation is discussed)
vi The Panel recommends initiatives by the Commission, possibly involving Supporting
Initiatives, to encourage stronger integration between CAN-MAST delegates and
project participants in the various Community and EEA states.( 16)
The Panel recommends the establishment of procedures to facilitate flexible approach
in interfacing marine, terrestrial and atmospheric programmes and for a closer
interaction of the scientific and the technological communities.(28)
The Panel notes with satisfaction the overall success of the MAST II Programme in
achieving its main objectives and strongly endorses the continuation of MAST
activities.f^ff,)
The Panel notes the success of MAST-II in increasing the cohesion within the marine
scientific and technological communities in Europe and stresses the continuing need to
provide a chance for less experienced groups to gain expertise through integrated
projects. (27)
The Management of the Programme
The Panel recommends that actions be taken to ensure that the project application
format is compatible with the short review process.f27)
The Panel recommends actions be taken to ensure that the composition of the
Evaluation Panels reflects scientific competence and, whenever appropriate,
competence in the technological and end user needs.(20)
The Panel recommends that for large projects the coordinator should have an
experienc