The Keynote Project – Audit
Description
The Keynote Project – External Audit © The Keynote Project 2002. Produced by The Nottingham Trent University, The London Institute and The University of Leeds funded under the Fund for the Development of Teaching and Learning by the Higher Education Funding Council for England and the Department for Employment and Learning. Please include this copyright information on any reproduction of these materials. © The Keynote Project 2002 1 External Audit 1.0 Introduction The aim of the Keynote project is to identify, disseminate and develop the key skills of textiles, fashion and printing students, thereby enhancing their employability whilst promoting the skills of lifelong learning. The project is a collaboration between The Nottingham Trent University (lead partner), The London Institute and The University of Leeds. iThe purpose of the audit was to establish how key skills, as defined by the Dearing Report, were ii being developed and assessed in higher education institutions (HEI’s) in England who offer programmes within the QAA Materials Technology heading. Analysis of the Materials Technology subject review report aided the institutional selection process. Institutions where key skills had been identified as a strength were targeted for audit. It was envisaged that 16 institutions offering Materials Technology programmes would be visited. This proved rather difficult and after discussions with our NCT ...
Informations
| Publié par | Uproi |
| Nombre de lectures | 25 |
| Langue | English |
Extrait
© The Keynote Project 2002
1
External Audit
The Keynote Project – External Audit
© The Keynote Project 2002. Produced by The Nottingham Trent University, The London Institute and
The University of Leeds funded under the Fund for the Development of Teaching and Learning by the
Higher Education Funding Council for England and the Department for Employment and Learning.
Please include this copyright information on any reproduction of these materials.
© The Keynote Project 2002
2
External Audit
1.0 Introduction
The aim of the Keynote project is to identify, disseminate and develop the key skills of textiles,
fashion and printing students, thereby enhancing their employability whilst promoting the skills of
lifelong learning.
The project is a collaboration between The Nottingham Trent University (lead
partner), The London Institute and The University of Leeds.
The purpose of the audit was to establish how key skills, as defined by the Dearing Report
i
, were
being developed and assessed in higher education institutions (HEI’s) in England
ii
who offer
programmes within the QAA Materials Technology heading.
Analysis of the Materials Technology
subject review report aided the institutional selection process.
Institutions where key skills had
been identified as a strength were targeted for audit. It was envisaged that 16 institutions offering
Materials Technology programmes would be visited. This proved rather difficult and after
discussions with our NCT project mentor permission was granted to make 14 visits in total and for
a limited number of these to included institutions offering programmes in the Art & Design subject
area.
Due to time constraints, it was necessary to select the Art & Design HEIs prior to the
completion of their subject review; therefore the Keynote project team’s own professional
networks were used.
Ultimately, visits were made to fourteen higher education institutions
comprising 11 Materials Technology and 3 from the Art & Design subject areas.
2.0 Methodology
Initially, letters providing background information about the Keynote project were sent to the
selected institutions and participation was invited. This type of ‘cold calling’ was largely ineffective.
Directed personal contact by email and telephone was found to be necessary to secure
involvement.
Participating institutions were sent further information about the project, together
with student/staff audit questionnaires.
Follow up visits to the institutions were devolved
regionally between the consortium partners.
During these visits, interviews were held with staff
and students.
The interviews followed an agreed format to ensure a balanced and uniform review
at each institution.
The interview transcripts and the responses to questions were collated and
the resulting information is summarised in the body of this report.
Student/staff comments are
included as appropriate in order to reinforce certain points made.
3.0 The Findings
3.1 Various factors have influenced institutional responses to the ‘key skills agenda’. These are listed
in no particular order of preference:
•
Professional bodies
•
Widening participation
•
Lifelong Learning
•
The development of the LTSN (Learning & Teaching Support Network)
•
Individual institutions’ Learning and Teaching Strategy
•
The Dearing Report
•
Benchmarking
•
The need to be able to define ‘graduateness’
•
The impact of Curriculum 2000
36% of the HEIs questioned cited professional bodies and/or employer networks as the main
drivers for institutional change regarding key skills development.
Three institutions stated that the increasing need to define ‘graduateness’ led to the
development of the key skills curricula.
“The university noted the requirement for key skills in the Dearing Report”
(member of staff)
© The Keynote Project 2002
3
External Audit
3.2 Recent evidence suggests that higher education is increasingly being affected by the policy
drivers within the compulsory education sector.
Five of the participating institutions mentioned the
impact of Curriculum 2000
iii
within which a new qualification called the ‘Key Skills Award’
iv
has
been developed and all young people are being encouraged to seek this certification.
Staff
interviewed stated that this initiative made first year students more conversant with the discourse
of key skills, resulting in a change in institutional culture with regard to key skills.
3.3 The development of key skills was a specific aim in the mission statement and/or learning and
teaching strategy of three institutions questioned.
4.0
Staffing
The roles and responsibilities of staff managing key skills development in institutions varied
enormously.
In two instances, the careers teams had sole responsibility for all issues relating to
key skills.
In three institutions this fell to the Learning and Teaching Coordinators.
In another, full
responsibility lay with the course Programme Director.
The remaining institutions did not have
clearly identified personnel heading up this area of work.
This very fragmented approach and
lack of an agreed staffing structure inhibits institutions working together in adopting a nationally
recognised policy on key skills development.
It is interesting to note that the responsibility for key
skill development cuts across academic and support staff, causing confusion as to whether it is
essentially an academic curriculum issue, a careers employability issue, or a combination of both.
5.0 Assessment
The approach to the assessment of key skills was fairly consistent.
85% of HEIs surveyed
indicated that key skills were fully embedded within curricula and are therefore developed and
assessed within the framework of the subject area.
“Consensus at the university was that they [key skills] should be fully integrated.”
(member of
staff)
5.1 Specific ‘stand alone’ key skills modules were only offered at two of the institutions and included:
•
Core Skills
•
Foundation level Mathematics
•
IT
•
Learning to learn
•
Study Skills
5.2
In the majority of the institutions, there was no explicit tracking of key skills development.
Key
skills need to be contextualised and embedded within the curriculum in order to ensure relevance.
However, this makes monitoring of development and progress very difficult.
One student
commented: [Key skills]
‘aren’t recorded to my knowledge – except we do have tutorial type
sessions.’
5.3 In two HEI departments, tracking of skills development was offered as part of the tutorial system.
In another, learning journals were used as a vehicle for this.
One Materials Technology
department, which had received project funding in this area, fully mapped and tracked key skills
development and assessment opportunities for all their undergraduates and offered an accredited
certificate in personal transferable skills.
However, very few students availed themselves of this
opportunity to gain key skill accreditation suggesting that integrated models of development and
assessment may meet the needs of most students, or perhaps that the perceived value of such
accreditation was low, given the pressures of other responsibilities.
© The Keynote Project 2002
4
External Audit
5.4 In general, key skills were not assessed at entry level.
When assessment was carried out it was
limited to identifying students who might need remedial support in a specific area, rather than
being a mechanism for tracking skill levels across the board.
“They may be asked to complete a diagnostic test in mathematics to assess what remedial help is
needed.”
(member of staff)
“Handwritten exercises help to identify dyslexic students.”
(member of staff)
5.5 In institutions which offered diagnostic testing, a full package of support was available for
students.
This included, in order of frequency mentioned:
•
On-line support
•
Study packs
•
Open learning centre resources
•
English as a foreign language support
•
Dyslexia support
•
Time management
This package was organised by learning support teams, library, or careers staff.
Interestingly, a
key skill support pack in one HEI had been written by the Students’ Union.
6.0 Definitions
64% of the sample had agreed definitions for key skills.
Within this group, 14% used the skills
identified by Dearing
v
whilst the remainder had devised lists internally which had been agreed at
departmental or university level.
For example, one institution’s key skills list included:
•
Leadership
•
Innovation
•
Determination
•
Initiative
6.1 There was a significant lack of agreement about the definition of key skills in institutions which did
not use the Dearing
5
skills list.
Thus, although business awareness was seen as a key skill in one
HEI, and leadership in another, these were not mentioned by any of the remaining institutions.
A
benefit of using the ‘Dearing definition’ is that it provides a common language across the sector.
A disadvantage is that at individual institutional level there may not be the same degree of
ownership of the key skills agenda (it may be seen as being externally imposed).
Institutions that
have negotiated and defined their own key skill attributes will feel a higher degree of relevance to
their particular area of study and related graduate employability needs.
Disadvantages could be
that the identified skills may be idiosyncratic, may not link with Curriculum 2000 or wider employer
needs and this may affect opportunities for student progression.
7.0 Staff Development
78% of the institutions had not organised staff development to specifically address the subject of
key skills.
In some instances, they had been referred to in staff development events, but this was
only done explicitly in three institutions.
This is surprising, given that many of the institutions had
adopted an embedded model requiring all programme team members to be confident in their
development and assessment.
8.0 Student Questionnaires
Wherever possible, institutional visits included focus group sessions with students.
After general
discussions to gather qualitative data (which followed an agreed format), the students were asked
to complete a questionnaire (quantative data) focusing on the five targeted key skills:
Communication, Numeracy, IT, Learning to learn and Working with others.
Questions included:
© The Keynote Project 2002
5
External Audit
•
Which of the skills do you think you have developed during this year?
•
Which skills do you think have been assessed formally during this year?
•
What is your perceived level of competence with regard to these skills? (at the
beginning and end of the course)
•
How important do you think these skills will be to your future career prospects?
73 questionnaires, 42 from Materials Technology and 31 from the Art and Design subject area,
were returned.
The sample included first, second, third and fourth year students.
In general, Year 1 students straight from ‘A’ levels or foundation courses had a very high perception of
their skill competence level. Year 2 students were much more aware of the different backgrounds and
varying skill competence levels of their fellow students, so their perceived skill competence level
rating was much more realistic. Consequently, perceived skill competence levels dropped at the
beginning of Year 2.
It was found that the perceived level of competence followed a natural curve – a
perception of very high competence levels in Year 1 with this level dipping at the beginning of Year 2
and 3, slowly rising to peak at the end of final year - when all students, who answered the questions
rated their perceived skill competence level as satisfactory and above for all areas.
8.1 Key Skills and assessment
When, where and how key skills are assessed is a difficult issue for both staff and students.
Is it
possible to assess all skills in every module, or more to the point is it necessary to assess all skills in
every module? Certainly, students’ perception is that formal assessment attaches a higher level of
importance to an activity.
“If it’s not assessed it’s not on my list of priorities.”
(student)
Students perceived that skills such as note taking, use of email, use of the internet and sourcing
material were ‘background’ tasks and consequently were not assessed formally.
They considered
working with others difficult to assess as this area contains elements which are largely based on the
perceptions of others.
8.2 Key Skills development
It is accepted that to be of benefit key skill development needs to be contextual and relevant to the
programme of study being undertaken.
© The Keynote Project 2002
6
External Audit
Materials Technology Students' Perceptions
Which key skills have not been developed during the year?
0
10
20
30
40
50
60
70
80
Draw ing
Layout
General
Mathematical
Skills
Graphics
Manipulation
CAD/CAM
Leadership
Motivational
skills
% of
Students
Over 45% of the students surveyed thought these skills had not been developed during the year
Art & Design Students' Perceptions
Which key skills have not been developed during the year?
0
10
20
30
40
50
60
70
80
T
e
c
h
n
i
c
a
l
R
e
p
o
r
t
s
G
e
n
e
r
a
l
M
a
t
h
e
m
a
t
i
c
a
l
S
k
i
l
l
s
C
o
l
l
e
c
t
i
n
g
&
R
e
c
o
r
d
i
n
g
d
a
t
a
A
n
a
l
y
s
i
s
o
f
d
a
t
a
P
r
e
s
e
n
t
a
t
i
o
n
o
f
r
e
s
u
l
t
i
n
g
d
a
t
a
T
e
x
t
M
a
n
i
p
u
l
a
t
i
o
n
G
r
a
p
h
i
c
s
M
a
n
i
p
u
l
a
t
i
o
n
C
A
D
/
C
A
M
% of
Students
Over 45% of the students surveyed thought these skills had not been developed during the year
© The Keynote Project 2002
7
External Audit
8.3 Communication
For the purposes of this audit this was subdivided into:
•
Written – essays, technical reports, project reports
•
Verbal – one to one, within a group, in front of a group
•
Listening skills – note taking
•
Visual – drawing, layout
•
Presentation skills
Students’ main area of concern at Year 1 was speaking in front of a group, with 33% of the Materials
Technology and 36% of the Art & Design students rating their competence level in this skill as less
than satisfactory.
This underlines the findings from the internal audit, undertaken by the Keynote
Project partners, where 32% of Year 1 students had had little or no experience of speaking in front of
a group.
Over 80% of students in both the internal and external audits rated this skill as very
important to career prospects.
8.4 Numeracy
•
General mathematical skills
•
Collecting and recording data
•
Analysis of data
•
Presentation of results
Materials Technology Students – How they perceive their numeracy skills
A minimum entry-level standard of numeracy was required by many of the Materials Technology
courses in the survey. Consequently, it was expected that a high percentage of the students would
rate their perceived competence level in this area as satisfactory and above.
The resulting data
confirmed this assumption.
Perceived level of
competence at the
end of the course
General
Mathematical skills
Collecting & recording
data
Analysis of data
Presentation of
results
Satisfactory and
above
79%
88%
93%
95%
Less than
competent
9%
2%
2%
0%
Didn’t answer the
question
12%
10%
5%
5%
Art & Design Students – How they perceive their numeracy skills
The debate about numeracy skills in Art & Design continues.
Numeracy was very rarely mentioned as
a learning outcome.
More than half the students surveyed preferred not to answer the questions
rather than rate their skill competency level in this area.
97% of the students surveyed did not think
that general mathematical skills had been assessed during the year and 74% did not think they had
been developed.
“Standard of numeracy is low.”
(member of staff)
“Numeracy is important but tends to be ignored.”
(member of staff)
“Courses are lacking in opportunity to practice this skill.”
(several students)
© The Keynote Project 2002
8
External Audit
Year 3 Art & Design students numeracy skills competence level ratings
Perceived level of
competence at the
end of the course
General
Mathematical skills
Collecting & recording
data
Analysis of data
Presentation of
results
Satisfactory and
above
44%
44%
22%
22%
Less than
competent
0%
0%
11%
11%
Didn’t answer the
question
56%
56%
67%
67%
8.5 Information Technology
•
Text manipulation
•
Graphics manipulation
•
CAD/CAM
•
Use of email
•
Use of the internet
Although more than 90% of students perceived that use of email and the internet had not been
assessed, over 65% of students thought that they had developed these skills during the year. This
contradicts the previous statement that assessment is one of the main drivers for students with
regard to skills development.
In the majority of the institutions surveyed, much of the plethora of
information, which needs to be circulated to students, is now made available through internet
technology, proving that necessity is also a great motivator for skill development.
8.6 Learning to Learn
•
Sourcing information
•
Assessing relevance
•
Summarising information
•
Time management
•
Working to deadlines
First year students were very happy with their competence levels in this area and this was
mirrored by the results of the internal audit - where, in one group 100% of the students considered
they were particularly competent at time management and working to deadlines.
8.7 Working with others
•
Teamworking with peers
•
Teamworking with staff
•
Leadership skills
•
Motivational skills
•
Adaptability
•
Collaboration and negotiation
In general, teamworking modules did not play a significant part in the first year learning
experience. Their introduction in Year 2 gave rise to problems like: how to deal with the
student who does not pull their weight, how to motivate the student who is not really
interested and whether anyone would seize the initiative by taking on the leadership role, or
would a natural leader emerge?
© The Keynote Project 2002
9
External Audit
8.8 Key skills career relevance
Materials Technology Students – Key skills and perceived career relevance
Surprisingly, none of the Information Technology sub-categories appear in the Top 12.
As expected,
Materials Technology students rate technical reports and project reports very highly.
Less than 40%
of students perceived that essay writing, drawing, layout skills or use of CAD/CAM are important to
career development.
Materials Technology Students
Key skills and perceived career relevance
95
95
93
93
93
90
90
88
86
86
86
86
80
82
84
86
88
90
92
94
96
V
e
r
b
a
l
-
w
i
t
h
i
n
a
g
r
o
u
p
V
e
r
b
a
l
i
n
f
r
o
n
t
o
f
a
g
r
o
u
p
T
e
c
h
n
i
c
a
l
R
e
p
o
r
t
s
P
r
o
j
e
c
t
R
e
p
o
r
t
s
T
e
a
m
w
o
r
k
i
n
g
w
i
t
h
p
e
e
r
s
W
o
r
k
i
n
g
t
o
d
e
a
d
l
i
n
e
s
A
d
a
p
t
a
b
i
l
i
t
y
P
r
e
s
e
n
t
a
t
i
o
n
S
k
i
l
l
s
A
n
a
l
y
s
i
s
o
f
r
e
s
u
l
t
s
T
i
m
e
M
a
n
a
g
e
m
e
n
t
L
e
a
d
e
r
s
h
i
p
C
o
l
l
a
b
o
r
a
t
i
o
n
&
n
e
g
o
t
i
a
t
i
o
n
% of
Students
Over 85% of the students surveyed rate these key skills as vital to their future career
prospects
© The Keynote Project 2002
10
External Audit
Art & Design Students – Key skills and perceived career relevance
None of the sub-categories from Numeracy or Information Technology appear in the Top 13.
All forms of verbal communication and learning to learn are included.
Only 50% of students
felt that essay writing, technical report writing, general numeracy skills and analysis of data
had relevance to career prospects.
Art & Design Students
Key skills and perceived career relevance
90
90
90
87
87
87
84
84
81
81
81
81
81
76
78
80
82
84
86
88
90
92
T
i
m
e
M
a
n
a
g
e
m
e
n
t
V
e
r
b
a
l
-
w
i
t
h
i
n
a
g
r
o
u
p
P
r
e
s
e
n
t
a
t
i
o
n
S
k
i
l
l
s
S
o
u
r
c
i
n
g
m
a
t
e
r
i
a
l
T
e
a
m
w
o
r
k
i
n
g
w
i
t
h
p
e
e
r
s
A
d
a
p
t
a
b
i
l
i
t
y
O
n
e
t
o
o
n
e
c
o
m
m
u
n
i
c
a
t
i
o
n
W
o
r
k
i
n
g
t
o
d
e
a
d
l
i
n
e
s
V
e
r
b
a
l
-
i
n
f
r
o
n
t
o
f
a
g
r
o
u
p
L
a
y
o
u
t
A
s
s
e
s
s
i
n
g
r
e
l
e
v
a
n
c
e
o
f
d
a
t
a
S
u
m
m
a
r
i
s
i
n
g
i
n
f
o
r
m
a
t
i
o
n
C
o
l
l
a
b
o
r
a
t
i
o
n
&
n
e
g
o
t
i
a
t
i
o
n
% of
Students
Over 80% of the students surveyed rate these key skills as vital to their future career prospects
9.0 Conclusion and recommendations
•
As a matter of urgency institutions need to focus on staff development for key skills.
•
Clearer signposting by staff is needed to ensure that students make the connection between
specific module learning outcomes and key skills development as students had difficulty in
identifying and articulating where and how key skills had been developed.
•
Students are of the opinion that presentation skills are very important to their future career
and they would benefit from further development within this skill area.
•
Academic provision should be analysed with particular reference to key skill staffing models in
order to identify the most effective way to meet students’ needs.
•
Students need to be made more aware that essay writing is a useful method of developing
transferable writing skills.
•
When integrated delivery and assessment systems are employed to develop key skills, they
must be subject to clear tracking and appraisal in order to evaluate their effectiveness.
© The Keynote Project 2002
11
External Audit
i
Communication, Numeracy, IT and Learning to Learn plus Working with others, which was added by the consortium partners,
as it is considered by employers as an essential attribute for graduate employability
ii
One Scottish HEI was included in the final sample
iii
Curriculum 2000 is the name given to the new school qualification framework
iv
The ‘Key Skills Award’ includes communication, application of number, IT, working with others, improving own learning and
problem solving
v
5
Dearing skill list – Communication, Numeracy, IT and Learning to Learn
© 2010-2024 YouScribe