Decisions, decisions …
1 / 23

Decisions, decisions Surveying the first-year experience Mantz Yorke Lancaster University - PowerPoint PPT Presentation

  • Uploaded on

Decisions, decisions … Surveying the first-year experience Mantz Yorke Lancaster University [email protected] EFYE Conference, Middlesbrough, 26 April 2006. Importance. The first year is critical for student success UK ‘loses’ by the beginning of the second year:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Decisions, decisions Surveying the first-year experience Mantz Yorke Lancaster University' - lana

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Decisions, decisions …

Surveying the first-year experience

Mantz Yorke

Lancaster University

[email protected]

EFYE Conference, Middlesbrough, 26 April 2006


The first year is critical for student success

UK ‘loses’ by the beginning of the second year:

~ 8% of young entrants to FT degree programmes

~ 15% of mature entrants to FT degree programmes

The proportion ‘lost’ varied with subject area

and institution. Significant influences are students’

background characteristics

Combined programmes, computing and engineering

have the highest incidences of non-continuation

Causes of non-continuation

  • Are multi-faceted

  • Four broad areas:

  • Flawed decision-making re entry to HE

  • Experience of the programme and institution

  • Difficulty in coping with the academic demand

  • Adventitious events

  • Institutions can exert most influence on (2)

Institutional research

  • Concept is well-established in the US

  • Termed ‘institutional self-study’ by Watson & Maddison

  • IR activities are often dispersed in UK HEIs

  • IR deals with reflective practice at the level of the HEI

  • IR’s main aims are

  • to assist an HEI’s understanding of its performance

  • to provide information for enhancement and accountability

Institutional researchers

  • Analyse databases

  • Conduct investigations of various kinds

  • (US studies of retention tend to be heavily quantitative)

  • Report to institutional managers

  • Share expertise and findings with the IR community

  • Analysis of existing data: an example

  • Context

  • A university faculty suffering excessive student attrition

  • Need for staff to review practice

  • Data available

  • Ratings from post-module questionnaires

  • Failure rates in assessments

  • Entry qualifications (quite a lot of missing data)

  • Analyses

  • Post-module questionnaire data

  • Attrition data set against entry qualifications

Based on



Based on




















The HE Academy FYE study

  • Purposes

  • To provide the sector with data that

  • is informative

  • can be used as a baseline for comparison with future

  • studies, particularly in a context of ‘top-up’ fees

  • can be used comparatively, within and across both

  • subject areas and institutions, to inform both policy

  • development and quality enhancement activity


  • Coverage of a range of

  • subjects

  • institutions

  • aspects of the first-year experience

  • with

  • an instrument of tolerable length

  • acceptable technical robustness

  • reasonable response rates

Sampling (Institutions as in early 2005)

New universities Colleges Old universities

Allied to Med

Bio Sci


Computer Sc

Eng & Tech

Social Studs

Bus & Admin


Creative A&D

Questionnaire - 1

  • When should the FYE survey be run? 3 options:

  • Early in the autumn

  • February/March

  • End of first year

  • Time needed to negotiate with institutions

Questionnaire - 2

Wide range of aspects of the first-year experience

Statements with 5-point agree/disagree scale

• some negatively expressed items, where judged politic

• scrambled order of presentation

Informed by a range of previous instruments

Completion on paper in class time, not in own time / web

Relatively limited number of items (but >> NSS)

•no strong tradition re lengthy surveys (contrast Australia, US)

• limited time for survey

• reliance on patience and goodwill of institutions, students

Administrative matters (not trivial!)

Need for a relatively relaxed timetable for survey

Preparedness to tolerate local variation in admin

Anticipating matters such as

• students taking joint/combined programmes

• students who are present solely because of options

• part-time students who are present


Coding of OMR forms

Management of paperwork, generally



Questionnaire items: content low

Questionnaire items: structure low

Sampling of HEIs low

Admin of survey in HEIs [ 9 – 68% of nominal ]

Students absent ?

Actual student non-response low

‘Yea-saying’ low

Responses (as at 7 April 2006)

Sent out to HEIs c20,000

Distributed in HEIs Far fewer, but not yet known

Returned 6,644

Blank 291

Jocular/offensive 5

‘Yea-sayers’ (?) 4

Usable 6,344

Completion rate of returns 95%


  • Returns from one HEI have yet to be included

  • Data from ‘stragglers’ have yet to be received

  • The data that follow are highly provisional

  • They are presented in very gross terms

  • Sensitive sub-analyses are essential


  • Because the data that follow are provisional, they

  • are not for quotation

Some demographics (valid %)

Age: 18-21: 75% Over 21: 25%

Gender: Male: 40% Female: 60%

Ethnicity: White: 80%

Family background: Managerial/prof: 40% Other known 40%

First in family to enter HE: 45%

Previous HE experience: 32%

Considered withdrawing: 30%

- proportion of whom wishing to switch: 38%

Some interim findings

Family background: no difference re experience in HEI

1st in family in HE: ditto

Older students: more motivated

more positive relationship with staff

Gender: females more motivated, engaged

Knowing a lot about programme: better scores all round

Knowing a lot about HEI: ditto

More time on private study: positive across the board

To scale, or not?

  • Feedback Corr coeff

  • Mean (a) (b)

  • (a) … has been prompt 3.07

  • (b) … has helped me in my learning 3.46 .56

  • Detailed comments received 3.18 .43 .53

  • Scale (a,b,c) 3.24

  • Reliability of scale (Cronbach alpha) = 0.75

Concluding comments

  • No study, whether qualitative or quantitative, is ever

  • perfect

  • Compromises are dictated by a range of considerations,

  • including

  • Type(s) of question being asked

  • Time and resources available

  • Acceptability of approach

  • Ethical issues

  • Institutional practicalities

  • Always a need to be able to explain the methodological

  • choices made, and the biases that these might incur