Decisions, decisions …
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

Decisions, decisions … Surveying the first-year experience Mantz Yorke Lancaster University [email protected] PowerPoint PPT Presentation


  • 91 Views
  • Uploaded on
  • Presentation posted in: General

Decisions, decisions … Surveying the first-year experience Mantz Yorke Lancaster University [email protected] EFYE Conference, Middlesbrough, 26 April 2006. Importance. The first year is critical for student success UK ‘loses’ by the beginning of the second year:

Download Presentation

Decisions, decisions … Surveying the first-year experience Mantz Yorke Lancaster University [email protected]

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Decisions, decisions …

Surveying the first-year experience

Mantz Yorke

Lancaster University

[email protected]

EFYE Conference, Middlesbrough, 26 April 2006


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Importance

The first year is critical for student success

UK ‘loses’ by the beginning of the second year:

~ 8% of young entrants to FT degree programmes

~ 15% of mature entrants to FT degree programmes

The proportion ‘lost’ varied with subject area

and institution. Significant influences are students’

background characteristics

Combined programmes, computing and engineering

have the highest incidences of non-continuation


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Causes of non-continuation

  • Are multi-faceted

  • Four broad areas:

  • Flawed decision-making re entry to HE

  • Experience of the programme and institution

  • Difficulty in coping with the academic demand

  • Adventitious events

  • Institutions can exert most influence on (2)


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Institutional research

  • Concept is well-established in the US

  • Termed ‘institutional self-study’ by Watson & Maddison

  • IR activities are often dispersed in UK HEIs

  • IR deals with reflective practice at the level of the HEI

  • IR’s main aims are

  • to assist an HEI’s understanding of its performance

  • to provide information for enhancement and accountability


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Institutional researchers

  • Analyse databases

  • Conduct investigations of various kinds

  • (US studies of retention tend to be heavily quantitative)

  • Report to institutional managers

  • Share expertise and findings with the IR community


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

  • Analysis of existing data: an example

  • Context

  • A university faculty suffering excessive student attrition

  • Need for staff to review practice

  • Data available

  • Ratings from post-module questionnaires

  • Failure rates in assessments

  • Entry qualifications (quite a lot of missing data)

  • Analyses

  • Post-module questionnaire data

  • Attrition data set against entry qualifications


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Based on

~3200

responses

Based on

~2150

responses


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Worse

D1

B

E1

E2

X

A*

C1

D3

D2

C*

A

C2

E3

B*

F

Better


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

The HE Academy FYE study

  • Purposes

  • To provide the sector with data that

  • is informative

  • can be used as a baseline for comparison with future

  • studies, particularly in a context of ‘top-up’ fees

  • can be used comparatively, within and across both

  • subject areas and institutions, to inform both policy

  • development and quality enhancement activity


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Challenges

  • Coverage of a range of

  • subjects

  • institutions

  • aspects of the first-year experience

  • with

  • an instrument of tolerable length

  • acceptable technical robustness

  • reasonable response rates


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Sampling (Institutions as in early 2005)

New universities Colleges Old universities

Allied to Med

Bio Sci

Psychology

Computer Sc

Eng & Tech

Social Studs

Bus & Admin

Humanities

Creative A&D


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Questionnaire - 1

  • When should the FYE survey be run? 3 options:

  • Early in the autumn

  • February/March

  • End of first year

  • Time needed to negotiate with institutions


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Questionnaire - 2

Wide range of aspects of the first-year experience

Statements with 5-point agree/disagree scale

• some negatively expressed items, where judged politic

• scrambled order of presentation

Informed by a range of previous instruments

Completion on paper in class time, not in own time / web

Relatively limited number of items (but >> NSS)

•no strong tradition re lengthy surveys (contrast Australia, US)

• limited time for survey

• reliance on patience and goodwill of institutions, students


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Administrative matters (not trivial!)

Need for a relatively relaxed timetable for survey

Preparedness to tolerate local variation in admin

Anticipating matters such as

• students taking joint/combined programmes

• students who are present solely because of options

• part-time students who are present

Piloting

Coding of OMR forms

Management of paperwork, generally


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Bias

Judged

Questionnaire items: contentlow

Questionnaire items: structurelow

Sampling of HEIslow

Admin of survey in HEIs [ 9 – 68% of nominal ]

Students absent ?

Actual student non-responselow

‘Yea-saying’low


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Responses (as at 7 April 2006)

Sent out to HEIsc20,000

Distributed in HEIsFar fewer, but not yet known

Returned 6,644

Blank 291

Jocular/offensive 5

‘Yea-sayers’ (?)4

Usable 6,344

Completion rate of returns 95%


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Caution!

  • Returns from one HEI have yet to be included

  • Data from ‘stragglers’ have yet to be received

  • The data that follow are highly provisional

  • They are presented in very gross terms

  • Sensitive sub-analyses are essential

  • PLEASE NOTE

  • Because the data that follow are provisional, they

  • are not for quotation


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Some demographics (valid %)

Age: 18-21: 75% Over 21: 25%

Gender: Male: 40% Female: 60%

Ethnicity: White: 80%

Family background: Managerial/prof: 40% Other known 40%

First in family to enter HE: 45%

Previous HE experience: 32%

Considered withdrawing: 30%

- proportion of whom wishing to switch: 38%


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Some interim findings

Family background: no difference re experience in HEI

1st in family in HE:ditto

Older students: more motivated

more positive relationship with staff

Gender: females more motivated, engaged

Knowing a lot about programme: better scores all round

Knowing a lot about HEI: ditto

More time on private study: positive across the board


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Considered withdrawal?


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

To scale, or not?

  • Feedback Corr coeff

  • Mean (a) (b)

  • (a) … has been prompt3.07

  • (b) … has helped me in my learning3.46 .56

  • Detailed comments received3.18 .43 .53

  • Scale (a,b,c) 3.24

  • Reliability of scale (Cronbach alpha) = 0.75


Decisions decisions surveying the first year experience mantz yorke lancaster university mantzyorkemantzyorke plus

Concluding comments

  • No study, whether qualitative or quantitative, is ever

  • perfect

  • Compromises are dictated by a range of considerations,

  • including

  • Type(s) of question being asked

  • Time and resources available

  • Acceptability of approach

  • Ethical issues

  • Institutional practicalities

  • Always a need to be able to explain the methodological

  • choices made, and the biases that these might incur


  • Login