1 / 26

Final note on marking

Final note on marking. Reliability marking report from WinDEM will include record of scores for double-marked items, AND marker IDs for first and second marking BUT This will only work if the same marker marks all (2 or 3) item in a booklet (i.e. the ID is linked to the booklets)

virgo
Download Presentation

Final note on marking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Final note on marking • Reliability marking report from WinDEM will include record of scores for double-marked items, AND marker IDs for first and second marking BUT • This will only work if the same marker marks all (2 or 3) item in a booklet (i.e. the ID is linked to the booklets) • Marking by booklet is no problem • Marking by item requires you to make sure that the markers score the same booklets (they will need to return to booklets to score most recently trained items) • We can help make sense of the output if you need it.

  2. Coding of parental occupations ICCS Marker Training Hamburg, July 2008

  3. Content of presentation • Open-ended questions in student questionnaire • Overview of coding requirements • Coding tools • Coding procedures • Rules for coding

  4. Questions on parental occupation • The ICCS student questionnaire includes two pairs of open-ended questions • Mother’s occupation • Father’s occupation • Open-ended responses need to be coded • For the analysis, the classified occupations will be recoded into status scores (International Socio-economic Index)

  5. Open-ended questions in StQ

  6. Response codes using ISCO classification • Framework recommended by International Labour Organisation (ILO) • Currently being updated to ISCO-08 • ICCS will have to use ISCO-88 • Some modifications of ISCO-88 classification!

  7. Coding tools • ISCO manual (ISCO-88) published by ILO in 1990 • http://www.ilo.org/public/english/bureau/stat/publ/index.htm • Also available in French or Spanish (http://www.ilo.org/public/english/bureau/stat/isco/isco88/major.htm ) • Countries may also use available national occupational classifications • BUT codes have to be converted into ISCO-88 afterwards

  8. Valid reasons for using national classifications • No translation of ISCO manual into country’s language available • Centre has coders who are familiar and experienced with national classification • Using national classification makes data more useful for national analysis

  9. ISCO-88 classification • Structure • Major groups (first digit) • Sub-major groups (second digit) • Minor groups (third digit) • Unit groups (fourth digit) • Mapping to occupational status • International Socio-economic Index (SEI) • Recoding of unit groups to SEI scores

  10. Modifications of ISCO-88 - 1 • General Managers’ classified as ‘[Small Enterprise] General Managers’ and ‘Department Managers’ as ‘[Large Enterprise] Department Managers’ • Several categories of armed forces personnel (including similar civilian occupations) • Distinction between ‘academic teachers’ (2321) and ‘vocational teachers’ (2322)

  11. Modifications of ISCO-88 - 2 • ‘Traditional Chiefs and Heads of Village’ (1130) was modified to cover all local (as opposed to national) officials. • Distinguished ‘Oilers and Greasers’ from other mechanics and fitters • Other codes for very broad categories:(1240) ‘Office managers’, (7510) ‘Non-farm manual foremen and supervisors’, (7520) ‘Skilled workers/artisans’, (7530) ‘Apprentices’, (8400) ‘Semi-skilled workers’

  12. Modifications of ISCO-88 - 3 • Additional codes: • 9501 for home duties • 9502 for student • 9503 for social beneficiary (e.g. unemployed, retired, etc.) • 9504 for “I don’t know” and similar responses • 9505 for vague responses

  13. General coding instructions • Only ISCO-88 codes should be submitted • Questionnaire responses to be entered into WinDEM • New for main survey! • replaces EXCEL template used in the field trial • Demonstration how to enter data in WinDEM will follow...

  14. Coding procedures • Enter data into WinDEM spreadsheet • Sorting function for occupations by job title • Generally, coding in sorted spreadsheets is more efficient and reliable than direct coding of hard-copies

  15. Rules for coding - 1 • Coders should not interpret according to own interpretations but follow instructions • Coders should only use information provided by the two open-ended responses

  16. Rules for coding - 2 • Begin by looking at job title (Q6a/Q8a) • if job title is sufficient for coding select appropriate ISCO code • if job title is not sufficient, use job description to find appropriate code • If more than one job title is provided, select first job for coding unless job description indicates otherwise

  17. Rules for coding - 3 • Every pair of responses should be coded to the most detailed level possible (4 digits) • If not possible, closest aggregated level should be selected (e.g. “teacher”) • Aggregated codes should be revised by supervisor of coding procedures • National centres should always apply the rules described in ISCO Manual (ILO, 1990) • “Approaches to some specific issues” • “Notes to some particular occupations”

  18. National coding instructions - 1 • National coding index should be supplemented with • National ICCS Coding Instructions • When using national classifications instead of ISCO-88 • add specific coding instructions for modified occupational categories • add any other necessary coding instructions for ICCS coding

  19. National coding instructions - 2 • Try to obtain national coding instructions from • relevant office for statistics in your country • (or) International Labour Organisation (ILO) • If no external version can be obtained, instructions will have to elaborated based on ICCS guidelines • Field trial data should then be used as a basis for coding index for the main study

  20. Coder training and queries • Training material • National ICCS coding instructions • Should be obtained from relevant statistical body in country and adapted for ICCS purposes • In case of coding problems • First try to resolve at national centre between coder and supervisor • If it cannot be resolved try to use other national expert advice • As a last resort, contact ISC (iccs@acer.edu.au)

  21. National expertise • Try to engage expert in national labour statistics • This person should not be directly involved in coding but give advice to supervisor in especially difficult cases • The ISC will try to help but national experts will have greater knowledge regarding country-specific coding problems

  22. Coders and coding - 1 • ISCO coders should be: • Literate and cooperative. • Clerically accurate and careful. • Willing and able to follow detailed instructions conscientiously. • Not likely to alter procedures in order to reduce the amount of work. • Persistent and willing to work steadily for long periods.

  23. Coders and coding - 2 • Screening test may used during recruitment to check some of the criteria recommended by the ILO • It is recommended to have two coders with one supervisor • Ensure support from specialist with experience in this area (if available) • Training should mainly consist of practical instruction under close supervision

  24. Double-coding • When training coders national centres may want to use double-coding of sub-set of responses • Can be used at the beginning to identify problems and • At the end for a final measure of agreement.

  25. Submission of occupational data • Ensure that all students have valid ISCO-88 codes

  26. Questions or comments?

More Related