1 / 29

Standardisation Briefing

Standardisation Briefing. REMOTE MARKERS SUMMER 2017. Key Contacts. Regulatory requirements. Examining is regulated by condition H1.1 of the General Conditions of Recognition. This requires that the awarding organisation must

eschilling
Download Presentation

Standardisation Briefing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Standardisation Briefing REMOTE MARKERS SUMMER 2017

  2. Key Contacts

  3. Regulatory requirements Examining is regulated by condition H1.1 of the General Conditions of Recognition. This requires that the awarding organisation must • have in place effective arrangements to ensure that, as far as possible, the criteria against which Learners’ performance will be differentiated are – (a) understood by Assessors and accurately applied, and (b) applied consistently by Assessors, regardless of the identity of the Assessor, Learner or Centre.

  4. Purpose of Standardisation The standardisation process is designed ensure that all examiners have a well-founded and common understanding of the requirements of the mark scheme (appropriate to their responsibilities) and can apply them reliably. Completing this process enables all examiners to mark candidates' work consistently and accurately. It establishes a common standard of marking that should be used to maintain the quality of marking during the marking period.

  5. Responsibilities • The Principal Examiner is responsible for the professional judgements underpinning the process of standardisation. • Team Leader’s are responsible for supporting examiners through the standardisation process.

  6. Standardisation for Remote Markers(1) • Question Paper can be accessed from the RM Portal day after paper sat date. • SSU (standardisation set-up meeting) take places 7-10 days after paper sat date (this is attended by Team Leaders and the Principal Examiner). • Mark scheme is finalised at SSU and practice and standardisation scripts are selected for the learning and testing phases for remote markers. RM Portal http://support.rm.com/generic.asp?Cref=GP1286643&SrcURL=/ca

  7. Standardisation for Remote Markers(2) • After the SSU your Team Leader will make contact with you to inform you that the mark scheme will be available via RM Assessor(this should be within 24 hrs of the end of the SSU). • Once you have accessed the mark scheme you should login to RM Assessor and access the practice scripts to begin the learning phase, followed by the testing phase. • This process is likely to commence approximately 10-12 days after paper sat date (timescales can be shorter for later examination papers).

  8. Standardisation for Remote Markers(3) Exam is Sat Marker accesses the Question Paper Marker can now access mark scheme from RM Assessor After SSU - TL contacts team SSU Meeting (Approx 7-10 days Paper Sat Date ) Marker can now begin the process

  9. RM Assessor All marking will be carried out on RM Assessor. URL https://ca-marking.assessor.rm.com

  10. Standardisation Approval Process (1) • Download practice scripts, then mark and submit one at a time. Once you have submitted the first script, view the definitive mark by accessing the script in your closed items before continuing with the next practice script. • Email your Team Leader once you have finished the practice scripts. Your Team Leader will then contact you and provide feedback. This is your opportunity to ask any questions about how the definitive marks were awarded. • After this discussion has taken place, download the standardisation scripts. This is your test of marking. Mark these scripts independently and submit all at once. • Your Team Leader will then review your scripts and either: • - approve you to complete live marking or • - not approve you and provide feedback. You will then be asked to complete a second test (STM scripts) which you will need to download, complete and submit to your Team Leader.

  11. Standardisation Approval Process (2) • After second test (if applicable) TL either approves marker for live marking or informs the marker and the examiner and assessors team that the examiner will not be progressing to live marking. Documents can be located at http://www.ocr.org.uk/assessor-communications/

  12. Standardisation Approval Process (3) Familiarisation and Learning Practice Scripts Testing Standardisation Scripts Non Approval Examiner does not progress to Live Marking Non Approval Second Test STM Scripts Approval Live Marking TL informs Examiner and Assessor Management

  13. Suspension during live marking (1) • If a Team Leader has concerns regarding your marking (as a result of your marking on seeded scripts) they will check some live scripts. • If following the live marking check your Team Leader still has concerns they will suspend you on the system and contact you to inform you that you are suspended from any further marking until the PE has reviewed the marking. • The PE will then review the judgements made by your Team Leader (as soon as possible) and then inform OCR and your Team Leader of the outcome. • This could result in the following: • Your Team Leader makes contact with you to inform you that you can continue marking or • An OCR Monitoring and Support Manager contacting you to inform you that you will not be able to mark any further scripts.

  14. Suspension during live marking (2) TL carries out check Completes live script checking form Check reveals concerns TL Suspends Marker Sends completed formto Monitoring and Support Manager and PE PE considers Evidence (24hrs) Examiner should continue marking Examiner must be stopped Confirms Decision to Monitoring and Support Manager/TL Monitoring and Support Manager contacts Examiner to inform them of decision TL informs marker and removes suspension

  15. Marker Monitoring • All examiners are monitored through seeding scripts and our marker monitoring alert system. • Where an examiner’s marking is not in line with the rest of the markers on the panel or their response to a seeding script is causing concern an alert is triggered. • Once the alert is triggered the Team Leader is asked to carry out a check on the live marking of the examiner the alert has been triggered on. • Following the check on live marking the team leader either informs the monitoring team the examiner is fine to continue orif the Team Leader has concerns then they will suspend the marker while the PE considers the evidence.

  16. TL is requested to carry out a check This request is made by the Monitoring and Support Team TL suspends marker on Scoris and advises them of suspension TL carries out check (48hrs) Completes Alert Response Form (ARF) Suspends Marker Sends completed ARF to Monitoring and Support Manager and PE Check reveals concerns PE considers Evidence (24hrs) Check reveals no concerns If a marker is to be stopped from marking an OCR Monitoring and Support Manager will make contact. Examiner should continue Marking Confirms Decision to Monitoring and Support Manager/TL Examiner must be stopped Sends completed form to Monitoring and Support Team Monitoring and Support Manager Contacts Examiner TL informs marker and removes suspension Confirms to PE/TL Examiner has been stopped

  17. Mark Return Dates and Grace Period Mark Return Dates • Marks return dates are available on the Assessor Communications website and will also be available to you within RM Assessor once you start marking Grace Period • The marking grace period starts at 48 hours. • It reduces to 12 hours after the 50% marks return date. • Reduces to 0 hours after the 100% marks return date. Assessor Comms Website (under the heading forms): http://www.ocr.org.uk/assessor-communications/

  18. Confidentiality of Marking • Marking must never be carried out in a public place. • There must be no discussion of work in progress with anyone other than members of the marking panel and OCR. Therefore, do not discuss marking with anyone else including other colleagues or your family. You should never discuss any aspect of marking or marking process with the media.

  19. Conflict of Interest (1) What to do if you download a response and you think you know the candidate Although it is very unlikely, there is a remote possibility that you may recognise that a response you have downloaded for marking has been written by a candidate who is known to you. This is a conflict of interest, so if you recognise a response that you have downloaded, you must reject this immediately. Where a response you have downloaded for marking has additional objects, you are advised to check these for any identifying details before you start marking it, so that you can reject it immediately if you recognise the candidate.

  20. Conflict of Interest (2) To reject a response: 1. From the Marking page for the response, click Reject Response. A confirmation request is displayed. 2. Click Yes. The response is rejected and the next available response is displayed.

  21. Conflict of Interest (3) Please note: this method of rejecting scripts should only be used where an examiner recognises a candidate response. For instances such as illegible scripts or where responses come from an unfamiliar option route, the appropriate exception process should be raised.

  22. Marking – Good Practice • It is important that you mark at a steady rate to ensure that you are not rushing to complete your marking to meet the deadlines set. • It is important to let you team leader know if you are planning to take any long breaks during the marking period e.g. not marking any scripts for a couple of days

  23. Annotation Purpose and Importance of Annotation • Provides an audit trail of why marks have been awarded. • Improves visibility of marking process for centres. • Allows TLs/PEs to look at AE scripts and understand where/how marks were awarded. This means useful feedback to AEs can be given. Mark scheme • Follow annotation principles as laid down in the mark scheme. • There must be evidence that every page has been seen by an assessor, including blank pages. Guidance will be provided in the mark scheme.

  24. Giving Marks (1) Implications of incorrectly entered marks • Incorrectly entered marks (e.g. through mis-keying) result in candidates being issued with the wrong grade. This brings the integrity of the examination into question. Ensure that you: • activate on screen mark confirmation so that when you enter a mark a large image of the number you have entered will appear briefly on screen. • are marking against the right image zone (for optional questions).

  25. Giving Marks (2) PLEASE READ CAREFULLY INFORMATION BELOW -THERE HAS BEEN A CHANGE TO THE GUIDANCE ISSUED IN PREVIOUS SESSIONS. Award No Response (NR) if: • There is nothing written in the answer space Award Zero ‘0’ if: • anything is written in the answer space and is not worthy of credit (this includes text and symbols). Team Leaders will confirm the correct use of the NR button with their markers before live marking commences and should check this when reviewing scripts.

  26. Rubric Errors PLEASE NOTE • Where candidates have a choice of questions and have provided more answers than required, then all responses are marked and the highest mark allowable within the rubric is given (by the system). • Enter a mark for each question answered into RM assessor, which will select the highest mark from those awarded. This slide is only applicable to papers were there is optionality

  27. Appraisal & Grading of Examiners At the end of the marking period your Team Leader will be asked to grade you on the following: • Quality of Marking • Administration/Communication This grading will determine whether or not you require any further training before you are invited to mark for future sessions.

  28. Payment for Marking • Payment will only be made for scripts that have been marked and submitted. • Payment for marking will be made automatically at regular intervals during the marking period for scripts that you have marked and submitted.

  29. Finally…………. Thank you for the marking that you will be undertaking for OCR for the June 2017 session.

More Related