james christie n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
James Christie PowerPoint Presentation
Download Presentation
James Christie

Loading in 2 Seconds...

play fullscreen
1 / 21

James Christie - PowerPoint PPT Presentation


  • 135 Views
  • Uploaded on

James Christie. Automated Marking for Essay Content ~ does it work?. Essay Definition 1 of 2.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'James Christie' - eshe


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
james christie

James Christie

Automated Marking for Essay Content

~

does it work?

essay definition 1 of 2
Essay Definition 1 of 2

… requires a response composed by the examinee, usually in the form of one or more sentence, of a nature that no single response or pattern of responses can be listed as correct, and the accuracy and quality of which can be judged subjectively only by one skilled or informed in the subject, …

essay definition 2 of 2
Essay Definition 2 of 2

… but even an expert cannot usually classify a response as categorically right or wrong. Rather, there are different degrees of quality or merit which can be recognized. …

attributed to

Stalnaker, 1951

possible criteria for automated essay marking
Possible criteria for automated essay marking
  • Ease of creating a scoring schema
  • Ability to score on various mark regimes
  • Ease of identification on non-scoring elements
  • Ease of modification
      • should scoring error(s) occur
  • Consistent and reproducible scoring
  • Acceptability of results to
      • human markers, essayists, …
  • Defensibility
  • Accuracy and precision
  • Coachability avoidance
  • Cost
model essay
Model Essay

The cat sat on the mat.

marking schema
Marking Schema

Mark Item

3 cat sat mat

[max:3]

content data structure
Content Data Structure

1 z a a 0 2 3 4

2 z f cat 1 0 0 0

3 z f sat 1 0 0 0

4 z f mat 1 0 0 0

essay set
Essay Set
  • ALPHA
    • The cat sat on the mat.
  • BRAVO
    • The cat sat on the floor.
  • CHARLIE
    • The dog lay on the floor.
  • MODEL
    • The cat sat on the mat.
process interface
Process interface

What LEVEL of Diagnostics to use [0 ... 3] : 0

What ESSAY SET to use .................... : catmat

Enter SCHEMA to use ...................... : catmat

ALPHA.EXT .

BRAVO.EXT .

CHARLIE.EXT .

MODEL.EXT .

Started on Thursday, February 06 2003 at 15:59:05

Finished on Thursday, February 06 2003 at 15:59:06

schema report
Schema Report

Essay set ............. catmat

Schema Report using ... catmat

Entities 1 - 4 :

Entity ID : 1 2 3 4

Entity Type : a f f f

Part ID's : z

Essay Name :

ALPHA.EXT: y y y y

BRAVO.EXT: _ y y _

CHARLIE.EXT: _ _ _ _

MODEL.EXT: y y y y

content report
Content Report

Essay set .............. catmat

Content Report using ... catmat

Essay Name : Words Sentences Usage[%] Coverage[%]Part: z[ 3] Mark[ 3] %[100]

ALPHA.EXT: 6 1 50.00 100.00 3 3 100.00

BRAVO.EXT: 6 1 33.33 50.00 0 0 0.00

CHARLIE.EXT: 6 1 0.00 0.00 0 0 0.00

MODEL.EXT: 6 1 50.00 100.00 3 3 100.00

Started on Thursday, February 06 2003 at 15:59:05

Finished on Thursday, February 06 2003 at 15:59:06

Marked 4 file(s): scanned 4 file(s)

marking performance
Marking Performance

Pearson / Spearman

Significance **= 0.01 *= 0.05

future work content
Future work [content]
  • maximise use of active and passive voices
  • cope with spelling (and grammar) errors
  • increased coverage of Bloom’s Taxonomy
  • include non-textual feature(s)
  • develop
    • better feedback to the essayist
    • better feedback to the examiner
    • plagiarism detection mechanism(s)
future work style
Future work [style]
  • obtain marked essays for style marking
    • plain ASCII essays
      • using a common set of metrics
    • word-processed essays
      • using a common set of metrics
      • augmented with word-processing based metrics
james r christie

James R Christie

Does it work? Yes, but …

http://www.jkp.christie.btinternet.co.uk

james.christie@btinternet.com