The BuildScoreModel command takes existing essays with assigned grades and returns a module that can be used by the Score command.  The Score command gives a rating for a new essay based on attributes of previously scored essays stored in built scoring model.  

The reliability of the Score commands depends greatly on the amount of training data used to build the model.  At least 150 scored essays are required to start getting minimum performance (on the order of a 50% - 70% correlation with human-graded essays).  Larger sets on the order of 300 - 800 training essays are preferable and have been shown to score QuadraticWeightedKappa values of better than .80 compared to human grading.

The reliability of the score also depends on many other factors including the length of the expected answer (longer is better), and the nature of the question.  

Questions can be wide-open with no set expected answer.  An open-ended question like, "tell a humorous story" has no one expected answer or pattern of keywords that must be common between answers.  In this case the style of writing becomes more important.  The results of computer scoring of such questions has proven to have a high correlation with human scoring.

Generally computer scoring is used in conjunction with human scoring. A set of essays will be scored by a human grader and the computer.  Any essays that receive scores that disagree by more than one or two points are then re-scored by an independent human grader.  Computer scoring has also been found useful for immediate feedback in homework and practice.

The text of the question can optionally be added to the model using the questionText = option.  It is recommended that this option be used.

This function is part of the EssayTools package, so it can be used in the short form BuildScoreModel(..) only after executing the command with(EssayTools). However, it can always be accessed through the long form of the command by using EssayTools[BuildScoreModel](..).

$\mathrm{Question}≔''What\; is\; Archimedes\; Principle?''\:$

$\mathrm{Answers}≔\left[''Archimedes\text{'}\; principle\; is\; a\; law\; of\; physics\; stating\; that\; the\; upward\; force\; (buoyancy)\; exerted\; on\; a\; body\; immersed\; in\; a\; fluid\; is\; equal\; to\; the\; weight\; of\; the\; amount\; of\; fluid\; the\; body\; displaces.''\,''Archimedes\text{'}\; principle\; says\; that\; the\; magnitude\; of\; the\; buoyancy\; force\; is\; also\; equivalent\; to\; the\; weight\; of\; the\; displaced\; fluid.''\,''Archimedes\text{'}\; principle\; states\; that\; a\; body\; immersed\; in\; a\; fluid\; is\; buoyed\; up\; by\; a\; force\; equal\; to\; the\; weight\; of\; the\; displaced\; fluid.''\,''Archemedes\; ran\; through\; town\; shouting\; Eureka\; because\; he\; noticed\; something\; about\; the\; water\; in\; his\; bath.''\,''Archimedes\; principle\; says\; water\; weighs\; more\; than\; a\; block\; of\; gold.''\,''Archimedes\text{'}\; principal\; is\; the\; law\; that\; states\; that\; a\; boyancy\; force\; is\; equal\; to\; a\; girlancy\; force.''\,''Archimedes\; principle\; says\; that\; a\; block\; of\; gold\; will\; displace\; a\; matching\; volume\; of\; water.''\,''Archimedes\; principle\; is\; a\; law\; that\; says\; a\; boyancy\; force\; is\; equal\; to\; the\; weight\; of\; the\; displaced\; water''\,''Archimedes\; principle\; was\; discovered\; when\; King\; Hiero\; suspected\; that\; his\; new\; crown\; was\; not\; pure\; gold.\; Archimedes\; figured\; out\; how\; to\; test\; for\; a\; fake.\; He\; compared\; the\; weight\; of\; the\; crown\; with\; the\; weight\; of\; displaced\; water.''\right]\:$

$\mathrm{Scores}≔\left[10\,10\,10\,3\,0\,0\,4\,7\,6\right]\:$

$\mathrm{with}\left(\mathrm{EssayTools}\right)\:$

$\mathrm{model}≔\mathrm{BuildScoreModel}\left(\mathrm{Answers}\,\mathrm{Scores}\,\mathrm{questionText}=\mathrm{Question}\right)\:$

$\mathrm{Score}\left(''I\; don\text{'}t\; know.''\,\mathrm{model}\right)$

$0$

$\mathrm{Score}\left(''Archimedes\text{'}\; principle\; is\; about\; buoyancy.''\,\mathrm{model}\right)$

$2$

$\mathrm{Score}\left(''Archimedes\text{'}\; principle\; is\; a\; law\; that\; equates\; the\; force\; of\; buoyancy\; and\; the\; amount\; of\; displaced\; fluid.''\,\mathrm{model}\right)$

$5$

$\mathrm{Score}\left(''Nothing\; happened.\; The\; fish\; just\; moved\; away\; slowly\; and\; the\; old\; man\; could\; not\; raise\; him\; an\; inch.\; His\; line\; was\; strong\; and\; made\; for\; heavy\; fish\; and\; he\; held\; it\; against\; his\; hack\; until\; it\; was\; so\; taut\; that\; beads\; of\; water\; were\; jumping\; from\; it.\; Then\; it\; began\; to\; make\; a\; slow\; hissing\; sound\; in\; the\; water\; and\; he\; still\; held\; it,\; bracing\; himself\; against\; the\; thwart\; and\; leaning\; back\; against\; the\; pull.\; The\; boat\; began\; to\; move\; slowly\; off\; toward\; the\; north-west.''\,\mathrm{model}\right)$

$2$

Butcher, Philip G. and Jordan, Sally E. (2010).  A comparison of human and computer marking of short free-text student responses.  Computers and Education, 55(2), pp. 489-499

Kakkonen, T., Myller, N., Sutinen, E., and Timonen, J. (2008).  Comparison of Dimension Reduction Methods for Automated Essay Grading.  Educational Technology & Society, 11(3), 275-288

Kaplan, Randy M., Wolff, Susanne E., Burstein, Jill C., Lu, Chi, Rock, Donald A., and Kaplan, Bruce A. (1998), Scoring Essays Automatically Using Surface Features, GRE Board Professional Report No. 94-21P

Miltsakaki, Eleni and Kukich, Karen, Automated Evaluation of Coherence in Student Essays

Shermis, Mark D. and Hamner, Ben, (2012) Contrasting State-of-the-Art Automated Scoring of Essays: Analysis

The EssayTools[BuildScoreModel] and EssayTools[Score] commands were introduced in Maple 17.

For more information on Maple 17 changes, see Updates in Maple 17.