Teaching · Practical guide
Build an AI-Assisted Rubric with Observable Criteria
Turn vague marking language into evidence-based criteria. Test an illustrative rubric against two responses before using it for assessment.
Begin with the performance you need to assess
A rubric should make the expected performance visible. “Excellent understanding” is not enough to show what a learner must demonstrate or what a marker should look for. Start with the task, outcome and evidence, then decide which dimensions deserve separate criteria. Do not allow an AI-generated table to choose the educational priorities for you.
For this exercise the outcome is to calculate a percentage and explain its meaning. The required evidence is a correct part-to-whole setup, an accurate calculation and an interpretation tied to the scenario. Writing elegance is not a criterion unless communication quality is explicitly part of the learning outcome. This keeps an unrelated feature from dominating the mark.
Separate criteria, levels and weighting
Carnegie Mellon describes rubrics through criteria, descriptors and performance levels. In your own design, distinguish the thing being assessed from the degree to which it is demonstrated. A criterion might be “interpretation”; its levels describe missing, partial or complete interpretation. Weighting states how much that criterion contributes.
Ask AI to draft descriptors using observable evidence. Avoid labels that merely repeat “good”, “better” and “excellent”. Before assigning marks, confirm your assessment’s approved total and any rules for partial credit. The illustrative six-mark structure below is a teaching example, not a marking policy for your course.
Worked example: a six-mark analytic rubric
Task: “A batch contains 200 records and 14 fail a check. Calculate the failure percentage and explain what it means.” Criterion 1, setup: 2 marks for 14/200 × 100; 1 for identifying part and whole but omitting the percentage conversion; 0 for an incorrect or absent relationship. Criterion 2, calculation: 2 for 7%; 1 for a minor arithmetic slip after a correct setup; 0 when no meaningful calculation is shown.
Criterion 3, interpretation: 2 for explaining that 7 in every 100 records fail under the stated batch data; 1 for saying “7% fail” without explaining the reference whole; 0 for an absent or contradictory interpretation. The maximum is 2 + 2 + 2 = 6. Decide in advance how to treat equivalent notation and follow-through errors so the same mistake is not inconsistently penalised.
A correct answer is “14/200 × 100 = 7%. This means seven failed records per hundred records in this batch.” It earns 6/6 under this illustrative rubric. The rubric assesses the displayed evidence; it does not infer that the learner understands every percentage problem.
Test the rubric against contrasting responses
Response A says only “7%”. It demonstrates the correct result but not the setup or interpretation, so it receives 2/6 under these stated descriptors. Response B says “14/200 × 100 = 7%, meaning 7 failed records in every 100.” It receives 6/6. This difference is defensible only if the task clearly requires the working and meaning before learners attempt it.
If you intend to assess only the numerical answer, this rubric is misaligned: the missing working should not then be treated as failure to meet an unstated requirement. Rewrite either the task or the rubric. Try a third invented response containing a correct method and arithmetic slip, then check whether the partial-credit rule distinguishes it from a fundamentally wrong denominator.
Use AI to challenge descriptors, not assign authority
Ask the tool to identify ambiguous boundaries, overlapping criteria and places where the same evidence is rewarded twice. Check its objections yourself. It may suggest adding many dimensions, but more criteria do not necessarily improve the assessment. Keep the rubric short enough that learners and markers can apply it consistently.
For consequential assessment, follow the course’s moderation and approval process. Compare how two markers apply the draft to the same sample and discuss disagreement before release. Do not claim consistency simply because AI generated a neat table. Keep real student work within approved data-handling arrangements; fictional responses are sufficient for this practice.
Practice: calibrate a small rubric in 20 minutes
Spend five minutes naming a task and three evidence criteria. Spend five minutes drafting three levels per criterion. Use five minutes to score two fictional answers independently, then use the final five minutes to repair a descriptor that produces uncertainty. Add the maximum marks and verify the total.
Example repair: replace “clear explanation, 2 marks” with “states the compared quantities and explains the percentage in the context of the task, 2 marks”. This gives the marker two concrete features to inspect. If a response states a rate but reverses which outcome occurred, explain why its calculation can be correct while its interpretation is wrong.
Self-review: can every criterion be traced to the outcome? Can a learner see exactly what separates neighbouring levels? Do the totals, partial-credit rules and sample scores agree? Retain the trial responses and reasons for each score as evidence of your design process.
Reusable prompt
Select and copy this template, or customise your own using the free builder.
Draft a rubric for [task] aligned only with [outcomes]. Use three observable criteria and clearly distinguish performance levels. Total marks: [approved total]. State partial-credit and equivalent-answer assumptions. Test it against two fictional responses and identify ambiguous boundaries or double-counted evidence. Do not invent institutional rules or grade real learners. Return the rubric, example scoring with reasons, and questions for human moderation.Open the prompt builder →
Pause · Think · Check
Test your understanding
Choose one answer per question, then check your reasoning. This is practice; scores are not saved.