Grading Power and Scoring in Kritik
Grading Power and Scoring in Kritik

Written by Support Team
Updated over a week ago

Grading power is the amount of influence a student's evaluation has on their peers' creation scores. A student's final creation score, unless manually edited by a professor, is the weighted average of the peer evaluations they received. The weight of each peer evaluation is dependant on the grading power of the evaluator.

This means that a student with a lower grading power will affect a peer's creation score less than a student with a relatively higher grading power. A student's grading power is loosely represented by the number of stars on their avatar border in Kritik.

How to gain or lose grading power?

Students gain or lose grading power based on how closely their evaluations match the final creation score of the peer they are evaluating. Scores that are closer to the final creation score result in gains in grading power, and vice versa.

The exact amount of grading power gained or lost is calculated based on two factors

• The score differential - the absolute value difference between the score an evaluator gave and the final creation score for the criterion.

• For example, if the final score is 4 stars and a student picked 3 stars, the score differential is 1 star.

• The level coefficient - the number of criterion levels divided by 10. This gives us a way to adjust for criterion that have very few or very many levels so that grading power can be adjusted appropriately

• For example, the level coefficient for a 5 level rubric is 0.5 (5/10)

Though grading scores appear during the grading stage of an activity, all grading power changes are applied when an activity is finalized. If there are multiple activities in the grading stage, grades are recalculated for all grading stage activities when an activity is finalized. Editing scores will also trigger a grade recalculation, but only during the grading stage.

Scoring Algorithm

How a student's grading score is calculated?

The grading score is a measure of how accurately a student scores their peers’ creations and is calculated with Kritik's proprietary grading algorithm:

• The weighted average of Grading and Written Scores make up your Evaluation Score for a given activity

• For more information on how each stage is scored, see our article on How Scoring Works

A student's score is based on a linear function, where the slope of the function depends on the number of levels in a rubric. This means that the more levels in a rubric, the less weight each star has towards a student's score. For example, in a 3-level rubric, each star would have more weight than they would in a 5-level rubric. Rubric criteria weights are incorporated when calculating the grading evaluation score and grading power for all activity types except for Calibration activities.

Below are images of the linear functions for 3-level, 4-level, and 5-level rubrics (the slope is different for rubrics with different levels).

The table below breaks down the score differential (the difference in the number of stars from the final score) and the score differential’s impact on the grading score. This could be used as a guide to better understand the score a student would receive depending on how far off their score is from the final score.

Ensuring Accuracy and Understanding

Students' grading power and grading scores play important roles in the learning experience in Kritik. The technology used to determine these values for students aims to ensure accurate and deserved grading power and grading scores. For more information, feel free to read more about how to run a Calibration Activity and how Kritik determines students’ overall Scoring at the end of an activity.