A calibration activity helps align student grading with how you would grade, improving your students’ grading accuracy.
How Calibration Works
The instructor selects 3 creations, either from a previous Kritik activity OR uploaded from the instructor directly. We recommend using a well-done creation, average creation, and lower-scoring creation. However, try not to choose anything too obviously good or bad - students should be challenged when evaluating the creations.
The instructor grades those 3 creations using the rubric.
The students then also grade the 3 creations using the same rubric.
In a calibration activity, students ONLY submit grades for the creations. They do NOT submit any creations themselves, nor do they submit written evaluations or feedback. The purpose of a calibration activity is to ensure accurate grading power is assigned to each student. NOTE: Calibration activity is 0 weight only.
The students' grading powers are calibrated through Kritik’s algorithms on the backend.
The new grading power allocated to each student will apply to any future Kritik activities.
NOTE: There is only the evaluation stage when the calibration activity is scheduled. Students are only required to grade the three creations.
How to Create AI-generated Calibration Activities:
Select the "Create an Activity with AI" button in your course.
Select "Calibration"
Understand the Calibration Activity Suggestions
Calibration activity suggestions are similar to individual assignments with a few key differences:
Assignment Type: The task will always be a written assignment. No PowerPoints, images, or recordings are included.
Content: The activities are designed to cover introductory concepts and fundamental topics within your course.
Edit the Pre-Created Activity
When you select a Calibration activity suggestion, the activity opens in edit mode. This allows you to make any necessary changes to the instructions, grading criteria, or other details.
All details, including the 3 sample creations, are pre-generated by AI.
Evaluate and Grade
The 3 sample creations are labeled by the AI as either "Good," "Medium," or "Bad."
In the Evaluation Notes section, the AI will also provide some additional context for each creation. While this is a useful guide, we encourage instructors to overwrite the AI-generated text with their own reasoning behind each grade.
Schedule the Activity
Once you’ve reviewed the activity and added your grading, simply schedule the activity.
How to set up a Calibration activity by pulling from a previous activity:
See a video on how to set up a calibration activity here:
TIP: It's best to conduct a calibration activity at the start of the term, either as your first activity OR immediately after your first completed activity.
Select the Activities icon on the left navigation menu
Select Create Activity at the top right-hand corner of the Activities Page in Kritik
Select ‘Calibration’ under Activity Type
Select ‘Apply Calibration to Past Activity’
Select if you would like to select creations from your current course or a previous course (if applicable)
Select the past activity you would like to pull the creations from
Fill out the Activity Details by adding a line or two about what the calibration activity’s goals are and how to complete it.
Select 'Edit Rubric' and update the activity rubric accordingly
Calibration works best using a rubric that has 3-6 criteria and at least 3 levels for each criterion.
***NOTE: Rubrics for active activities CANNOT be edited if they have entered/surpassed the Evaluate stage.***
Select 3 creations
Either click ‘Select Creation’ to choose a creation from your previous activity or click ‘Create a Creation’ to make your own.
When using creations from a previous activity, you can edit these creations as you see fit. This will have no impact on the student's original submission.
Score the 3 creations and select ‘Save’
Select ‘Save Activity’
Note: Calibration activities only have an Evaluate stage. Once a calibration activity ends, it is automatically closed and can not be reopened.
NOTE: You can go back and edit your scores after you select save. Make sure you make any edits needed before the Calibration activity commences.
NOTE: Editing grades will not impact the students’ scores on any previous activity.
How to set up a Calibration activity with new creations:
Select the Activities icon on the left navigation menu.
Select Create Activity at the top right-hand corner of the Activities Page in Kritik
Select ‘Calibration’ under Activity Type
Select ‘Apply calibration to New Activity’
Fill out the Activity Details
Select 'Edit Rubric' and update the activity rubric accordingly
Calibration works best using a rubric that has 3-6 criteria and at least 3 levels for each criterion.
Create and evaluate 3 creations.
Use the textbox provided or attach a file.
Score the the creation and then select 'Done'.
Select ‘Save Activity’
Note: Calibration activities only have an Evaluate stage. Once a calibration activity ends, it is automatically closed and can not be reopened.
NOTE: You can go back and edit your scores after you select save. Make sure you make edits before the Calibration activity commences.
Adding Evaluation Notes
Instructors have the ability to add notes to each calibration creation. These notes are designed to provide students with insights into how the instructors graded the creations so students can become better graders.
The notes are made visible to students only after they have completed grading the calibration creations themselves. This timing ensures that students' grading is not influenced by the instructor's notes.
Interpreting Calibration Scores
Once a Calibration activity ends, students' grading powers are then calibrated through Kritik’s algorithm on the backend. The closer a student grades to how the instructor grades, the higher that student's grading power will be.
The new grading power allocated to each student will apply to any future Kritik activities.
Micro-Calibration
Manually editing students’ creation scores will adjust the grading power and evaluation scores of the students who evaluated that creation.
The peer evaluators of the adjusted-score creation will have the grades they assigned the creation compared with yours instead of with one another.
In order for micro-calibration to have the intended effect, it MUST take place before an activity has been marked as complete.
Editing Rubrics for Imported Calibration Activities
Instructors can now edit the rubrics of imported Calibration activities from different courses. See here the steps on how you can import activities in your Kritik course. To edit the rubric of a Calibration activity that has been imported, follow the below steps:
Once you have imported your copied Calibration, click into this activity from your course Activities page and select "Edit Activity".
Looking through your activity's settings, scroll down until you locate the Rubric section. Here, you can either select the dropdown above to change the rubric assigned to this activity (see here how you can create and manage existing rubric templates), or you can select "Edit Rubric" to make changes to the rubric template currently attached.
NOTE: Make sure you make any edits needed before the Calibration activity commences. Rubrics for active activities CANNOT be edited if they have entered/surpassed the Evaluate stage.