The rise of generative AI has brought a great shift to the education industry. Although it exists as a great resource for students and instructors to expand knowledge and learning, it can also be unethically used with the intention to plagiarize its free information.
As a solution, Kritik has built an in-app AI-checking tool for instructors to determine whether students' submissions contain or derive from AI-generated content.
How does it work?
Kritik's AI-checking tool is built with technology created by GPTZero, a model that detects if text has been written by AI. This model works at the sentence, paragraph, and document level, so it can determine if students' submissions contain any AI-generated material (even if they contain both text and attached files). If there are multiple attachments and content in the text box, Kritik's tool scans the entire creation and provides three distinct scores: one indicating the likelihood of it being written by a human, another for AI, and the third for a combination of both human and AI authors for the full submission.
GPTZero's AI-Detection tool scans for material made by any generative AI model (including but not limited to Chat GPT). The tool's accuracy is increased as more text or content is scanned, meaning that AI scores provided by the checker are more reliable at the document level than the paragraph level, with the sentence level being of least accuracy given content size.
You can learn more about the accuracy of GPTZero and their technology here.
Instructors should keep in mind that there may be cases in which actual human content is flagged as AI-generated, and vice versa. In addition, instances may exist where highly formal or procedural text is identified as AI-generated. Instructors should approach the tool's produced AI scores by understanding these possibilities when taking next steps in cases of suspicious or alarming scores.
How to use Kritik's built-in AI-Detection tool
To use Kritik's AI-Detection tool, follow the following steps for each submission you would like to scan:
Go to the activity and click on a student.
When viewing their Creation in the Creation tab, you will find the tool button labelled "AI-Generated Content Check". Click this function to trigger a scan of the student's overall submission.
You will then receive a score indicating the likelihood of the creation being written by a human, by AI, or a mix of both human and AI authors. (based on textbox content, file content, or both).
For Creations containing both written text and file submissions, our tool will scan the entire contents and provide holistic scores for each component, indicating the extent of AI-generated content.
How Detection Percentages Are Calculated
The percentages are derived from analyzing text characteristics against a database, evaluating linguistic patterns, and complexity to distinguish human from AI authorship. This system categorizes text as AI-generated, human-written, or mixed. The model uses a post-hoc calibration step for accuracy, providing a probability score for each category, and confidence scores guide interpretation. This nuanced approach accounts for modern writing processes that blend human and AI inputs, ensuring more accurate and interpretable results.
Bulk AI Content Check
The bulk AI content check tool allows instructors to check all submissions through our AI-Detection system at once. To generate the report, click into an activity any time after the create stage is completed and select “Export AI-Content Report” from the “Actions” menu.
You will then receive an email containing a CSV file with the bulk report of AI content probabilities for each submission.
