Faculty Guidelines for AI Use and Abuse
Writing has always been an assessment staple in liberal arts colleges and universities. The ability to find, understand, absorb, and manipulate information and ideas to re-present them in a new form – the essay form – is essential to training flexible and adaptive thinkers that can produce results independently.
Writing also reflects progression in a discipline as the same writing exercise grows in intensity by becoming longer and more in-depth: from the in-class essay to term paper, to capstone thesis, to Master’s thesis, to book-length efforts required for a PhD. Protecting the integrity of writing in the learning process is therefore of paramount importance.
Observations
Generative AI bots like ChatGPT are having a massive impact in many areas, from education, to work settings, to creative arts.
The availability, ease of use, and sophistication of these tools are going to increase in the future. The rate of increase is predicted to be exponential.
We are all familiar with spellcheck and grammar check in word processors, as well as add-ons like Grammarly. Generative AI represents a leap forward towards automation by an order of magnitude because of its ability to generate text of any length that can convincingly pass for a human product, at least at a cursory examination. At a cursory examination, AI produces text that appears informed and relevant.
Unfortunately, AI often produces claims that are incorrect or completely invented. Experts have labelled such inventions “hallucinations.” When AI hallucinates, it delivers summaries, quotes, sources, and even citations that are completely made up, and does so with assurance and conviction.
It is also noteworthy that when AI produces correct text or answers, these often lack specificity, depth, and nuance.
AI generates imitations of existing text, based on the training data it was fed. The product is a plausible sequence of words but is no indication of any actual understanding (intelligence) in the human sense.
At-home assignments are the most vulnerable to improper AI use, no matter how cleverly designed.
Practical suggestions
Consider all at-home assignments a weak measure of student progress in the class. Just because no at-home assignment/paper can be safely assumed to be AI-free does not mean we should abandon this pedagogical tool.
However, to protect the integrity of the learning process and the value of our grades, consider the following strategies:
Reduce the relative weight of such assignments in the final grade. For example, a term paper previously weighted at 20-30% could be reduced to 10-15% of the final grade. This will undoubtedly introduce an imbalance between grade weight and the required effort but the risk of rewarding excessively the use of AI shortcuts is too great to ignore. At the same time, we may not want to give up long-form assignments entirely because of how formative they are. Reducing their relative weight in the final grade may be an unpalatable but necessary compromise.
Use TurnItIn for paper submissions. This tool, integrated in our LMS (Canvas), can now detect AI-generated text. Unlike with traditional plagiarism, AI-generated text has no original that can be used as proof of misconduct. However, a positive report for AI use is sufficient grounds for challenging the authenticity of the student’s work. This challenge should take the form of a short conversation in which the student displays his or her command of the subject matter presented in the paper. See this article for details on how Turnitin detects AI.
Look for significant discrepancies with original sources in summaries or paraphrases. AI has been known to invent what it does not have access to.
Check cited sources against our library databases or against the publication’s table of contents, accessible on the journal’s home page. AI has been known to invent an article by a real or fake author, usually attributed to a real journal, to support fake claims.
At least 30% of the grade for the class – preferably 60% to 80% – should come from in-class work done in controlled conditions, i.e. in the instructor’s presence and without access to the internet. While paper tests are always an option, AUR’s LMS (Canvas) allows administering tests and essays on laptops using a Lockdown Browser. It can be used for:
Midterms/Finals in the form of quizzes requiring a mix of short and long answers.
Matching titles to authors of sources assigned as readings.
Identifying quotes from assigned readings and providing minimal context.
In-class essays.
Include short in-class essays (at least one) in every course. These can be administered via the lockdown browser. Having a writing sample done in controlled conditions will allow the instructor to compare the quality of the writing to at-home work. Excessive disparity is sufficient grounds for challenging the authenticity of the submitted assignment.