Monday, November 27, 2023

Assessing writing with AI

Writing with AI is a complex skill that overlaps with traditional manual writing, but it is not the same. Many instructors struggle to grasp this new skill because it is unfamiliar to them. Teaching something you haven't mastered is challenging, leading to noticeable unease at all educational levels. Even those eager to incorporate AI in teaching, often open to new innovations, face this difficulty. The issue essentially lies in redefining the objectives of writing instruction. If the belief is that students should ultimately write independently, then traditional practice is paramount, leaving no role for AI tools. However, the more challenging conceptual shift is recognizing the need to teach students how to write with AI. This is like the transition from penmanship to typing. We lose something in this shift: the beauty, the discipline, and the rigorous exercises of handwriting. I recall diligently practicing letter formations in my first-grade penmanship class. Although I was never adept at it and gladly transitioned to typewriters when they became accessible, I understand the pain of losing the esteemed art of writing, cherished for centuries. This pain, particularly acute for those who have spent decades mastering and teaching writing, must be acknowledged. Yet, this shift seems inevitable. We are dealing with a technology that is being adopted faster than any in history, and it is not a passing fad. The benefits are too clear. We face a stark paradox: educators use AI to create lesson plans and assessment rubrics, yet often bar their students from using the same technology. This is unsustainable and awkward. 

As a profession, we are only taking the first steps in integrating AI into writing instruction. Here's another baby step: I revised Sacramento State University's Undergraduate Writing Portfolio Assessment criteria, considering the new skill of "wrating." 

Writing Placement for Juniors Portfolio (WPJ)

5 - Exceptional Wraiter: Demonstrates mastery in "wraiting," producing AI-assisted compositions at a publishable level in their respective discipline. Showcases exceptional skill in generating rich, engaging prompts and collaboratively refining AI outputs. Exhibits a deep understanding of AI's strengths and limitations, skillfully navigating these in producing original, high-quality work.

4 - Strong Wraiter: Effectively employs AI tools in "wraiting," producing texts of high quality that reflect a sophisticated understanding of AI's capabilities. Demonstrates the ability to create rich prompts and engage in the iterative process of refining AI-generated content. Shows a clear grasp of AI's strengths and limitations, using them to enhance original thinking and critical evaluation.

3 - Competent Wraiter: Demonstrates a solid understanding of "wraiting," using AI tools to assist in writing tasks. Capable of creating effective prompts and engaging in the process of refining AI outputs. Shows awareness of the strengths and limitations of AI in writing, but may require further guidance to fully exploit these in creating high-quality texts.

2 - Developing Wraiter: Beginning to understand the role of AI in "wraiting." Can generate basic AI-assisted texts but requires further instruction in creating effective prompts and refining outputs. Shows potential in understanding AI's strengths and limitations, but needs more practice to integrate these effectively in writing tasks.

1 - Emerging Wraiter: Early stages of grasping "wraiting." Struggles with effectively using AI tools, often producing clichéd, uninspired texts that lack human input and originality. Needs substantial guidance in understanding AI's capabilities, constructing prompts, and refining AI-generated content.

0 - Incomplete Portfolio: Portfolio does not demonstrate the basic competencies in "wraiting" or effective use of AI in writing tasks. Requires additional work to understand and skillfully employ AI tools in the writing process. What do you think?

Thursday, November 16, 2023

The fundamental misunderstanding of AI-assisted writing

The debate rages on in various Facebook groups dedicated to AI in education, encompassing educators, publishers, and even lawyers. They grapple with the ethics, practicalities, and legality of using AI-generated text, often under the flawed assumption that there's a clear demarcation between human-generated and AI-generated content. This is a classic case of misunderstanding the nature of large language models (LLMs) – it is not just technically impossible to make such a distinction, but theoretically as well.

Imagine writing assistance by AI as a spectrum. On one end, there's the lazy prompt: "Write me an essay for my class based on these instructions." On the other, a minimal request: "Here's my text, just correct the grammar." In the former case, the content is mostly computer-generated. (Although some instructors give such detailed assignment descriptions for students that the paper is practically written by the instructor, but that is another issue). Yet, the most effective and transformative uses of AI lie somewhere in the middle. This is where the magic happens: turning a raw idea into a paper outline, transforming a rough argument into coherent text, asking ChatGPT for feedback on a draft, or enriching a paragraph with vivid examples.

This is not a simple case of either-or; it is a true collaboration between human intellect and machine assistance. By pigeonholing AI as a tool that merely replaces human effort, many reveal their unfamiliarity with what I like to call 'wraiting' – a blend of writing and AI. The current clamor for distinct labeling of human vs. AI-generated text, or setting limits on the extent of AI use, can come across as naïve or even embarrassing to those well-versed in AI-assisted writing.

The beauty of 'wraiting' lies in its collaborative essence. It redefines authorship, shifting the focus from the creation process to the act of releasing the final product. The most important wraiting skills is the ability to wring great content from the machine by giving it most of the ideas. Equally important is the final editing, the ability to discern between mediocre and great content.

Just as the user of a word processor or spell-checker is considered the author, the human guiding the AI in 'wraiting' holds the rights of authorship. The key lies in understanding and experiencing this process firsthand. So, before jumping into heated debates or formulating policies, it might be wise to take AI for a spin in your next writing project. Only then can one truly appreciate the nuances of this new era of authorship, where the lines between human and machine are not just blurred but non-existent. Regulating a thing you don’t know much about is always going to be risky. 

Prompt as a magic incantation

In engagements with AI, the crafting of prompts—a crucial interface between human intention and machine output—has acquired an almost mystic...