AI needs to think before giving feedback — How an evaluation & regeneration loop yields better results for students and teachers

Shepard argues that for AI-generated feedback to earn teacher trust, systems need structured reflection loops rather than one-shot generation. He highlights the Stanford SCALE GenAI Repository paper by Jie Cao and Ken Koedinger (AIED 2024) proposing a three-step G-E-RG process: Generate initial feedback, Evaluate it against a rubric using a different prompt/LLM, then Re-Generate with the evaluation as context. The paper found this produced more accurate feedback with more strengths identified and more concrete action items. Shepard connects this to coding (LLMLoop: 71%→80%→90%+ with iterative retries) and data science (DeepAnalyze's analyze-reflect-revise tokens), arguing that reflective loops are foundational to reliable AI agents. While newer models have built-in thinking modes, domain-specific evaluation frameworks still add value — especially in "squishy" domains like education where ground truth is harder to define than unit tests. Two commenters add: Adam Lupu proposes a "flipped feedback framework" where students evaluate drafts themselves, and Priya Mathew Badger notes parallels between code review loops and writing feedback loops.