How to set up a “team” of editors to improve your writing
By breaking her editor's feedback into individual AI "skills" (AI Check, Guardrails, First-time Reader, Mom, Hitchcock), Katie Parrott cut line edits dramatically and eliminated rejections entirely. Here's how to build your own editorial team and free up your human editors for the work that actually matters.
As a copywriter, there’s nothing scarier than when your editor leaves hundreds of comments on your draft and suspects ChatGPT of writing said draft. When Katie Parrott, staff writer at media and software company Every Inc., was caught sounding like AI, she created an AI workflow that would catch her first.
Parrott realized that telling AI to sound more human or feeding it one comprehensive style guide would never produce the perfect draft. Instead, she turned feedback she’d gotten from her editor into a team of AI reviewers, each trained to identify and correct one type of problem, such as sounding too much like AI or using jargon.
The result? After implementing these skills, Parrott gets fewer line edits and hasn’t had a piece rejected since. Most importantly: her human editors can now focus less on sentence-level cleanup and more on bigger-picture feedback.
The problem: One comprehensive style guide kept failing
Parrott thought that if she documented enough of her editors’ preferences and AI tells in her writing, AI would eventually produce a perfect first draft. Unfortunately, this was never the case.
One persistent example was the “not X, but Y” construction often found in AI-written content. Parrott says she couldn’t get it out of her copy no matter how many times she put it in her style guide.
Her “aha moment” came when she removed that rule from her style guide and asked AI to flag just this tendency in a separate review. This successful editing round showed Parrott that AI could be more useful as an editor than a writer.
The solution: An AI editing framework
Parrott converted her master style guide into a set of AI review skills, built through OpenAI’s Codex, each responsible for a very specific task.
“These guardrails give me a set of tools that I can go in with, like a mechanic, and fine-tune what needs to be fine-tuned,” she says.
So, what kind of tools does Parrott have in her toolbox?
Here are a few notable ones:
- AI check: Flags recognizable AI constructions, empty metaphors, and overly polished language.
- Guardrails: Looks for Parrott’s personal bad writing habits, which came from a list her editor compiled for her.
- First-time reader: Identifies missing context and questions a new reader would have.
- Mom: Flags technical terms that need to be explained on first use.
- Hitchcock: Assesses suspense, withheld information, and narrative momentum.
Parrott can run one reviewer as needed or chain several together in a final pass.
An example of what Parrott’s First-time Reader reviewer looks for in a draft.
How to turn editorial feedback into an AI guardrail
Want to build your own AI quality control system based on feedback you’ve received about your writing? Here’s how.
1. Look for an issue that keeps coming back
First, identify at least one problem that repeatedly appears in your writing or your team’s work.
Parrott’s original Guardrails skill was the result of a difficult meeting with her editor-in-chief, who presented Parrott with a list of recurring issues with her writing.
For example, Parrott says “every time we would write about a pull request, [my editor] would be like, ‘what’s a pull request?’”
Repeated questions about terms like “pull request” became the “Mom” review skill, which reads writing “from the perspective of someone who loves you and wants to be supportive—but doesn’t really get what you’re talking about. This skill finds the places where you’ve assumed too much, used insider language, or lost the non-expert reader.”
Parrott’s Mom review skill
Parrott’s rule of thumb is that if she encounters the same feedback from her editor three times, she turns it into a new AI review skill.
2. Give each reviewer one clearly defined job
Don’t ask AI to make something more human, on-brand, and concise. Instead, assign each reviewer only one goal.
While Parrott built her system in OpenAI’s Codex, you can create yours in a regular AI chat. Simply open your favorite AI tool and describe what you’d like it to do.
Here’s an example of a review skill Parrott created to flag when her writing needs more context:
“I’ve noticed that our drafts sometimes make claims without enough support. Create a reusable review skill that identifies unsupported claims, explains what evidence is missing, and flags where we may need data, examples, or attribution. Do not rewrite the draft or apply changes automatically.”
Copy, paste, and edit Parrott’s prompt to meet your needs.
The “AI Check” in action, note the edits are not made directly.
3. Show it examples of the problem
Give AI a description and examples of the problem you want to identify and address.
Specifically, feed it:
- Two or three examples of writing that demonstrates the problem
- An explanation of why each example is problematic
- An example of acceptable writing, when useful
- Clear instructions about whether the reviewer should diagnose, recommend, or rewrite
When Parrott wanted her AI check skill to catch meaningless, overly metaphorical language, she described the pattern and provided examples, such as “navigating the landscape” or “stand as a testament to.”
4. Run the reviewer, then decide what to change
After you’ve created multiple review skills, run the reviews on a draft and see what they identify. By telling your system not to apply changes automatically, you can stay in the driver’s seat.
When Parrott ran her AI check reviewer on a draft, it flagged the following problems:
- Sets of three without a conjunction: a common AI construction.
- “The lift was practical and emotional”: an example of overly metaphorical language.
- “Human dignity survives contact with comic timing”: writing that sounds like it’s saying something without actually communicating much.
An example of Parrott’s AI Check in action. The reviewer flags instances when her writing sounds AI-generated and makes suggestions (on the left), but doesn’t automatically apply changes.
Rather than training her AI reviewers to automatically fix errors, Parrott created a human-in-the-loop system that keeps her responsible for applying—or not applying—recommendations.
“Even if I’m having AI implement the edits, I’m choosing which edits to apply,” she says.
Parrott rejects feedback that would flatten her voice and gives specific directions when she wants AI to implement a change.
5. Feed new lessons back into the reviewer
Finally, train your AI reviewers to improve themselves. If a skill isn’t working like it should, tell your AI–in descriptive language, not code– what went wrong, give it examples, and prompt it to update its instructions for that skill.
“I trust that AI knows how to write better instructions for itself than I do,” Parrott says, which is why she doesn’t touch the code behind a skill.
Each of her skills has a “lessons” section that grows as she discovers new preferences. As new preferences emerge, the systems incorporate them so Parrott doesn’t keep overlooking the same mistake.
Remember when Parrott’s editor repeatedly asked her to explain jargon like “pull request?” Parrott used that feedback to create a reviewer that looks for unexplained technical terminology. The Mom reviewer prompts her to define those terms on first use.
The result: AI handles the predictable feedback so humans can handle the important feedback
Equipped with her toolbox of AI reviewers, Parrott now receives fewer sentence-level comments from her editor. She and her human editors are free to focus on bigger-picture feedback, like whether an article is genuinely useful or if it will resonate with the right audience.
The goal is not to obviate the need for a second set of human eyes ever. I’m trying to make my human editors’ lives easier” and redirect their attention toward higher-value work.
An unexpected result of the AI reviewer system? Parrott says she’s writing more now because her system handles more of the cleanup.
Related
More data from the AI Lab.
