The Evaluate Stage

How does News Manager decide which articles to draft? Learn how Evaluate scores relevance, the two ways you steer it, and why it runs only on rows you select.

Evaluate is the AI relevance check that reads each article you select and decides whether it's worth drafting for your brand. It's a worked example of an evaluative prompt — one that judges rather than writes.

What you'll learn

  • What Evaluate decides, what it records, and how scoring works.
  • The two ways you steer it: the evaluation prompt and excluded topics.
  • Why it runs only on the rows you pick, and why your brand's voice is kept out of it.

The concept: a reader that knows your brand

Scout fills your queue with everything your sources publish. Most of it won't fit your brand. Evaluate is the step that reads through and separates the relevant from the noise, so you only draft from articles that match what your brand is about.

Evaluate runs per brand and uses AI, so it needs your own AI key configured (bring your own key). For each article you hand it, the model returns a relevance score and a decision, and Evaluate does one of two things:

  • Approves it — the article advances to a processing state, ready for drafting.
  • Skips it — the article is set aside, with a short reason recorded on the item so you can read why it was rejected.

That recorded reason matters. When Evaluate skips something, open the item and see why — so you can tell whether the check is reading your brand the way you intended. While it works, Evaluate also tags each article with the topics it detected, which makes the queue easier to scan and filter afterward.

This is an evaluative prompt, and m18t treats it differently from a writing prompt: your brand's Voice, Rules, and Glossary are deliberately not injected into it. Only the brand's Positioning is — because a yes/no relevance decision should reflect what your brand is about, not how chatty or formal it sounds. See The Brand AI Profile for why that split exists.

How you tune it

Evaluate has two controls, and they work together.

1. The evaluation prompt

Each brand has its own evaluation prompt (a prompt config with the slug news-evaluate). This is where "relevant" is defined for the brand — who the audience is, what subjects to favor, what to treat as off-topic. The more specific the brand's Positioning and this prompt are, the closer the approve/skip decisions land to your own judgement.

If Evaluate keeps approving articles you'd reject (or skipping ones you'd keep), tune here first. Read the skip reasons on a few items to see how the model is interpreting your brand, then tighten the wording. The platform ships a tuned default prompt; you can customize it if a brand genuinely needs a different judgement structure.

2. The excluded-topics list

Alongside the prompt, each brand has an excluded-topics list (managed via the Categories panel). Any topic you exclude is hard-rejected during evaluation: a REJECT clause is added to the prompt so an article on an excluded topic is skipped regardless of what the rest of the prompt says.

Use exclusions for clean, absolute cut-offs ("never this topic, ever") and the prompt for the softer judgement calls ("favor this, downplay that").

Why it runs only on your selection

Evaluate uses your AI key, which means it costs you tokens. To keep that predictable, it runs only on the articles you select in the Feed Items tab — it never works through your whole queue on its own. Tick the rows you want checked, then click Evaluate in the selection bar. Scouted in a hundred articles but only want twenty evaluated? Evaluate twenty.

You can also run Evaluate on a single discovered item from its row actions. If an item errors during evaluation (for example, the model returns an unreadable response), it's flagged as errored and you can Retry it; a crashed run that strands items in the "evaluating" state can be cleared with Reset stuck.

FAQ

Why is the Evaluate button disabled? It needs your own AI key for the brand. Add an OpenAI, Google, or Anthropic key to the vault and it enables. See Why m18t uses BYOK.

Evaluate keeps approving things I'd reject. How do I fix it? Sharpen the brand's Positioning and the evaluation prompt so "relevant" is defined precisely, and add hard cut-offs to the excluded-topics list. Read the skip reasons to see how the model is reading your brand.

Why doesn't my brand's voice change how Evaluate scores? By design. Evaluate is a judging task, so Voice, Rules, and Glossary are kept out of it — only Positioning is used. Voice belongs in writing steps like Generate.

An item is stuck on "evaluating" — what now? Use Reset stuck in the toolbar to reclaim items stranded by a crashed run, then re-run.

Does Evaluate ever publish anything? No. It only approves or skips. Drafting happens later in Generate, and even that produces a draft you review.

What's next

Was this article helpful?