Employer work. The architecture here is described at the level of the pattern — no internal service names, schema, or infrastructure detail. What's public is what I can explain without publishing someone else's system.
Grounded on outcomes, not on a catalogue
A recommendation has to answer a question a learner can't ask directly: *what is actually within reach for someone like me?* Taste can't answer that. Comparison can.
A reference corpus of admitted-student profiles sits behind an embedding index, extended every admissions cycle. It is the thing the whole system measures against.
The learner's profile is embedded into the same space, and the nearest comparable outcomes describe the level they can credibly aim at — not what they'd enjoy, what people in their position went on to do.
Scoring is calibrated against that corpus too, so a score means *relative to people who got in* rather than an arbitrary band somebody chose.
Retrieval grounds generation. The model decides *what* to suggest; the corpus decides *how ambitious* it's allowed to be.
- Student profile: Interests, past experience, current standing and the outcome they're aiming at. Everything downstream is a function of this one record.
- Embedding: Puts the profile in the same vector space as the reference corpus, so "similar" means similar in outcome terms rather than similar in wording.
- Vector index: Profiles of students who were actually admitted, grown each cycle. This is the asset — it's what separates a grounded recommendation from a plausible one.
- Comparable outcomes: The closest matches, never shown to the learner. They exist to establish the level a suggestion should be pitched at, and to calibrate the score against real outcomes. This is the retrieval worth keeping. The activity-catalogue lookup it replaced could only ever return something somebody had already written down.
- Generator: Proposes activities per category, with the retrieved band as grounding and a schema the gates read specific fields off. A rejected schema fails loudly rather than quietly persisting something unusable.
- Level gate: Checks the ideas sit above the level this learner has already reached, and inside the band the corpus said was reachable.
- Personal gate: Checks each idea is anchored in the learner's own stated interests rather than their academic subject alone.
- Claim gate: Rejects fabricated statistics — the "only 3% of…" openers the model liked to invent.
- Cap + dedupe: Forces the answer down to a deduplicated, capped list on every return path — including the one taken when the gates have given up.
What each gate refuses
Level. Ideas must sit above the standing the learner has already reached, and inside the band the corpus said was reachable. The comparison is ported from the retired *catalogue* lookup — same arithmetic, now run against generated ideas instead of retrieved rows.
Personalisation. Each idea must be anchored in the learner's own interests, not their academic subject alone. The prompt always asked; nothing verified it. Stating the requirement as a *field* and then checking the field is the move.
Claims. Fabricated statistics rejected by regex. A second auditing model hallucinates too and catches half as much, and n-sampling costs k× tokens on every request. The fabrication has a shape we specified, so it's a closed set — a regex costs nothing and never flakes.
Narrow on purpose. "8–16 weeks" and "2 interests" are scope, not evidence, and have to pass.
The bug that shaped the exit path
Every template asks for exactly five ideas. Nothing enforced it — the schema left the array unbounded and no gate counted.
One call returned 66 ideas, nine titles repeated four times each, all of a type the template explicitly forbids.
The gates worked. They caught every violation and exhausted all three attempts — then the caller passed the last attempt through as-is, straight into the database, where a human saw 66 suggestions under one activity.
The fix is boring, and in the right place: cap and dedupe on every return path, including the exhausted-retries fallback. Not in the prompt, which already asked and was ignored. Not in the schema, which can bound an array's length but cannot express *no two items share a title*.
The lesson: a validation layer that can be bypassed by its own failure path is not a validation layer.
Cost is a design parameter
Reasoning bills as output, which makes it a cost lever rather than a quality dial — measured at zero thinking tokens on the lowest setting against thousands on the highest, for longer output and only marginally better ideas. It's set per-environment, so giving a category more headroom is config rather than a deploy.