Employer work. The architecture here is described at the level of the pattern — no internal service names, schema, or infrastructure detail. What's public is what I can explain without publishing someone else's system.
What it already knows
A generic assistant starts every conversation from nothing. This one starts from everything the platform has already recorded about a learner — which is what makes the difference between advice and guidance.
- Student profile: Who the learner is, what they're enrolled on, and the score that put them there. Read from the platform rather than duplicated, so it can't drift.
- Meetings + tasks: Transcripts and action items from sessions with their consultant. The copilot inherits what a human has already agreed with them, so it never contradicts the person they actually spoke to.
- Student drive: The files the learner keeps on the platform — the working artefacts of the activity rather than a description of them.
- Context assembly: Builds the working context for a single turn, in a fixed order: cross-activity patterns first, then this activity's rolling summary, then its open gaps, then exactly which sub-task they're on right now.
- Memory: Per-activity memory holds a rolling summary and the open gaps for one piece of work. Global memory holds patterns that recur across everything. Intervention counts per sub-task are the stuck signal — the same task needing help three times means it was harder than it looked. Writes are best-effort. An extraction that fails must never take the session down with it, so the learner's work is never held hostage to bookkeeping.
- Copilot agent: Breaks the activity into sub-tasks and works one at a time, choosing per turn whether to nudge, explain, or walk through step by step — based on what memory says about where this learner stalls.
- Response judge: Scores each exchange instead of trusting a thumbs-up, which most people never click. Its verdicts steer the next turn and feed the golden set the CI gate grades against.
- Plan: The activity broken into weeks and then into sub-tasks, generated once and worked through one at a time. It is also what the stuck counts attach to, so a sub-task that keeps needing help is visible as a property of the plan.
- Evidence check: Works out what would actually demonstrate a sub-task is done, then calls whichever connector holds it. The sub-task decides the tool — a repository for something built, a design file for something drawn, a document for something written. Adding a connector doesn't change the agent. It's another source the check can reach for, not another branch in the logic.
- GitHub: One connector among several. "I've pushed the API" and three commits touching one route are different claims.
- Figma: Another. A finished wireframe set and three empty frames get described the same way by whoever made them.
- Docs: And another. Feedback lands on the draft as it stands rather than on what the learner remembers writing.
Memory, and knowing where someone is stuck
Two layers, kept apart on purpose. One flat store would have been less code, and a tutor that brings up a research project while you're building a website.
- Session: One working session on one sub-task: what was attempted, where it stalled, and how many times the copilot had to step in.
- Extractor: Summarises the session and pulls out the gaps it exposed. Fire-and-forget. If this fails the session still closes cleanly — bookkeeping is never allowed to take the learner's work down with it.
- Per-activity: A rolling summary and the open gaps for one piece of work, and nothing else. This is the layer that keeps a research project's context out of a portfolio site's.
- Stuck counts: A counter per sub-task, incremented live during the session rather than inferred afterwards. Three interventions means the task was harder than the plan assumed — a fact about the plan as much as about the learner.
- Global: What shows up across everything they do. "Struggles with written structure" belongs here; "hasn't set up the database yet" does not.
- Context assembly: Reads all three before every turn, always in the same order. Fixed order is what makes behaviour reproducible when something goes wrong — otherwise a bad answer has no reconstructible cause.
- Next turn: The assembled context decides which of the three the copilot does. The stuck count is what tips it from explaining to walking through step by step.
The model is not allowed to mark work complete
The agent has a tool that looks like it completes a task. It doesn't — it signals readiness, and the shared status writer refuses any completed write that doesn't carry an explicit confirmation.
The hard part: knowing whether it got worse
Generative output has no build error. Change a prompt, swap a model, add a skill — it still responds fluently and you have no idea whether it's now subtly wrong. So quality became a CI gate: a versioned golden dataset graded on every change, exiting non-zero if any case regresses.
Cheap tiers first. Zod validates structure for free and short-circuits the rest; embeddings are an on-topic signal only, never the verdict; the judge scores 1–5 on faithfulness, relevance, completeness and pedagogy. Pass is 0.70.
Goldens are expected-behaviour specs, not exact answers. Grading a generative system on string equality only teaches you that it produced different words.
The judge is calibrated against human grades, so its scores track what a person would have said rather than what a model finds agreeable. An uncalibrated judge is a confidence generator, not a measurement.
Every generative call writes a trace — model, prompt version, latency, tokens, thumbs — fire-and-forget, so telemetry never blocks generation. That's how the golden set grows from real traffic instead of from someone remembering to write test cases.