Every AI video tool with an MCP server can claim "MCP-first." The phrase doesn't mean anything on its own — it's a checkbox you tick by shipping a server, the same way "AI-powered" stopped meaning anything around 2023. What actually decides whether an agent-driven pipeline is worth building on is a handful of unglamorous architecture choices underneath the phrase: does the agent see the same system a human does, who's actually running the model on each call, what happens when a token or a pointer is wrong, and what the pipeline does the moment the agent gets something wrong. This is those choices, in our own codebase, including the one we got wrong in production.
"MCP-first" means one service layer, not two
The easy way to bolt MCP onto an existing product is to write a thin adapter layer: a handful of tools that each call your REST API under the hood, translating between MCP's shape and your existing endpoints. That's a legitimate way to ship an MCP server fast, and it's also how you end up maintaining two versions of every feature that quietly drift apart.
Ours is mounted in-process instead. The MCP app lives at `/api/mcp` inside the same FastAPI process as the REST API — one deploy, one database connection pool, one set of service functions underneath both. A tool like `commit_script` and the REST handler a human's browser hits when they save a script edit call the exact same function in `services/`. There's no adapter layer to keep in sync, because there's nothing on the other side of an adapter — the MCP tool *is* the same code path, wearing a different transport.
The practical payoff shows up the first time you're debugging something an agent did wrong: you're reading the same service code you'd read for a UI bug, not reverse-engineering a translation layer that might itself be the thing that's wrong.
The rule that decides who runs the model
Every AI-bearing method in the service layer — draft a script, plan a scene, write publish copy — takes an `executor` argument: `"studio"` or `"caller"`. `"studio"` is the UI path: our configured model runs server-side and hands back a finished result, the way you'd expect a normal web app to work. `"caller"` is every MCP tool, and it does something different on purpose — it returns materials, not a result, and does no model call of its own:
{
"materials": {
"prompt": "Write a 45-second script for scene 3...",
"grounding": { "readme_excerpt": "...", "prior_scenes": [...] },
"inputs": { "tone": "direct", "target_seconds": 45 }
},
"expected": "A script object with narration lines and per-line duration hints",
"submit_via": "commit_script"
}
The calling agent's own model reads the materials, does the actual thinking, and submits the result through a normal write tool — `commit_script` in the example above. One method, one prompt-builder, and the `executor` argument is the only thing that decides whether our model or yours does the work. That matters more than it sounds like it should: it means the grounding a scene gets — the README excerpt, the prior scenes, the brand voice — is built once, in one function, and both paths read from it. There's no second prompt template living in a different file that someone forgets to update.
The one deliberate exception is media and infrastructure AI — text-to-speech, image generation, transcription, embeddings. Those always run server-side regardless of executor, because there's no "caller's model" equivalent for synthesizing audio — an LLM client can't generate a voice clip. We think of the split as hands versus brain: the brain (anything that composes or judges content) goes to whoever's paying for the reasoning; the hands (anything that turns a decision into media) are infrastructure either way. It's enforced by a dedicated test, not a comment someone can miss in review — a test that fails the build if any MCP-reachable code path ends up calling our own LLM.
Auth that fails closed, not open
A bearer token on an MCP request resolves through three checks, in order: an instance-wide ops token (an env var, unscoped, meant for us), a per-workspace token you generate from your own settings, or an OAuth-issued bearer. The part worth calling out is what happens when none of those match: the request gets an *empty* scope, not a fallback to the unscoped ops-token's permissions. A token that doesn't resolve fails every project and reel lookup closed. That's the opposite of the more common shortcut — "if we can't figure out who this is, just let the request through with default access" — which is exactly the kind of default that turns a typo'd token into an access-control bug.
The other deliberate choice: a request that resolves to a real identity but the wrong tenant returns 404, never 403. If your token is valid but doesn't own the project you asked about, you get "not found," not "forbidden" — because a 403 confirms the resource exists at that id for someone, and an id enumeration attack only needs that one bit of confirmation to start working.
A pointer that forgets everything, on purpose
One genuinely convenient piece of state: an agent can say "add a beat to the reel" without passing a reel id, because the backend tracks which reel and project are "active" for your session — set either by a tool call (`switch_project`) or by the editor itself, which posts to the backend whenever you open or leave a reel in the browser. If you have a reel open in the UI, your agent defaults to acting on that same reel with zero id-passing. It's a nice piece of ergonomics, and it's implemented as literally a module-level variable in one Python file — two names, no database row, no cache.
Which means it forgets everything on every restart. A deploy, a crash, a routine redeploy — the pointer resets to nothing, and an agent that was relying on it silently starts acting on "no active reel" instead of the one you had open five minutes ago. We didn't fix this by making the pointer durable. We fixed it by treating the forgetting as the correct behavior and designing around it: the instance-wide ops token specifically does *not* get the implicit-pointer convenience (it always requires an explicit id), the pointer is exposed as a readable resource so an agent can check it rather than assume it, and every tool's documentation tells the agent to pass an explicit id whenever correctness matters more than convenience — which, right after a deploy, it does. A durable pointer would have been more work to build and would have hidden a class of bug instead of making it visible and checkable.
The agent looks at its own frames
This is the piece of the architecture we like best, because it's the one that actually lets an agent catch its own mistakes instead of marching forward on a broken result. When an agent builds a scene, the preview tool doesn't return a text description of what it thinks got rendered — it renders three real frames (start, middle, end of the scene's window) through the same rendering path a full export uses, and hands them back as genuine image content — the same kind of image block you'd get if you pasted a screenshot into the chat. The agent looks at those frames with its own model, the same way you'd squint at a preview before hitting render. There's no server-side scorer grading the composition for it.
That sounds like a small design choice, but it changes the shape of the whole authoring loop. Write a component, submit it, get back three frames, judge them, and if the middle frame shows something cramped or illegible, the fix is a specific note back to the agent — "widen the gap between these two elements" — not a full regeneration, because everything that already looked right stays untouched. The alternative, common in a lot of "generate" pipelines, is a tool that returns `{"status": "ok"}` and nothing else — which teaches the calling agent nothing and gives it no way to notice a scene came out wrong until a human looks at the finished export.
One grounding source, or the two paths quietly drift
Here's the failure mode that this whole "one service layer" story is actually defending against, and it's the one we shipped to production. The animation catalog — the full list of visual components an agent can place in a scene, each with its own data shape — is generated from the same code that renders it, parsed live off the real component registry. That's the correct design: the catalog can never say a component exists that the renderer doesn't actually have, because it's reading the renderer's own definitions.
The UI-facing prompt for one generation path carried its own hand-typed copy of that list instead — written once, by hand, and then never updated as new components shipped. Weeks later it was 25 components behind the real catalog. The failure wasn't loud: every component the model correctly asked for that didn't exist on the stale list silently flattened into a plain text block. No error, no warning — a whole reel came back as a slideshow of text where it should have had charts, terminals, and diagrams, because the grounding one path used was quietly lying about what existed. A second, related bug hit the same day: a data contract documented that one component's numeric field had to be an actual number, and a prompt on a different path never consulted that contract, so the model reasonably wrote a string where a number belonged and the renderer printed it literally — a value that was supposed to read as a metric read as broken text instead.
Both bugs have the same root cause and the same fix. The good version of the grounding already existed — the live catalog parser, the documented data contracts — and one code path just wasn't using it. The fix wasn't "write better prompts"; it was deleting the hand-maintained copy and pointing both paths at the exact same generated source, so there is structurally nowhere left for them to disagree. We've since made this a standing rule for ourselves: a literal list of component names or fields, typed by hand into a prompt or a module, is a bug that hasn't happened yet. If the code can derive it, the code derives it, for every path that needs it.
Publish is deliberately the least agentic step
Everything above is in service of an agent that can act with real autonomy inside a project — write, render, iterate, judge its own output. Publishing to a real channel is the one place we pulled the other way. `get_channel_status` has to run before `publish_reel`, and it returns only which channels are actually connected — a channel you haven't authorized is never offered as a choice, only pointed at the page where you'd connect it, because that's an OAuth popup no agent can drive on your behalf anyway. `publish_reel` then checks connection status again itself, server-side, regardless of whether the agent already checked — an agent's belief about what's connected is never trusted as the actual permission.
The two platforms don't even offer the same shape of caution. YouTube gives you draft, unlisted, or public — the video can sit private until you decide. Instagram has no draft state at all: publishing through the API makes it live immediately, and the caption is locked into the upload the moment the container is created, with no edit-after endpoint. So the agent doesn't ask "draft or live" for Instagram — there's no draft to offer — it confirms you understand the caption is final before it goes out. Getting that distinction right mattered more than it sounds like it should, because an agent that recites a generic "draft or live?" question for every platform is technically following a script and functionally lying about what's about to happen.
Where it's honestly still rough
The restart-amnesia tradeoff above has a sharper edge than "the pointer resets." An agent doesn't read source code before deciding what to say next — it follows the instructions it was handed at connection time, and if those instructions describe a fixed sequence ("after the user picks a platform, ask about privacy") without also saying "check the current state first," the agent will recite the scripted next step even when the live state disagrees with it. We hit exactly that: an agent asked a YouTube privacy question right after a user picked Instagram, on a project where YouTube wasn't even connected, because the instructions described a sequence rather than requiring a state check first. The fix was rewriting the instructions to force the state check before the question, not changing what any tool actually does. The lesson generalizes past this one incident: an agent's behavior is only as reliable as the gap between what its instructions assume and what the system can actually guarantee at that moment, and closing that gap is instruction-writing work as much as it's code.
Try it
None of this is architecture for its own sake — the whole bet is that an agent doing more of the work only holds up if the system underneath it is honest about what it knows, fails closed instead of guessing, and lets the agent see its own mistakes before you do. If you already drive a pipeline from Claude Code, the fastest way to see the shape of this is to watch the tool calls themselves — what a scene's materials actually contain, what a rejected render tells the agent, what `get_channel_status` returns before anything gets asked about publishing. `/mcp` has the connection steps; if you're building the video side of a DevRel or dev-tools workflow specifically, `/for/devrel` is the closer read.
Related