We built an AI control plane with AI—and refused to trust it

We spent 24 hours using an AI coding agent to build part of an AI-agent management platform. The agent wrote a substantial amount of working software. It also produced several results that looked green and were not safe to ship.

That is the part worth publishing. The useful lesson was not that AI can write code quickly. Everyone has seen that demo. The lesson was that serious agent engineering begins when you stop treating the builder's own report as proof.

We used an agent to build an agent-management platform, but refused to trust any layer's own claim that it worked.

Independent checks caught credential leaks behind passing tests, database functions that compiled but failed against the real schema, a hierarchy renderer that silently hid agents, lifecycle code that had no deployable runtime, and an automation loop that could report progress from stale state. Those failures did not make the experiment unsuccessful. Catching them before production was the success.

What were we building?

The project is a control plane for operating multiple specialized AI agents. A tenant-level manager coordinates work across client workspaces. Each workspace can have its own agent hierarchy, tasks, approvals, runtime state, audit history, and bounded workers.

A managed Kubernetes cluster supplies isolated runtime capacity. A protected web application supplies the management surface. PostgreSQL and Supabase hold tenant-scoped state, row-level security policies, approvals, and immutable audit events.

The most important architecture decision was separation. The public web application holds no Kubernetes credentials. Read-only collectors observe runtimes. Lifecycle executors perform a very small set of approved actions. Human approval stays outside both.

• Eyes: a read-only collector reports health, readiness, restart counts, resource use, versions, and bounded redacted logs.
• Hands: a separate executor can perform only explicitly approved lifecycle verbs on labelled workloads.
• Judgment: a human approves consequential action; the executor cannot approve its own task.
• Memory: the control plane records the request, approval, claim, result, and audit evidence.

That separation costs more design work than giving one agent a powerful token. It also prevents one compromised component from becoming the entire system.

Why more compute was not the first answer

The project began with a hardware question: should we add local inference capacity or expand cloud infrastructure? Restating the real bottleneck changed the answer. The existing general-purpose agent had too much context and too many responsibilities. Several specialist agents needed to work at the same time without competing for one host or one giant conversation.

Kubernetes became the foundation because the first scaling problem was logical isolation: narrow contexts, bounded jobs, separate permissions, durable shared task state, and predictable resource limits. Local inference hardware can still become a model endpoint later. It is not a cure for an overloaded agent design.

The builder was productive—and not the verifier

We ran a dedicated builder agent in Kubernetes with its own workspace, resource limits, branch assignment, and explicit prohibitions. It did not receive a production database credential or a GitHub credential. Each slice had a narrow goal and four mechanical gates: tests, typecheck, lint, and production build.

Then a separate verification path re-ran those gates and attacked the exact claim the slice made. A security slice received hostile credential-shaped strings. A hierarchy slice received malformed graph shapes. A database slice executed its functions against the real production schema inside rolled-back transactions. A lifecycle slice was inspected for forbidden verbs and permissions, not merely tested on the happy path.

We also replayed important probes against the broken code. A probe that passes both before and after a repair proves nothing. The pre-fix replay is how we proved the test could actually detect the defect.

Green tests hid a real credential leak

One lifecycle and health slice passed its tests, typecheck, lint, build, and a basic credential scan. Independent verification still blocked it. Several protected API serializers returned stored JSON without structural redaction, so nested task payloads, approval requests, and log strings could preserve bearer tokens or API-key-shaped values.

The repair extracted one structural redactor and routed every serializer through it. The verifier then exercised nested fields, authorization headers, common key assignments, approval payloads, and hostile strings against the shipped functions. Eleven adversarial leak cases were blocked before the branch was allowed forward.

The lesson is uncomfortable and simple: tests written by the implementation agent are necessary evidence, but they are not independent evidence.

A migration can apply and still not work

TypeScript cannot prove that a PostgreSQL function will execute against a live schema. We found database functions with ambiguous column references, incorrect array-scope comparisons, and functions that existed in PostgreSQL while the API layer continued reporting them missing because its schema cache had not reloaded.

Every database slice therefore followed a stricter acceptance path:

1. Audit the SQL for destructive operations.
2. Apply only additive, idempotent changes.
3. Execute every shipped function against the production schema inside a transaction that rolls back.
4. Verify relevant row counts remain unchanged.
5. Reload the API schema cache after function changes.
6. Only then treat the migration as operable.

Resource created, migration applied, and feature operable are three different claims. Each needs its own proof.

The product model corrected the database model

During the build, the business model became clearer: the primary manager agent belongs at the tenant level, not inside every client workspace. Clients have their own agents, while the manager needs an intentional way to coordinate across workspaces.

That clarification exposed assumptions hidden in an otherwise clean schema. Client scope could not always be mandatory. Tenant-level identifiers needed uniqueness even when client scope was absent. Cross-workspace hierarchy relationships required explicit policy. Workspace labels needed the company name, not a portal contact's name.

This is a useful warning for custom software: a technically elegant schema can encode the wrong company. Product intent wins.

Least privilege had to be proven from both directions

The lifecycle executor was designed for a short list of actions such as restart, suspend, resume, and rollout. Independent review did not stop at confirming those verbs existed. It also proved that delete, secret reads, pod exec, namespace mutation, persistent-volume access, and unrelated workloads were unreachable through both application code and Kubernetes RBAC.

The first implementation still failed readiness. It assumed a command-line Kubernetes client existed in its runtime image, but the image did not contain one, and the branch did not ship a deployable executor manifest. The code could pass tests without being deployable.

That is the green-badge problem: every layer can display success while the complete path still does not exist.

Automation needs verification too

We created a progress loop to check builders, verify completed slices, open pull requests, and advance the queue. It exposed its own operational weaknesses almost immediately.

• Running inside an active chat session meant a new Slack message could abort its work.
• A detached job finished with no completion hook, creating dead time before anyone noticed.
• A durable system service lost environment values that existed in the interactive shell and started from the wrong Git base.
• Stacked pull requests merged into intermediate branches without placing all expected code on main.
• A long tick could report a pull request or deployment state that had changed while it was working.

The loop was moved into an isolated session, taught to re-read live state immediately before every report, and constrained to one builder at a time. The broader lesson is that autonomous loops accumulate verification debt and comprehension debt unless their own state transitions are observable.

What we would repeat on the next agent system

1. Use small, testable vertical slices instead of one giant autonomous build.
2. Keep the builder and verifier separate in role, prompts, and evidence.
3. Make every sensitive claim adversarial: redaction, authorization, hierarchy, database behavior, and lifecycle permissions.
4. Replay critical probes against known-broken code.
5. Treat database execution and deployment readiness as separate gates from application compilation.
6. Separate observation, mutation, and human approval identities.
7. Give every background job a completion signal and a resumable state record.
8. Re-read live external state immediately before reporting it.
9. Keep credentials out of the public application, builder workspace, logs, and Git history.
10. Assume a green badge is a claim to investigate, not a verdict.

Does independent verification slow AI development down?

It slows down the moment when code is called done. It speeds up the moment when the system becomes trustworthy. That trade is worthwhile anywhere an agent can expose data, change infrastructure, contact customers, spend money, or quietly make a bad operational decision.

Can an AI agent safely manage other agents?

Yes, if management authority is decomposed. The manager can route work without holding every runtime credential. A collector can observe without mutating. An executor can act without approving. A human can authorize without operating the machinery. Audit records connect the chain.

The bottom line

AI dramatically increased implementation speed during this build. The safe speed came from the surrounding system: narrow assignments, least privilege, adversarial probes, rolled-back production checks, independent verification, and human authority at consequential boundaries.

The future of serious agent engineering is not one brilliant model with every password. It is a team of constrained components whose claims can be checked, whose actions can be traced, and whose failures can be contained.

That is less magical than the demo. It is also how AI becomes dependable enough to run real work.