An MCP gateway for the whole company
Over the last year, MCP servers stopped being a side project at work and became something everyone wanted to build. One team wrapped the deploy system. Another wrapped the data catalog. Someone wired up incident tooling so an agent could read a runbook and open a ticket. Individually, each server was useful. Collectively, they were a mess.
Every server was its own island. Configs were hand-distributed in Slack threads. Tokens were copy-pasted into local files. There was no shared audit trail, no consistent auth, and no one who could answer the only question security actually cares about: who can call what, and what happened when they did?
So I built a gateway. Every internal MCP server now registers behind a single control plane. Engineers publish a server once, and from the user's side the approved tools just appear inside Claude Desktop and other AI tools — no per-machine JSON, no shared secrets, no tribal knowledge.
The shape of the problem
The naive setup is point-to-point. Each AI tool connects directly to each MCP server, and every connection carries its own credentials and its own config. With a handful of servers that is fine. With hundreds, it collapses: N clients times M servers is a lot of places to leak a token or skip a review.
Every client wired directly to every server. Tokens and configs copied everywhere.
One front door. Clients trust the gateway; servers trust one caller.
The gateway turns a mesh into a hub. Clients trust one endpoint. Servers trust one caller. Everything interesting — identity, policy, rate limits, logging — happens in the middle where it can be reasoned about once instead of re-implemented in every server by every team.
A proxy per server, not one big server
The core design decision: the gateway does not merge everything into a single monolithic toolset. When a team registers a server, the gateway stands up a lightweight proxy in front of it. The proxy speaks MCP to the client and MCP to the upstream, and in between it owns the boring, important parts.
- Central authentication. The proxy authenticates the human or agent once, at the edge. Upstream servers never see raw user credentials — they receive a short-lived, scoped token minted by the gateway.
- Access control. Registration declares who a server is for. A finance server can exist without being visible or callable by everyone; the proxy enforces the allow list before a single tool is listed.
- Rate limits. Per-user and per-tool budgets live on the proxy, so one runaway agent loop cannot take down an upstream that a whole team depends on.
- Isolation. Because each server gets its own proxy, a misbehaving or slow upstream is contained. One bad server does not degrade the others.
Keeping a proxy per server also means the unit of governance matches the unit of ownership. The team that owns the catalog server owns its proxy's policy. The gateway gives them the rails; it does not take their tools.
Registration is the product
The thing that made this adopted rather than mandated was making publishing boring. An engineer describes their server once, and the gateway handles distribution, discovery, and auth from there.
{
"name": "data-catalog",
"owner": "data-platform",
"upstream": "https://catalog.internal/mcp",
"visibility": "team:data-platform, team:analytics",
"tools": {
"search_tables": { "access": "read" },
"request_export": { "access": "write", "approval": "required" }
},
"limits": { "per_user_rpm": 60 }
}That manifest is the entire contract. The gateway reads it, provisions a proxy, applies the visibility rules, and the server becomes discoverable to exactly the right people. No one emails an endpoint around.
Governance, built in — not bolted on
Governance is the reason the gateway exists, so it is not a feature you opt into. It is the path of least resistance.
- Identity on every call. The gateway always knows which human or agent is invoking which tool on which server. Anonymous calls are not a thing.
- Policy at the edge. Allow and deny rules are evaluated per team and per tool before a request reaches the upstream.
- Reads and writes are separate. A tool that reads data and a tool that changes the world are treated differently, and destructive actions sit behind explicit approval.
- A real audit trail. Every invocation is logged with caller, server, tool, the shape of the arguments, and the outcome — enough to replay any decision after the fact.
The best internal AI platform is not the one that feels magical. It is the one where, six months later, you can answer exactly who called what, and prove it.
Auto-discovery into AI tools
The part people actually notice is discovery. Once a server passes registration and a user is in its allow list, the gateway surfaces it to their AI client automatically. In practice, an engineer opens Claude Desktop and the tools they are entitled to are simply there — already authenticated, already scoped.
Because clients only ever point at the gateway, this is a single configuration that never changes. New servers, rotated credentials, and revoked access all happen behind the front door. The user's config does not move; their available tools do.
// one config, set once, for any MCP-aware AI tool
{
"mcpServers": {
"work-gateway": {
"url": "https://mcp-gateway.internal",
"auth": "sso"
}
}
}What I'd tell someone building this
- Treat the gateway as a product, not a proxy you forget about. Its users are engineers, and adoption follows ergonomics.
- Make publishing a server self-serve and boring. If it needs a meeting, it will not scale to hundreds.
- Put identity, limits, and audit in the middle once. Never ask each team to re-implement them.
- Log enough that you can replay any tool call. Future-you, and security, will need it.
The model was never the hard part. The hard part was giving a lot of people a safe way to publish their own tools without each one turning into a security review. A governed gateway is how that becomes routine.