1. Without a platform, every team starts over
The first agents often appear in different teams, each with its own tools. After a few months, the same problems show up everywhere:
- model access keys scattered across code and configuration;
- guardrailsGuardrailsAutomatic checks that block risky use: sensitive data leaks, forbidden content, out-of-scope actions.See the glossary reinvented by each team, with uneven security levels;
- costs invisible until the invoice, impossible to attribute;
- agents and tools nobody can find, so they get rebuilt elsewhere;
- no overall view for security, audit or compliance.
2. One platform, four blocks
An AI Platform pools whatever has no reason to differ from one team to the next. Teams keep their business, their data and their choices; the platform provides the restRESTThe most common style of web API: data is exchanged through addresses and simple verbs (read, create, update, delete).See the glossary .
- Operations
- Running agents and teams: runtimeIsolated runtimeThe environment an agent runs in, separated from others and limited to its scope, to contain errors and misuse.See the glossary environments, security and identity, onboarding a new team, cost tracking, observabilityObservabilitySeeing what happens inside agents: every run, its sources, tool calls, errors and costs.See the glossary and evaluationEvaluationTesting an agent on a set of known cases to measure the quality of its answers before updating it.See the glossary .
- Catalogue
- Everything reusable: agents, tools and MCP serversMCP serverThe small service that exposes a tool (an application, a document base) to agents through the MCP protocol.See the glossary , workflowsWorkflowA sequence of steps defined in advance. Some steps can be handed to an AI model, but the order stays fixed.See the glossary , promptsPromptThe written instructions given to the AI model to steer its answer.See the glossary and skillsSkillA reusable know-how for an agent: a folder of instructions, templates and sometimes scripts, loaded only when needed.See the glossary , applications. Each item has an owner, a version and an approval status.
- Model gateway
- A single access point to every model, proprietary, open sourceOpen sourceSoftware whose code is public and reusable under a licence. It can be audited, self-hosted and modified.See the glossary or customised: routing by need, quotas and cost per team.
- Governance and standards
- The shared rules: guardrails, usage policies, recommended frameworksFrameworkA development toolkit that provides an application’s structure; for agents: LangGraph, Strands, CrewAI…See the glossary and patterns, best practices and reusable assets.
3. Three audiences, one platform
One platform serves three audiences, each asking a different question. It has to answer all three at once.
| Consumes | Assembles | Governs | |
|---|---|---|---|
| Who | The business user | The use-case owner | The IT and platform team |
| Their question | “Does it save me time every day?” | “Can I build my use case without coding?” | “Do I control what leaves the information system?” |
| Their needs | Talk to an agent in natural language, get a reliable, sourced result, produce professional deliverables | Build an agent for their business, test, iterate, measure value, go to production when it works | Expose APIs as MCP servers and govern them, ensure security, compliance and audit, run the platform |
| Their tools | AI front endFront endThe interface where employees use AI: chat window, list of assistants, documents to attach.See the glossary , Teams, Slack | Agent studio, LLM gatewayLLM gatewayThe single checkpoint for every call to a model: model choice, quotas, cost, filtering and logging.See the glossary , API managementAPI managementThe platform that publishes, secures, documents and monitors a company’s APIs (MuleSoft, Kong, Azure API Management…).See the glossary | VS Code, coding agents, MCPMCP (Model Context Protocol)The open standard that lets an agent use tools and data: read tickets, search documents, query a CRM.See the glossary , command line |
One platform, three entry points. IT governance does not slow business teams down: it protects them.
4. Three ways to use the platform
Teams have neither the same needs nor the same skills. A good platform does not force one model on them: it offers several levels of autonomy.
| Level | The team… | The platform provides… | Typically |
|---|---|---|---|
| Use | uses agents and applications from the catalogue, and exposes its data as tools | everything else | Business teams without developers |
| Assemble | builds its own agents, tools and applications with the platform’s building blocks, then publishes them to the catalogue | runtime, gateway, security, observability, guardrails, memory | Product or data teams with a few developers |
| Build independently | builds its own technical stack, independent of the platform core | the standards and governance to follow, and the catalogue to feed | Mature engineering teams, very specific needs |
What stays the same at every level: everything goes through the catalogue and follows the shared governance. That is what lets one team reuse another team’s agent.
5. Three gateways, three governances
Every flow goes through a gateway, but not the same one. Model calls, tool calls and agent-to-agent calls have neither the same contracts, nor the same rules, nor the same maturity.
- Multi-model routing and fallback
- Normalising provider APIs
- Tracking tokens, cost and latency
- Catalogue and input/output contracts
- Exposure policies, versioning
- Routing to internal systems
- Discovering available agents
- Propagating identity and context
- Long sessions, state
Why not a single component? Different contracts, different governance units, different observability and different technology maturity. Merging them would force unacceptable trade-offs.
6. The federated model
Two traps lie in wait. Centralising everything in one AI team that builds it all creates a bottleneck. Decentralising everything brings back the initial chaos. The federated model splits the roles: application teams keep their applications, orchestrationOrchestrationCoordinating several agents and tools to run an end-to-end process: who does what, in which order, with which checks.See the glossary and data in their own environment; the platform team provides and runs the shared foundation.
Two worlds, one interface contractInterface contractThe stable commitment between the platform and applications: what is offered, how to call it, with which guarantees, without exposing internals.See the glossary : applications consume the platform’s capabilities through a stable contract, without knowing its internal layout. Several platform teams can even coexist, as long as they follow the shared contract set by the centre of excellence. Local autonomy speeds things up; central governance keeps them consistent.
| Centralised | Decentralised | Federated | |
|---|---|---|---|
| Who builds the agents | One central AI team | Each team, alone | The teams, on a shared foundation |
| Strength | Consistency, control | Speed, close to the business | Both at once |
| Risk | Bottleneck | Duplicates, cost, security gaps | Needs a solid platform team |
7. The centre of excellence
Next to the platform team, a small centre of excellence carries the rules and the know-how:
- set the standards: frameworks, patterns, approved control middlewareMiddlewareA component that slots into an agent’s loop (before or after the model, around a tool) to add a control without changing the agent itself.See the glossary ;
- approve models before they are made available;
- own the responsible AI framework: allowed uses, sensitive data, human approvalHuman approvalA person reviews and approves before the agent’s action is carried out (also called human in the loop).See the glossary ;
- produce reusable assets: agent templates, skills, prompts, examples;
- support team onboarding and measure adoption and value.
A good centre of excellence equips more than it polices: it makes the right path easier than the wrong one.
8. Where does MCP fit?
MCP and A2AA2A (Agent2Agent)The open protocol that lets agents talk to each other and share work. MCP links an agent to a tool; A2A links two agents.See the glossary are the standard plugs connecting agents to tools and to other agents. They make the platform’s life easier, but they are only one building block: the platform decides which plugs are allowed, for whom, and under which rules. Controls sit in two complementary places: in the gateways, for every agent at once, and in each agent’s middleware, closest to its decisions.
MCP: the protocol, its limits, and what it needs around itWhat MCP brings, its use cases, its limits, the four identity patterns, the A2A and AG-UI protocols, and the solutions on the market.9. Where to start
- Set up the model gateway: one access point, with cost tracking per team from day one.
- Open a minimal catalogue: a few approved agents, tools and prompts, each with an owner.
- Plug in central observability and evaluation.
- Onboard a pilot team at the “use” or “assemble” level, then industrialise onboarding for the next ones.
- Set up a light centre of excellence: standards, model approval, responsible AI.
Further reading
- AWS — Generative AI operating models in enterprise organizations (opens in a new tab)
- AWS — Build a multi-tenant generative AI environment for your enterprise (opens in a new tab)
- AWS — Bedrock AgentCore overview (opens in a new tab)
- Microsoft — AI gateway capabilities in Azure API Management (opens in a new tab)
- Microsoft — Use a gateway in front of model deployments (opens in a new tab)
- Microsoft — Establish an AI Center of Excellence (opens in a new tab)
- Google Cloud — Agentic AI architecture guides (opens in a new tab)