AI Centre of Excellence: Three Models and How to Choose

There are three common operating models for an AI Centre of Excellence: centralised, federated, and hub-and-spoke. Centralised puts one team in control of build, platform, and approvals. Federated pushes ownership out to business units with light central guardrails. Hub-and-spoke splits the difference — a central hub owns platform and governance while embedded practitioners in each business unit handle delivery. The right fit depends on three things: how many AI initiatives you are running concurrently, how mature your business units are as builders, and how much you are willing to invest in central governance versus central delivery. Below, we unpack each model, when it suits, and where it tends to break down.

Running three or four AI pilots is manageable with a single enthusiastic team and informal coordination. Running initiatives across multiple business units, each with its own data access needs, model choices, and risk profile, is a different problem. At that point, the constraint stops being technical and becomes organisational: who owns standards, who approves production deployment, and who stops duplicated effort before it starts. "Set up a Centre of Excellence" is not itself a plan — it is a label that can sit on top of at least three genuinely different operating models, each with different trade-offs for speed, consistency, and control.

What Is an AI Centre of Excellence?

An AI Centre of Excellence is a cross-functional structure — people, standards, and shared infrastructure — responsible for coordinating how an organisation builds, governs, and scales AI capability across multiple teams or business units. It is not a single team that builds all the AI; it is the mechanism that decides how AI work gets done consistently, wherever it happens.

A CoE typically owns some combination of: technical standards (approved models, evaluation methods, data access patterns), governance (risk review, approval gates, compliance sign-off), shared platform components (feature stores, retrieval infrastructure, monitoring), and capability-building (training, playbooks, internal consulting support for business-unit teams).

Why Do Organisations Need a Formal Model to Scale AI?

Without a defined operating model, AI initiatives tend to scale by accident rather than by design. Different teams pick different vendors, duplicate data pipelines, apply inconsistent risk review, and ship models with no shared monitoring standard. This multiplies technical debt and audit risk at roughly the same rate as adoption — the same dynamic we see in legacy systems that grew without a modernisation plan, which is why application modernisation work so often starts with an honest audit of what exists before deciding what to build.

The operating model you choose determines where decisions get made, how fast teams can move, and how much duplicated effort you tolerate in exchange for speed. There is no universally correct answer — the right model depends on your risk tolerance, the maturity of your business units, and how centralised your data and infrastructure already are. This is closely related to the decisions covered in ai product strategy work: operating model and product strategy should be designed together, not sequentially.

What Is a Centralised AI CoE Model?

A centralised model is one where a single, dedicated AI team builds and owns most production AI capability on behalf of the whole organisation, with business units acting as requesters rather than builders. This team typically reports into a CDO, CTO, or Head of AI, and controls the roadmap, the platform, and the approval process end to end. The team doing the building usually sits close to, or inside, core ai engineering practice — evaluation, deployment, and monitoring standards live with them, not with the requesting business unit.

Centralisation is strong on consistency — one set of standards, one platform, one place to audit. It is weaker on responsiveness: business units with urgent or domain-specific needs often queue behind a shared backlog, and the central team can become a bottleneck as demand grows. It tends to suit organisations earlier in their AI maturity, where the risk of inconsistent governance outweighs the cost of slower delivery.

What Is a Federated AI CoE Model?

A federated model distributes AI ownership to individual business units, each running its own initiatives with its own embedded practitioners, while a small central group sets shared standards and governance but does not build on the units' behalf. Business units move independently within guardrails rather than through a central queue.

Federation is strong on speed and domain relevance — teams closest to the problem build the solution. It is weaker on consistency: without active central coordination, business units can drift toward incompatible tooling, duplicated infrastructure, and uneven risk practices. Federation tends to work best where business units already have reasonably mature engineering capability and the organisation is willing to invest in strong, well-enforced central standards rather than central delivery.

What Is a Hub-and-Spoke AI CoE Model?

A hub-and-spoke model combines a central hub — responsible for platform, governance, and shared services — with embedded "spoke" practitioners inside each business unit who build domain-specific solutions using hub-provided standards and infrastructure. It is a deliberate middle ground between the other two.

The hub typically owns the things that are expensive to duplicate and risky to get wrong: approved model registries, evaluation frameworks, data access and governance policy, and shared platform components such as retrieval infrastructure or feature stores — the kind of shared technical foundation covered under data infrastructure work. The spokes own delivery speed and domain context. This model requires more deliberate design up front — clear interfaces between hub and spoke, and real investment in the hub's platform — but it tends to scale better than either pure model once an organisation is running more than a handful of concurrent AI initiatives.

Comparing the Three Models

Dimension	Centralised	Federated	Hub-and-spoke
Delivery speed	Lower (shared queue)	Higher	Moderate to high
Governance consistency	Higher	Lower without strong enforcement	Higher
Duplicated effort risk	Low	Higher	Low to moderate
Best suited to	Earlier-stage AI maturity, few initiatives	Mature, independent business units	Multiple concurrent initiatives, mixed maturity
Central investment required	High (build capacity)	Low (standards only)	High (platform and governance)

How Do You Choose the Right Model?

Start by counting your actual initiatives, not your ambitions. A handful of concurrent AI projects rarely justifies the overhead of a hub-and-spoke model — a lean centralised team, or even informal coordination, is often enough. Once you are running many initiatives across different business units with different data needs, the coordination cost of a federated-only model usually starts to show up as inconsistent risk review and duplicated platform spend.

Assess business-unit maturity honestly. Federation assumes the units can build and operate AI responsibly on their own; if that capability does not exist yet, federation without a strong central standards function tends to produce inconsistent, hard-to-audit outcomes rather than speed.

Finally, be realistic about what you are willing to fund centrally. Centralised and hub-and-spoke models both require genuine investment in a central team or platform — not just a policy document. A CoE that owns governance on paper but has no budget for shared infrastructure or enforcement tends to default to whatever the loudest business unit decides, regardless of which model is named on the org chart.

None of these models is static. Organisations that start centralised often move toward hub-and-spoke as business units mature and initiative count grows. The model should be revisited as a deliberate decision, not left as an artefact of how the first pilot happened to be staffed.

If you're weighing up which operating model fits your organisation, or need help designing the governance and platform foundations behind it, we'd welcome the conversation — read more in more insights or get in touch to talk through where you're at.