AI Team Structure: Roles, Team Topologies, Sizing by Stage and a RACI for Enterprise AI

An effective AI team structure pairs a small central team, which runs the shared AI platform, sets standards and reviews risk, with delivery pods embedded in the business units that own the results. Each pod usually combines an AI product owner and a domain expert with AI and data engineers. The shape shifts with maturity: one cross-functional team for the first pilots, a platform team with several pods at scale, and named owners and human reviewers for every agent once agents act in production.

What an AI team has to cover

Every AI programme has five kinds of work to staff: deciding what to build or buy, building it, running it in production, governing its risk, and helping people use it well. Programmes that staff only the building tend to stall at the first production release, when nobody owns operations, approvals or adoption.

The roles are also shifting faster than job descriptions. Gartner’s survey for its 2027 CIO and Technology Executive Agenda, which polled 2,002 CIOs and technology executives from May to July 2026, found that organisations with explicit plans to redesign roles for a human-agent workforce were 2.4 times as likely to be in the group Gartner calls vanguards, yet only 4% of CIOs had such plans. Gartner expects organisations to need fewer distinct role types, each with a wider span of responsibility, as AI absorbs specialised tasks.

This guide covers the people. Where decisions sit, how AI work is funded and which archetype fits are covered in the AI operating model explainer, and the charter and first 90 days of a central AI team in the guide to setting up an AI center of excellence.

AI team roles and responsibilities

These are the core enterprise AI roles. In a small programme one person often covers two or three of them; in a large one, each becomes a team.

Two leadership roles sit above these: an executive sponsor who holds the budget and settles priority disputes between units, and a named AI leader who runs the programme day to day. The AI leader needs authority over the shared platform and the standards, not over every builder.

AI engineer, ML engineer or data scientist?

The titles overlap, so define the work rather than the label. An AI engineer builds products on top of existing models, which describes most enterprise generative AI work. A machine learning engineer changes the model itself by training, fine-tuning or distilling it. A data scientist frames questions, analyses data and designs experiments. If you call models rather than train them, hire AI engineers first and add ML engineers when tuning begins.

Governance, risk and security roles

These are the roles most often added late. In McKinsey’s State of AI survey published in March 2025, 13% of respondents said their organisations had hired AI compliance specialists and 6% AI ethics specialists, and AI-related roles were still described as largely challenging to fill. Place the governance and risk lead in the second line, outside the delivery reporting chain, so that approving a high-risk system is never a judgement on one’s own team’s work.

Domain experts, change and enablement

Domain experts decide what a correct answer looks like, which makes their time the usual bottleneck in evaluation; budget it explicitly instead of borrowing it. For the enablement method, see the AI change management guide. Gartner, which found in October 2026 that only 38% of CHROs and CIOs share an understanding of how AI affects the future of work, recommends embedding HR staff temporarily in AI delivery teams, which it calls fusion teams, or having them coach those teams on skills maps and literacy programmes.

AI team topologies: a central platform team with embedded pods

Matthew Skelton and Manuel Pais’s Team Topologies names four fundamental team types and three ways they interact. The model maps cleanly onto AI work:

The interaction modes matter as much as the boxes. Pods consume the platform as a service, enablement works with pods by facilitation, and a new pod collaborates closely with the platform team on its first use case before moving to self-service. Team Topologies also treats cognitive load as a hard limit, so do not hand one pod a long list of unrelated use cases.

AI pod team structure

A pod is the unit of delivery: small enough to move, complete enough to ship without waiting on other teams at every step. A common shape is an AI product owner, one or two AI engineers, a data engineer and a domain expert for part of the week, with platform, security and risk support drawn from the centre. Pods are getting smaller as agents absorb routine work: McKinsey describes one product development function in which pods of eight to ten people became hybrid teams of four to six supported by agents (McKinsey Quarterly, October 2026).

AI engineering team structure inside the platform team

The platform team’s work splits into six areas: model gateway and routing, retrieval and data connectors, agent runtime and tool registry, evaluation and testing, environments and deployment, and observability and cost reporting. A handful of engineers can cover all of it at first; later each area may need its own owner. Run the platform as an internal product with a backlog, a roadmap and named users.

AI team org chart

A simple org chart for a programme at the scaling stage looks like this. Nesting shows who manages whom; dotted lines show who sets standards for whom.

• Executive sponsor (budget and priorities across units)
  • AI leader (programme, portfolio and standards)
    • AI platform team (gateway, retrieval, agent runtime, evaluation, LLMOps)
    • AI enablement (training, champions, communications)
    • AI security engineers (dotted line to the CISO)
• Business unit leaders
  • AI delivery pods (product owner, AI engineers, data engineer, domain expert), with a dotted line to the AI leader for standards
• Chief risk or compliance officer
  • AI governance and risk lead, who works alongside the AI leader but reports outside the programme

Sizing the AI team by stage

There is no reliable benchmark for AI team size, because headcount follows the number of systems in production and how much is bought rather than built. The stages below follow the levels in the AI maturity model explainer, and the readiness self-check shows which one you are at.

A worked example (illustrative)

The figures below are a fictional example that shows the arithmetic, not a benchmark. A 4,000-employee insurer at the scaling stage runs three pods: claims, underwriting support and customer service.

• Each pod: one AI product owner (1.0 FTE), two AI engineers (2.0), one data engineer (1.0) and a claims or underwriting expert for half the week (0.5), so 4.5 FTE. Three pods come to 13.5 FTE.
• Central platform team: two platform engineers, one LLMOps engineer and one evaluation engineer, so 4.0 FTE.
• Governance, security, enablement and leadership: governance and risk lead 1.0, AI security 0.5, enablement 1.0 and AI leader 1.0, so 3.5 FTE.
• Total: 13.5 + 4.0 + 3.5 = 21 FTE. If the three product owners and the 1.5 FTE of domain experts come from existing business roles, the specialist capacity to recruit or redeploy is 21 − 3.0 − 1.5 = 16.5 FTE.

If the insurer buys a platform instead of building one, the central team shrinks towards the LLMOps and evaluation roles, because the vendor maintains much of the stack.

Build the team, upskill or partner?

Most organisations will do all three, and the skills market explains why. The World Economic Forum’s Future of Jobs Report 2025, based on more than 1,000 employers, found skills gaps to be the most cited barrier to business transformation, named by 63% of employers for 2025 to 2030. In the same survey, 85% planned to prioritise upskilling, 70% expected to hire staff with new skills, 50% planned to move staff from declining to growing roles and two-thirds planned to hire talent with specific AI skills.

When partners help build, put knowledge transfer into the statement of work, pair their engineers with yours, and keep code, prompts, evaluation sets and runbooks in your own repositories. The procurement guide covers choosing vendors and delivery partners.

Upskilling also has a legal floor in the EU. Article 4 of the AI Act, as replaced by the Digital Omnibus on AI (Regulation (EU) 2026/1744, published on 24 July 2026), requires providers and deployers to take measures to support the AI literacy of their staff and others operating AI on their behalf; it does not require a guaranteed level of literacy for each person. Training is a weak point in practice: in McKinsey’s 2026 AI Trust Maturity Survey, nearly 60% of respondents named knowledge and training gaps as the main barrier to implementing responsible AI practices, up from about 50% a year earlier. A plan by role is set out in a role-based AI literacy programme, and the VDF AI Academy offers structured learning paths.

A RACI for the AI life cycle

Adjust the role names to your organisation, but keep exactly one accountable role per activity.

If two roles both feel accountable for a row, the decision right has not been settled; resolve it in the operating model rather than in the RACI. Several rows produce records you will need later, such as an entry in the AI risk register and the steps in your plan for AI incidents.

How the AI team changes when agents go to production

Agents that act on systems, rather than drafting text for a person to check, add work a chatbot programme never had. Five responsibilities appear, each with a named holder:

• Agent owner. A business manager who answers for what the agent does, approves changes to its scope and signs off its retirement.
• Agent operations. Platform or LLMOps engineers who deploy, version, monitor and roll back agents; drift monitoring becomes routine work.
• Reviewers and approvers. Named people who check sampled outputs or approve consequential actions, with the authority to pause the agent.
• Permission steward. A security or identity role that grants, reviews and removes the tools, data and systems each agent can reach.
• Evaluation and red team. Engineers who test agents before release and after every change; see red-teaming agents before production.

Regulation makes part of this mandatory. Under Article 26 of the EU AI Act, deployers of high-risk AI systems must assign human oversight to people who have the necessary competence, training, authority and support, and must monitor the system and suspend its use when they have reason to think it presents a risk. Since the Digital Omnibus, these duties apply from 2 December 2027 for the high-risk systems listed in Annex III. The NIST AI RMF asks any organisation for the same thing: defined roles and responsibilities for human-AI configurations and oversight of AI systems (GOVERN 3.2).

Plan reviewer capacity as deliberately as engineering capacity: an approval step that sends hundreds of items a day to one person becomes a rubber stamp, so use sampling, risk thresholds and escalation rules. Headcount assumptions need the same care. In McKinsey’s State of AI survey published in August 2026, 14% of respondents said AI had reduced their organisation’s workforce over the past year, against the 32% who had expected reductions a year earlier, and 39% now expect AI-related declines in the coming year. For technology teams, Gartner’s CIO survey points the other way: 40% of respondents expect to increase IT headcount in 2027, particularly in cybersecurity and data and analytics.

Common AI team structure mistakes

• Adding governance at the end. A risk lead who first sees a system at the release gate can only approve or block it; involved from intake, they shape it.
• Borrowing domain experts’ time. Evaluation stalls when the people who know the right answers have no hours set aside for it.
• One team that builds, runs and approves. It works for a pilot and fails an audit.
• Agents that outlive their sponsors. Ownership has to transfer when people move, or agents keep running with nobody accountable.

How VDF AI fits

A team built around a central platform and several pods needs a platform where the centre sets rules once and the pods build within them. In VDF AI, pods build agents in VDF AI Agents and multi-agent processes in Networks, knowledge assistants answer from company documents through private retrieval in VDF AI Chat, and engineers can use VDF Code, the governed coding assistant.

Role-based access control is part of every plan, so the roles in your RACI, such as builder, reviewer, approver and user, can be reflected in what each person is allowed to do, and audit logs give the governance lead and the security team a record of agent activity. VDF AI Router keeps model choice with the platform team: it applies the approved-model list and routing policy before a call reaches a cloud model or a registered on-premises endpoint, and its air-gap mode limits routing to local models. For sign-in, self-hosted installations use Microsoft Entra ID natively, while any other identity provider connects through an SSO-aware reverse proxy.

The platform does not hire, train or decide who owns a use case; those stay with the team design above. VDF AI runs in VDF’s managed cloud or in your own data centre, private cloud or air-gapped network, with cloud plans on the pricing page.