GxP AI Validation: How to Validate AI and LLMs in Pharma Under FDA CSA, Part 11 and EU GMP Annex 22

Pharmaceutical companies already know how to validate software. What AI changes is the nature of the system under test. A language model can give different answers to the same question, can be updated by its provider without notice, and can produce fluent text that is wrong. Validation methods built for deterministic software need adjusting, and regulators are still writing the AI-specific rules.

This guide sets out where those rules stand, checked against FDA, European Commission and EMA sources on 22 October 2026, and turns them into a working approach. For the broader picture of AI in pharma, see our guide to private AI in pharmaceutical and life sciences companies.

The rules that apply to AI in GxP work in October 2026

The short version: no binding rule written for AI applies to GMP manufacturing yet. Existing computerised system rules apply in full, and the drafts show where regulators are heading.

Intended use is where GxP AI validation starts

Every framework in that table begins in the same place. FDA’s CSA guidance starts its risk framework with identifying the intended use. The FDA credibility draft starts with the question of interest and the context of use. Annex 22’s draft asks for a detailed description of what the model assists or automates, written and approved by a process subject matter expert before acceptance testing.

For an AI system, a good intended use statement covers:

• the task, such as “draft a first summary of a deviation report for a QA reviewer”;
• the inputs, including their common and rare variations, and inputs the model must not receive;
• who uses the output and what decision it feeds;
• whether a human reviews every output, a sample, or none;
• the GxP records the system creates or changes.

A vague statement such as “assist QA with documents” cannot be validated, because there is nothing to test against. Narrow statements also keep the risk assessment honest. Summarising a deviation for a reviewer is a different risk from deciding whether the deviation needs a CAPA.

Risk-based validation: computer software assurance applied to AI

FDA’s computer software assurance guidance replaced section 6 of its 2002 software validation guidance for production and quality system software. The February 2026 version says the approach can be applied to automation tools, data analytics tools, artificial intelligence and machine learning tools and cloud computing used in production or the quality management system.

Its central test is process risk. FDA treats a software function as high process risk when its failure may cause a quality problem that foreseeably compromises safety. Its examples include software that measures or decides product acceptability with limited or no human review. Functions that only collect data for review, route CAPAs or manage records are generally not high process risk.

Assurance effort follows that split. High-risk functions call for more rigour, such as scripted testing with recorded test cases and expected results. Lower-risk functions can rely on unscripted methods: scenario testing, error guessing and exploratory testing by people who know the process. The guidance also lets manufacturers leverage validation work already done by developers, suppliers and cloud service providers.

The guidance is written for device makers under 21 CFR Part 820, which became the Quality Management System Regulation in February 2026. Drug manufacturers are not bound by it, but GAMP 5 Second Edition already discusses CSA, and the same thinking runs through Annex 22’s draft. The words differ; the logic is the same.

GxP impact level and the validation activities that match

Most generative AI projects in pharma sit in the second row. That is where effort pays off: a clear intended use, a test set that reflects real documents, trained reviewers, and records that show who approved what.

Static versus dynamic models, and why the Annex 22 draft keeps LLMs out of critical uses

Annex 22’s draft draws three lines. It covers static models, which do not change during use, and excludes dynamic models that keep learning from new data. It covers models with deterministic output, which give identical outputs for identical inputs, and excludes probabilistic ones. Then it applies both lines to generative AI: the draft “does not apply to Generative AI and Large Language Models (LLM), and such models should not be used in critical GMP applications”.

The reasoning follows from the validation method. Acceptance testing assumes the system you tested is the system you run. A model that adapts, or that samples a different answer each time, weakens that assumption. An LLM can be run with greedy decoding to make its output more repeatable on fixed hardware and software, but the draft does not say whether that would be enough to count as deterministic.

The same paragraph leaves room for LLMs in non-critical GMP applications, which it defines as those without direct impact on patient safety, product quality or data integrity. There, the draft says qualified and trained personnel should always be responsible for checking that outputs suit the intended use, and the rest of the annex may be applied where relevant.

The draft also has limits in scope. It covers the manufacture of medicinal products and active substances. Pharmacovigilance, clinical trials and regulatory submissions fall under other guidance, such as GVP and GCP, even though the same risk thinking applies.

What EMA heard at the June 2026 workshop

The consultation drew industry support for allowing generative AI with controls. EMA’s GMP and GDP Inspectors Working Group responded with a multistakeholder workshop on 30 June and 1 July 2026, focused on a risk-based approach to generative AI in manufacturing. Its report, EMA/156789/2026, was published on 2 October 2026.

The report records a largely aligned industry position across six topics. Participants argued that acceptability should depend on intended use, risk and the effectiveness of controls across the lifecycle, rather than on the underlying technology. Supplier and cloud provider oversight came up repeatedly, because the regulated company’s accountability cannot be transferred to the provider. The report concludes that there may be viable pathways for adaptive, probabilistic and generative AI in GMP applications, subject to appropriate controls.

That is input to the drafting group, not a new rule. The report’s next steps say the drafting group will use the outcomes to inform the revision of Annex 22, and no adoption date has been announced. Until a final text appears, plan around the draft and keep LLMs in non-critical uses with human review.

Human-in-the-loop is a control you have to validate too

Human review is the main reason an LLM can be used near GxP work at all, so it has to work in practice. The Annex 22 draft is specific about this. Where a model gives input to a decision made by a human operator and testing effort has been reduced on that basis, the intended use should state the operator’s responsibility, and the operator’s training and consistent performance should be monitored like any other manual process. Records of the review should be kept, and depending on criticality this may mean reviewing every output.

In practice:

• Define what the reviewer checks. “Review the summary” is not enough. List the points to verify against the source, such as batch numbers, dates, test results and the deviation category.
• Make the source easy to check. Outputs should cite the documents and passages they used, so the reviewer compares rather than rereads.
• Record the review. Capture who approved, when, and what they changed, in the same audit trail as the model output.
• Watch for automation bias. Seed occasional known errors into training exercises and track whether reviewers catch them.

Data integrity and ALCOA+ for prompts and outputs

Data integrity rules apply to AI records as they apply to any GxP record. FDA’s data integrity guidance uses ALCOA: data should be attributable, legible, contemporaneous, original and accurate. The plus, used in European and PIC/S guidance, adds complete, consistent, enduring and available.

For an AI-assisted record, that means keeping:

• the prompt or task, and the user who submitted it;
• the documents and passages retrieval supplied as context;
• the model name, version and settings that produced the output;
• the raw output and the reviewed, approved version;
• time stamps for each step, in a secure audit trail that users cannot edit.

Part 11 already requires validation of systems that hold electronic records and secure, computer-generated, time-stamped audit trails for record changes. If an AI output becomes part of a batch record, deviation report or other GxP record, those requirements reach the AI system that produced it. The revised Chapter 4 draft moves in the same direction, with risk-based data governance across paper, electronic and hybrid records.

Change control for model versions

The Annex 22 draft asks for a tested model, the system it runs in and the process it supports to be under change control before use. Any change to the model, the system or the process should be evaluated for retesting, and a decision not to retest should be justified. A tested model should also be under configuration control, with measures to detect unauthorised changes. Performance and input drift should be monitored during operation.

Hosted model APIs make that hard. Providers retire model versions on their own schedules, and an alias such as “latest” can change behaviour overnight. You can still validate a hosted model, but your change control then depends on the provider’s notices, and your supplier assessment has to cover how it announces changes.

Self-hosted open-weight models are easier to control. The weights are files with checksums. The serving version, quantization and settings can be fixed in configuration, and nothing changes until your change control approves it. When you do upgrade, rerun the same regression set used at validation and file the comparison with the change record.

How VDF AI fits a GxP validation scope

VDF AI is a self-hosted platform for private assistants, retrieval over approved documents and governed agents, installed on your servers, in a private cloud or air-gapped. Models, retrieval indexes, conversation history and audit logs stay inside your environment, so they fall under your own computerised system controls. VDF AI holds no GxP validated status, because validation belongs to your intended use; the platform sits inside your validation scope like any other computerised system.

Several platform controls map to the needs above. VDF AI Agents can pin agents to specific model versions, pause for human approval before high-impact actions, and keep an immutable audit log of each prompt, retrieval hit, tool call and model response. Retrieval respects each user’s permissions, and role-based access control comes with every plan. VDF AI Router registers local model endpoints, and air-gap mode keeps every request on models you host.

Typical first workloads are non-critical by design: answers from approved SOPs and compliance documents, pharmacovigilance case intake with human entry and the regulatory submissions playbook. For the wider rollout, see the healthcare and life sciences solution and the private GPT guide for life sciences; EU teams should also read our EU AI Act guide for healthcare and medical devices. This article is general information, not regulatory advice.