On-Premises AI for the Public Sector: Sovereignty, Compliance, and Trust

The Unique Governance Context of Public Sector AI

Public sector organizations are different from private enterprises in several ways that directly affect AI architecture choices.

Citizen data is not commercial data. Public administrations hold some of the most sensitive data that exists about individuals — health records, tax information, criminal histories, immigration status, social welfare records, educational records. The legal basis for holding this data is statutory, not contractual. Using it to train AI models, exposing it to third-party AI services, or processing it on infrastructure outside national control requires explicit legal authorization that most public bodies do not have.

Public accountability is qualitatively different from regulatory compliance. A private company that fails to meet a compliance obligation faces regulatory consequences. A public body that is found to have sent citizen data to a commercial AI provider faces political consequences, public trust damage, and potential ministerial accountability. The bar for AI deployment decisions is correspondingly higher.

Concentration and dependency risk has national security dimensions. When a government agency’s operational capabilities depend on a commercial AI provider’s infrastructure, the agency is exposed to vendor decisions — price changes, service modifications, infrastructure outages — in a way that affects public service delivery. For critical public services, this dependency is unacceptable. National AI infrastructure should be under national control.

Procurement and competition law constraints. Public procurement rules in EU member states create specific requirements for transparency, competition, and value for public money that affect how AI systems can be procured and how long-term dependencies can be created. On-premises platforms based on open-weight models are generally more consistent with these constraints than proprietary cloud AI services with long-term lock-in.

EU AI Act High-Risk Categories Affecting the Public Sector

The EU AI Act’s high-risk classification covers a significant proportion of the AI use cases that matter most to public sector organizations. Public sector bodies must be aware of which of their planned AI systems fall into these categories and what obligations follow.

Education and vocational training. AI systems used to determine access to educational establishments, assess learning outcomes, or evaluate learners are classified as high-risk. For universities, schools, and vocational training authorities that are exploring AI-assisted assessment or admissions, this classification brings documentation, logging, and human oversight obligations.

Employment and worker management. AI systems used to make or inform decisions about employment, including recruitment, task allocation, monitoring, and performance evaluation, are high-risk. Public employers that use AI in civil service recruitment or workforce management must meet these obligations.

Essential public services. AI systems used to determine access to essential public services — including social benefits, health services, and public housing — are high-risk. These are precisely the use cases where AI offers the most potential productivity benefit to public administrations, and where the compliance obligations are most demanding.

Law enforcement. AI systems used in law enforcement, including risk assessment tools and crime prediction or prevention systems, are subject to the EU AI Act’s strictest requirements, and certain applications are prohibited entirely.

Border control and migration. AI systems used in border control and immigration are classified as high-risk, with specific obligations around documentation and human oversight.

Administration of justice. AI systems that assist courts and judicial authorities are high-risk. For ministries of justice and court systems exploring AI, this classification is directly relevant.

For each of these categories, the obligations under the EU AI Act — risk management, data governance, technical documentation, logging, transparency, human oversight, accuracy, robustness, and cybersecurity — must be met before the system is placed into operation. On-premises deployment provides the technical foundation for meeting these obligations under public sector control.

GDPR Obligations for Public Sector AI

The GDPR applies fully to public sector processing of personal data, and several of its provisions are particularly significant for AI systems.

Lawful basis for processing. Every AI system that processes personal data about citizens must have a lawful basis. For public authorities, the lawful basis is usually a legal task or public interest, but this must be specifically established for the AI processing — not inherited from the general basis for holding the data. Sending citizen data to an external AI API for processing is a new processing activity that requires its own lawful basis.

Data minimisation. AI systems should not process more personal data than is necessary for their purpose. Many large AI models, if given access to broad data stores, will process far more data than any individual query requires. On-premises RAG architectures with permission-aware retrieval support data minimisation by retrieving only what is relevant and authorized.

Data residency and cross-border transfers. GDPR restricts transfers of personal data to countries outside the European Economic Area without adequate protection. Sending citizen data to AI services hosted in the United States or other third countries requires specific legal mechanisms — Standard Contractual Clauses, adequacy decisions, or derogations — that create ongoing compliance obligations. On-premises deployment within EU territory eliminates the cross-border transfer question entirely.

Rights of data subjects. Citizens have rights under GDPR including the right to know whether automated decision-making is used, the right to meaningful information about the logic involved, and the right to human review of significant automated decisions. AI systems that inform decisions about citizens must be designed to support these rights — which requires detailed logging, explanation capabilities, and human oversight workflows.

What a Sovereign Public Sector AI Architecture Looks Like

A sovereign AI architecture for the public sector is designed to keep citizen data, AI processing, and audit evidence within nationally controlled infrastructure. The core components are:

On-premises model inference. Open-weight large language models running on government-controlled GPU infrastructure, either in agency data centres, government cloud infrastructure, or contracted national cloud providers that operate under national data protection law. No citizen data leaves the national infrastructure boundary during AI processing.

Private RAG with document-level access controls. Government agencies hold vast document stores — legislation, regulations, policy guidance, case records, precedents, administrative procedures. A private RAG layer makes this knowledge accessible to AI agents without sending document content to external services. Access controls must enforce that each civil servant or case worker can only retrieve documents their role authorizes — not all documents in the knowledge base.

Agent orchestration with governance controls. An orchestration layer that routes tasks to appropriate agents, enforces policy constraints, produces complete interaction logs, and supports human oversight for decisions with significant citizen impact. This layer ensures that AI outputs that affect individual citizens are reviewed by a qualified civil servant before they are actioned.

Audit logging and evidence packaging. Complete, tamper-evident logs of every AI interaction, accessible to the agency’s data protection officer, internal audit function, and competent authorities without depending on a commercial provider’s cooperation. Log retention periods should be set to meet both GDPR data minimisation requirements and the evidence retention periods that regulatory oversight requires.

Model governance and change management. Documented processes for approving, validating, and deploying model changes within the government’s AI systems. Model changes should follow the same change management disciplines applied to critical public sector software — with testing, documentation, approval, and rollback capability.

High-Value Public Sector Use Cases

On-premises AI can create significant value for public sector organizations without compromising citizen data protection:

Policy and regulation document Q&A. Civil servants spend significant time searching through legislation, regulations, policy circulars, and administrative guidance. A private RAG system over government document stores allows policy questions to be answered faster, with source attribution, and without sending government documents to external services. The time savings per civil servant are substantial at scale.

Case worker knowledge assistance. Social welfare case workers, immigration officers, and benefit administrators work with complex, frequently updated rules. An AI assistant that can answer procedural questions from authoritative internal guidance reduces errors and improves consistency in decisions affecting citizens.

Permit and application processing support. Many government agencies process high volumes of permits, applications, and registrations. AI agents can classify incoming applications, extract key information, check completeness, and flag issues for human review — accelerating processing while keeping human officers in the decision loop.

Public communication drafting assistance. Government communications must be clear, accurate, and legally sound. AI can assist with drafting press releases, citizen letters, FAQ documents, and consultation responses, with the final text reviewed and approved by human officials before publication.

Internal compliance and audit support. Government bodies are themselves subject to oversight and audit. AI agents can support audit preparation, help teams identify compliance gaps in their procedures, and assist with the documentation requirements that oversight bodies require.

Practical Considerations for Public Sector AI Procurement

When procuring on-premises AI capabilities, public sector organizations should consider several practical dimensions:

Open-weight model access. An on-premises deployment that depends on a proprietary model creates a new form of vendor dependency. Prefer platforms that support open-weight models — models whose weights can be downloaded, hosted locally, and operated without ongoing licensing to a single provider. This preserves vendor neutrality and supports procurement compliance.

Data centre and infrastructure requirements. GPU infrastructure for model inference has specific power, cooling, and physical security requirements. Agencies should assess whether existing data centre infrastructure can support the GPU requirements of their planned AI workloads, or whether investment in new infrastructure is required.

Integration with existing systems. Government AI systems must connect to existing case management, document management, and identity management systems. Integration architecture should maintain the access control policies of those systems, not bypass them.

Staff training and change management. Civil servants using AI tools need to understand both the capabilities and the limitations of the systems. Training should cover how to interpret AI outputs, when to escalate to human review, and how to exercise the oversight role required for high-risk AI systems.

VDF AI’s platform is designed for this deployment context — running entirely within customer infrastructure, supporting private RAG over government document stores, enforcing role-based access controls at the retrieval layer, producing audit logs to the standard required for public sector accountability, and supporting the human oversight workflows that EU AI Act obligations require.

Conclusion

The public sector case for on-premises AI is not primarily technical. It is political, legal, and ethical. Citizens have a right to expect that government agencies treat their data with the care that statutory obligations and public trust require. AI systems that route citizen data through commercial infrastructure outside government control do not meet that expectation.

Sovereign on-premises AI architecture — with government-controlled infrastructure, private model inference, permission-aware retrieval, full audit logging, and human oversight for citizen-affecting decisions — is not a constraint on public sector AI ambition. It is the foundation on which public sector AI can be deployed responsibly, at scale, with the accountability that public institutions require.