Air-Gapped RAG
A RAG (retrieval-augmented generation) system grounds LLM answers in your own documents — indexing them into a vector store, retrieving the relevant passages per question, and generating cited answers instead of hallucinations, operating on a network with no connection to the public internet — models, updates, and telemetry all move by controlled offline transfer, so the system functions fully inside a classified or isolated enclave.
Air-gapped RAG is the killer app of classified AI: enclaves hold enormous document corpora that keyword search barely penetrates. Because the corpus, embeddings, and models all live inside the gap, analysts get cited, source-grounded answers over classified material with zero disclosure surface — capability that simply cannot exist in any cloud architecture.
The hard problem in an air-gapped RAG system is not retrieval quality — it is the supply chain. Every embedding model, reranker, OCR model, container image and Python wheel has to arrive as a signed artifact through your cross-domain transfer process. A pipeline that quietly pulls a tokenizer from Hugging Face on first run works perfectly in staging and fails accreditation the moment it is inside the enclave.
The second constraint is re-indexing economics. In a connected deployment, swapping an embedding model is a Tuesday afternoon: you re-embed against a hosted endpoint and move on. Inside the gap, a model swap means a full corpus re-embed on enclave GPUs, with no ability to burst capacity. That single fact should drive your embedding-model choice harder than any leaderboard — pick a model you can live with for a year, and size the backfill before you size the query load.
Third, you have to evaluate blind. You cannot ship query logs to a vendor, and you cannot ask a hosted judge model to score answers. Retrieval quality has to be measured with a golden-question set that lives inside the enclave, run by an offline harness, with results reviewed by people who are cleared to read both the questions and the documents.
Why teams run their RAG system air-gapped
Built for defense, intelligence, critical-infrastructure and classified-environment teams.
Zero external connectivity, by design
An air-gapped RAG system makes no outbound calls — no license pings, no telemetry, no model API fallbacks. If a component phones home, it fails certification; the architecture must assume the internet does not exist.
Built for classified and SCIF environments
Defense, intelligence, and critical-infrastructure operators need AI capability where cloud AI is categorically prohibited. The RAG system runs entirely on enclave hardware and clears accreditation reviews because there is nothing external to assess.
Controlled update path
Models, embeddings, and software updates arrive as signed offline bundles through your cross-domain transfer process — the same discipline you already apply to any software entering the enclave.
Core capabilities of an enterprise RAG system
Document ingestion & chunking
Index wikis, policies, contracts, and tickets with structure-aware chunking so retrieval returns answers, not fragments.
Hybrid retrieval
Combine vector similarity with keyword and metadata filters — the difference between demo-grade and production-grade accuracy.
Cited, source-backed answers
Every answer links to the source passages, so users can verify and auditors can trace.
Access-aware retrieval
Retrieval respects document permissions per user — the answer engine never becomes a permissions bypass.
What an air-gapped deployment changes
Everything ships as a self-contained bundle: container images, model weights, embedding models, and documentation must install from local media with no registry or CDN access.
Local models only: the RAG system serves open-weight models on enclave GPUs; there is no cloud fallback tier, so model selection and routing happen entirely inside the gap.
Audit evidence must be exportable on your terms — logs stay in the enclave and leave only through your controlled review process.
The air-gapped RAG system stack
Air-gapped RAG, layer by layer — with the reason each choice holds up under this deployment mode.
| Layer | Typical choice | Why, here |
|---|---|---|
| Embedding model | bge-m3 or multilingual-e5-large, staged as local weights | Both are open-weight and run offline. Choose once: re-embedding a large corpus on enclave-only GPUs is the most expensive routine operation in the system. |
| Vector store | Qdrant or Milvus self-hosted; pgvector for smaller corpora | All three install from local images with no registry access. pgvector keeps the footprint to a database you already back up, which matters when the enclave has no object storage tier. |
| Generation | vLLM serving an open-weight model on enclave GPUs | There is no cloud fallback tier, so the generator has to be sized for peak in-enclave concurrency rather than for an average with burst. |
| Ingestion & OCR | Apache Tika plus a pre-staged OCR model | Scanned material is common in classified archives. OCR models are frequently forgotten in the bundle and are the most common cause of a failed first ingest. |
| Identity & ACLs | In-enclave Keycloak or AD FS, ACLs mirrored at index time | Permissions must be captured with each chunk during ingest — there is no live call to the source system at query time to re-check them. |
| Artifact transfer | Signed OCI bundles into an offline registry (Harbor or Zot) | Gives the enclave a real registry so upgrades are a normal pull rather than an ad-hoc file copy, and gives your reviewers a signature to verify. |
Sizing an air-gapped RAG system
| Profile | Scale | Hardware | What actually binds |
|---|---|---|---|
| Pilot enclave | 50–200 analysts, corpus under ~500k chunks | One 48 GB GPU for generation; embedding on the same card off-hours | The initial embedding backfill, not query traffic, sets the day-one timeline. Budget for it explicitly. |
| Program scale | 500–2,000 users, 2–10M chunks | 2–4 GPUs for generation plus a separate CPU/RAM node for the vector store | Vector-store memory is usually the first wall, not GPU. Index parameters (HNSW graph degree) decide whether the corpus fits. |
| Multi-enclave | Several isolated networks running the same corpus | Independent GPU and storage per enclave; one identical bundle set | The real constraint is version drift between enclaves. Treat the bundle, not the deployment, as the unit of release. |
Regulations that point to air-gapped
Classified handling
The RAG system operates inside SCIF and enclave boundaries; there is nothing to accredit outside them.
ITAR / export control
Technical data entering the RAG system never transits foreign-controlled infrastructure.
NIS2 / NERC CIP
Critical-infrastructure isolation is met structurally by the RAG system, not contractually.
Zero-trust postures
No third-party endpoints to allow-list — the RAG system’s attack surface is your own network.
When air-gapped is the right call — and when it isn’t
Choose air-gapped when
- The network the RAG system must serve is already isolated — classified programs, OT networks, offline research enclaves.
- Policy prohibits any external AI API, including via proxy or private link.
- You need AI capability in disconnected field or vessel environments with intermittent or no connectivity.
Consider another mode when
- You can tolerate controlled outbound connectivity → an on-premises RAG system is simpler to operate and update.
- Your requirement is legal jurisdiction rather than physical isolation → the sovereign variant fits; air-gapping an RAG system is stricter than most regulators ask.
Same capability, different deployment mode:
RAG: Air-Gapped vs the alternatives
| Deployment mode | Typical owner | What you gain — and give up |
|---|---|---|
| Air-Gapped (this page) | CISO / Classified Program Lead | Structural security no contract can match — in exchange for moving every model, index, and software update through an offline bundle process. |
| On-Premises | CTO / Head of Infrastructure | Maximum physical control and the strongest economics at steady volume — in exchange for owning the hardware, the capacity plan, and the upgrade cycle. |
| Self-Hosted | Platform Engineering Lead | Complete stack and model freedom with no usage meter — in exchange for your team owning operations, CVE response, and the upgrade cadence. |
| Sovereign | CIO / Chief Data Officer (public sector & regulated EU) | Legal control that survives foreign disclosure orders and sanctions — in exchange for in-country hosting constraints and heavier procurement diligence. |
How to deploy an air-gapped RAG system
- 01
Fix the corpus boundary and its permission source
Decide exactly which repositories are in scope and where their access control lives. In an air-gapped system the index inherits permissions at ingest time, so an unclear ACL source of truth becomes an unclear security boundary.
- 02
Assemble and transfer the offline bundle
Package container images, embedding and OCR models, the generator weights, and the index tooling as one signed bundle. Verify on a disconnected staging network that nothing attempts an outbound call before it goes through cross-domain transfer.
- 03
Build the index and red-team the permissions
Run the first full ingest inside the enclave, then test retrieval with accounts at several clearance levels. Query for documents each account must not see and confirm they are absent from the retrieved context, not merely absent from the answer.
- 04
Measure retrieval before wiring generation
Score a golden-question set against retrieval alone. If the right passage is not in the top results, no amount of generation quality will fix the answer — and inside the gap you cannot debug that with a vendor.
- 05
Set the re-bundle cadence
Agree how often model and software updates are packaged, reviewed, and transferred. A quarterly cadence with a defined rollback bundle is far more sustainable than ad-hoc updates that each need their own approval.
Where air-gapped RAG system projects fail
Permission leakage through the retrieved context
The answer looks correctly redacted while the underlying passage was still retrieved and passed to the model. Enforce access control at the retrieval filter, not in a post-processing step over generated text.
Treating a model swap as a routine upgrade
Changing the embedding model invalidates every vector in the store. Inside an enclave that is a scheduled, capacity-planned re-embed — teams that discover this mid-upgrade lose retrieval for days.
No offline evaluation harness
Without golden questions and an in-enclave scorer, quality regressions after a re-index are invisible until analysts stop trusting the system. By then the credibility cost is much harder to reverse than the technical fault.
How to evaluate an air-gapped RAG system
Does retrieval enforce per-user document permissions at query time?
Are answers cited to sources, with retrieval quality measurable on your corpus?
Which embedding models are used, and do they run inside your environment?
How does the pipeline handle updates — re-indexing cadence, deletion propagation?
Can the RAG layer serve multiple agents and applications, not just one chatbot?
Air-gapped deployments trade update convenience for structural security; budget for the offline bundle process, but the RAG system itself prices like any fixed in-enclave infrastructure — no meters, no per-token exposure.
An air-gapped RAG system, on the VDF AI platform
VDF AI’s private RAG layer indexes your corpus inside your perimeter, enforces document ACLs at query time, and serves cited answers to both chat users and agent workflows.
Air-Gapped RAG questions, answered
What is an air-gapped RAG system?
A RAG (retrieval-augmented generation) system grounds LLM answers in your own documents — indexing them into a vector store, retrieving the relevant passages per question, and generating cited answers instead of hallucinations, operating on a network with no connection to the public internet — models, updates, and telemetry all move by controlled offline transfer, so the system functions fully inside a classified or isolated enclave.
Why do enterprises choose an air-gapped RAG system over a cloud service?
An air-gapped RAG system makes no outbound calls — no license pings, no telemetry, no model API fallbacks. If a component phones home, it fails certification; the architecture must assume the internet does not exist. Air-gapped deployments trade update convenience for structural security; budget for the offline bundle process, but the RAG system itself prices like any fixed in-enclave infrastructure — no meters, no per-token exposure.
Which regulations drive air-gapped RAG system adoption?
The most common drivers are Classified handling, ITAR / export control, NIS2 / NERC CIP, Zero-trust postures. Classified handling: The RAG system operates inside SCIF and enclave boundaries; there is nothing to accredit outside them.
Can VDF AI run as an air-gapped RAG system?
Yes. VDF AI’s private RAG layer indexes your corpus inside your perimeter, enforces document ACLs at query time, and serves cited answers to both chat users and agent workflows. VDF AI ships as a signed offline bundle with the embedding models, vector store, and generator included, and enforces document ACLs at the retrieval filter so a cleared user and an uncleared one get genuinely different context.
Can RAG work without any internet connection?
Yes. Retrieval-augmented generation needs no external service at query time: the embedding model, vector store, and generator all run locally. What internet access normally provides is convenience during setup — pulling models, images, and dependencies. In an air-gapped deployment those arrive instead as a signed offline bundle, which is a packaging problem rather than an architectural one.
How do you update the embedding model in an air-gapped RAG system?
Transfer the new model as part of a signed bundle, then re-embed the corpus inside the enclave and rebuild the index. Because old and new vectors are not comparable, plan it as a migration: build the new index alongside the current one where storage allows, validate it against your golden questions, then cut over. Expect the re-embed to be the longest step.
Which vector database works in an air-gapped environment?
Qdrant, Milvus, and pgvector all run fully offline from local container images and need no license check or telemetry endpoint. The practical selector is operational rather than functional: pgvector if you want one fewer system to back up and your corpus is modest, Qdrant or Milvus once index size or query concurrency outgrows a single database node.
How large does a corpus need to be before air-gapped RAG needs dedicated GPUs?
Generation needs a GPU from the first user, but embedding is the workload that forces dedicated capacity. Below roughly half a million chunks the backfill can share the generation GPU off-hours. Past a few million chunks, a re-index becomes long enough that it competes with production traffic, and a separate embedding card stops being optional.
Related guides and resources
Evaluate your knowledge stack
Find out how a private RAG and retrieval layer would perform on your data — accuracy, latency, governance, and what to fix before you scale.