How to Build a Private RAG Pipeline, Step by Step

Retrieval-augmented generation (RAG) answers questions from your own content in two halves: retrieve the passages that answer the question, then give them to a model to write the answer. The retrieval half decides whether the answer can be right at all. If the right passage is never retrieved, no model can cite it.

This lesson builds the retrieval half end to end in VDF AI Data, and builds it privately: the source stays in your database, and every embedding is computed by a model served inside your network. The source is the synthetic IT service desk used throughout the Academy, with 30 knowledge-base articles in a PostgreSQL database.

Step 1: Decide what the pipeline must answer

Before you build anything, write down ten questions real users ask and the article that answers each. Write them the way people actually type, not the way the articles are titled.

Question	Should retrieve
I got a new phone and can’t approve sign-in requests	KB-1002, Resetting multi-factor authentication
The printer says offline for everyone on our floor	KB-1006, Printer shows as offline
What priority is a whole site outage?	KB-1021, Incident priority matrix

This list becomes the test set in the last lesson of this path. Write down what the pipeline must not do as well: read tables it does not need, send content outside your network, or return articles a user is not allowed to see, which is the subject of the third lesson.

Step 2: Connect the source with a read-only account

Open VDF AI Data, then Data Connections, and choose Add Connection. The connector list covers Exasol, Jira Data Center, Microsoft SQL Server, MySQL, Oracle, PostgreSQL, Presto, S3-compatible storage and SAP HANA. For PostgreSQL, enter a name, the host and port, the database and a description, then the credentials: username, password and an optional SSL mode.

Use an account that can only read the tables you intend to index. Ours, academy_reader, can select from three tables and do nothing else. Put the owner in the description, so a reviewer knows whom to ask about the data.

Save and select the connection. The details panel shows the connector, the database and the secrets, which are always masked. Network requirements reads Private connectivity: the platform reaches the database inside your network.

Choose Test Connection, then Discover Assets. Discovery lists every table the account can read: kb_articles, resolved_tickets and ticket_resolutions. If a table you did not intend to share appears here, fix the account’s grants before going further.

Step 3: Check the data before indexing

Open EDA, choose public.kb_articles as the dataset and select Run Health Check. It profiles every column and summarises the table.

Ours came back with no missing values, no duplicates and no outlier columns, and each of the five columns (article_id, title, category, body, updated_at) is fully populated. Look for two things in your own data. An empty body indexes as noise that can still be retrieved. Duplicate articles rank twice and push other answers out of the results. Fix both in the source, not in the index.

Step 4: Create the vector index

Open Vector DB Builder and choose New Index. Give it a name, choose the connection and the asset, here public.kb_articles, and review the three settings that shape retrieval.

• Embedding model. The model that turns each chunk, and later each question, into a vector. Our deployment embeds with bge-m3, an open multilingual model running on the on-premises model server, so neither the articles nor the questions leave the network. Enter bge-m3:latest so the index records the model that built it.
• Chunk size. The longest piece of text, in characters, that becomes one vector. The default is 500.
• Chunk overlap. How many characters neighbouring chunks share, so a sentence cut at a boundary survives in one of them. The default is 50.

Keep the defaults for a first build. The last lesson in this path measures whether they suit your content. Choose Save.

Step 5: Build it and check the chunks

Select the new index and choose Build, then Refresh until the status reads success. The panel records the chunk size, the overlap, the embedding model, the status and the number of chunks embedded.

Our 30 articles became 36 chunks. Six articles are longer than 500 characters, and each was split in two. Always check the count against what you expect: far more chunks than rows means the chunk size is small for your content, and exactly one per row means nothing was split at all.

Step 6: Search it the way users ask

Open Semantic Search, choose the index, type a question the way a user would and set Top K to 3. Choose Search.

For “I got a new phone and can’t approve sign-in requests”, our index returned:

1. KB-1002#part-1, the first half of Resetting multi-factor authentication, at 0.648.
2. KB-1020, Enrolling a mobile phone, at 0.580.
3. KB-1002#part-2, the second half of the same article, at 0.534.

Read three things in every result. The source ID names the table, the row and the part, so an answer can cite exactly where it came from. The score is similarity to the question, useful for comparing results of one query rather than as a confidence level. And the split: the second half of the MFA article, which carries the rule that resets for administrators and finance staff are P2 and need a second approver, ranked third on its own. A model given only the top result would never see that rule.

In a complete RAG system, the top passages and their source IDs go to a model together with the question, and the model writes an answer that cites them. This path stays with retrieval, because an answer can only be as good as the passages it is given. Next: retrieving from databases and tables, then keeping retrieval inside each user’s permissions, then measuring retrieval quality with the test set from Step 1.