Start with the Outcome
Nobody wants RAG for its own sake. People want outcomes: an employee gets the right answer about the leave policy, a service rep pulls up everything on a customer's account, a compliance team finds every contract with a given clause. RAG is one way to get there, and most enterprise RAG projects stall on the way.
The model is rarely the problem. Most of the time goes to compliance, privacy and security reviews, and to tuning a chunking strategy for each source. Both come back to the same gap: nobody can say what is in the files.
The missing piece is classification: knowing what each file is, what it contains and who may read it, before any AI system touches it. This post explains why RAG projects stall without it, and how agents use it to get to the outcome.
A Vector Database is a Copy of Your Data Without Its Permissions
To build RAG, you split documents into chunks, turn each chunk into an embedding, and store them in a vector database. That database is a copy of your data. The source files had access controls, retention rules and owners. The chunks have none of that unless your pipeline rebuilds it.
In principle, each chunk inherits its parent file's permissions. In practice, this is where projects get stuck. Access lists change every day as people join, move and leave. Groups sit inside other groups. A file shared with one person yesterday is shared with a whole department today. The pipeline has to carry every change to every chunk, and a delete at the source has to reach every chunk and every backup.
The embedding does not hide the text either. In a 2023 study, researchers who could query an embedding model reconstructed 92% of 32-token inputs exactly from their embeddings, and recovered 89% of full names from embedded clinical notes (Morris et al., 2023). OWASP lists vector and embedding weaknesses among its top 10 risks for LLM applications, and its first recommendation is "permission-aware vector and embedding stores" (OWASP LLM08:2025).
So Only the Safest Files Get In
Faced with that, security, privacy and compliance teams do the reasonable thing. They approve only the files nobody minds copying. The RAG system ends up built on the least sensitive content in the company: HR policies, the employee handbook, IT how-to guides.
Those assistants work, but the outcomes are small. You can answer a question about the leave policy, but not pull up everything you hold on a customer. The contracts, engineering documents, customer records and case files where the value is stay out. Nobody can say exactly what is in them, and nobody can show that the copy is as well protected as the original, so reviewers default to no.
It is the same problem we described in Sovereign AI starts inside your files: the most valuable data is the data AI cannot reach.
Six to Nine Months Later
Even with approved data, a RAG system takes months to get right. Every source needs its own chunking strategy, and every strategy needs testing against the questions people actually ask. In our experience the whole cycle runs six to nine months, and most of that time goes to reviews and tuning rather than to the model.
Each chunking strategy trades one problem for another. Fixed-size chunks cut tables and sentences in half. Recursive splitting keeps sentences together but loses the heading a passage sits under. Structure-aware chunking needs a parser for every format. Semantic chunking needs another embedding pass and a threshold to tune. Each choice changes what the retriever finds, so each source gets tuned on its own and tuned again when the documents change.

At the end, you have a system that works well for that one data set. Point it at a new source and much of the work starts over.
How a Student Finds What They Need
Think about how a child learns to write a first school report. Nobody cuts the library into 60,000 pieces and asks the child to sort them by how similar they sound to the topic. We teach the child to use a table of contents, an index, the library catalog and the Dewey Decimal System. They find the right book, turn to the right chapter and read it.
That is how people do research, because it is the efficient way to find information. Brute-force similarity search over chunks works the other way around. It compares the question with every fragment and hopes the right ones rise to the top, without the structure that tells a reader what a document is, where it belongs and who may read it.
AI agents can work like the student. A capable model with the right tools will look up an index, open the right file, go to the relevant section and follow its references. What it needs from you is good tools and a well-organized library, not a pile of chunks.
Classification is the Missing Piece
Classification gives your data the structure the student relies on. It records what each file is, what it contains, how sensitive it is and who may read it, and it extracts the fields people actually look for, such as customer, counterparty, program or policy version. With that catalog, an agent goes straight to the right file instead of searching every chunk.
The student's skill gets used twice. DataXray's agentic classification uses it to build the catalog: an AI agent reads each long, messy file the way an analyst does. It checks the table of contents, searches inside the file, goes to the relevant section and follows references, then records what it found as labels and fields. Agents answering questions use the same skill to reach the outcome: they search the catalog, open the right file and go to the right section. Both run today, inside your environment.
| Outcome you want | What classification gives the agent |
|---|---|
| Answer questions about HR policy | The current approved version of each policy, with drafts and superseded copies set aside |
| Look up everything on a customer | Every file tied to that customer, labeled by type and sensitivity, and readable only by people allowed to see it |
| Find every contract with a given clause | Contracts labeled by type and counterparty, with the relevant clauses extracted as fields |
| Answer a records request or a regulator | Files found by subject and date, with personal data flagged before anything is released |
The same catalog serves all of these outcomes, so a new use case starts from what is already classified instead of from a new pipeline.
Classification also changes the review. Once every file carries labels, security can approve a policy by label instead of reviewing each source. For example, customer records might be allowed for the service agent and redacted for everyone else, while export-controlled designs are blocked for every agent. New use cases inherit that policy instead of starting a new review.
Putting It to Work with the AI Platform
The DataXray AI Platform is where agents put that classification to work on your own data, inside your own environment. DataXray reads and classifies every file where it sits, which builds the catalog: what each file is, what it contains and how sensitive it is. The AI Platform gives agents the tools to use that catalog, and checks every step against your policy.
- Search and read the catalog. Agents search and read the files DataXray has classified with the AI Platform's built-in tools, so they start from what each file is and what it holds.
- Tools built for the use case. Each agent gets an approved set of tools. Depending on the job, that might be a connection to a database, a search tool that finds the documents acting as a table of contents or index for a collection, or a small language model fine-tuned on your own data. We build these with customers and add them to the platform.
- Policy on every step. Every model call, tool call and document read is checked against your policy. Access can be limited to each user's own permissions in DataXray, and each label can be set to allow, redact or block.
- Your models, in your environment. Agents can use Anthropic, AWS Bedrock, OpenAI or Azure OpenAI, or your own models through vLLM or Ollama, and policy can route a run to a private model. The platform runs next to DataXray in your environment, including air-gapped networks.
- A record of every run. Policy decisions, tool calls, document reads, usage, cost and latency are kept as a replayable timeline.
This also removes the copy that held up the reviews. DataXray's index holds metadata about each file, including its labels and entitlements, not a copy of its content. Agents read each file through DataXray from where it sits, with permissions and policy checked at each read. See how it works on the AI Platform page.
Chunking and vector search still have a place, for example semantic search over a narrow, uniform set of documents. They work best as one tool among several, used on data that has already been classified and cleared.
Questions to Ask Before Your Next RAG Project
Before you start another chunking pipeline, ask whether you need one. These questions show where the months will go, and whether an agent with the right tools could answer the same questions without a new copy of your data.
- What outcome is this for, and how will you measure it?
- Which files will this system copy, and who has to approve each source?
- How will each chunk inherit its file's permissions, and stay current when they change?
- What happens to the chunks when a file is deleted, moved or relabeled?
- How many chunking strategies will you need, and who will maintain them?
- Could an agent reach the same outcome with a catalog search, an index lookup or a database query instead?
- Can you show which files an answer came from, and that the person asking was allowed to see them?
If those answers are hard, look at how the DataXray AI Platform gives agents governed tools over your own data, or book a demo.
Frequently asked questions
Why is RAG so hard to get into production?
Mostly because of the data, not the model. A vector database is a copy of your files without their permissions, so each source needs security, privacy and compliance review, and each source needs its own chunking strategy and tuning.
What is a chunking strategy in RAG?
A chunking strategy is the rule a RAG pipeline uses to split documents into pieces for embedding and retrieval, such as fixed-size, recursive, structure-aware or semantic chunking. It decides which text is retrieved together and which context is lost.
Do chunks inherit the permissions of the source file?
Not by default. The pipeline has to copy each file's access list onto its chunks and keep it current as permissions change, files move and files are deleted.
What is agentic classification?
Agentic classification gives an AI agent tools to read files the way an analyst does: it checks a document's table of contents, searches inside it, goes to the relevant section and follows references. What it finds is recorded as labels and fields, which build the catalog other agents use.
Why is classification the missing piece for AI on enterprise data?
Because an AI system can only use what it can find and is allowed to read. Classification records what each file is, what it contains and who may read it, so agents go straight to the right file with permissions checked at every read. Each new use case then builds on the same catalog instead of a new copy of the data.