Which internal assistant for company documents do you need?
Start with the task, not the technology. These options meet different needs and can work alongside one another.
| Option | When it helps | What to check |
|---|---|---|
| Document chat | For asking questions about or summarizing individual files. | Who can upload files and what the tool does with them. |
| Search tool with excerpts | For finding text and reading the source directly. | Permissions, document currency, and ease of finding documents. |
| RAG assistant | For answering common questions using a shared document collection. | Source support, permissions, retrieval quality, and maintenance. |
A document chat may be enough for a one-off analysis. A shared document collection needs access controls and people responsible for updates.
How do you know if an answer is supported?
The system should show the document and excerpt it used. The person should be able to open the source and check that it's current.
A RAG assistant finds content related to a question and gives it to the model to write an answer. Microsoft's RAG documentation explains that retrieval can combine keyword and semantic searches.
A citation helps you check an answer, but it doesn't prove the answer is correct. The assistant might find an outdated excerpt, mix documents, or add a condition the source doesn't contain. When information is missing, it should say so and direct the question to the responsible team.
Illustrative example, not a client: someone in accounts receivable asks how to request a certificate confirming that a balance has been paid. The assistant searches the current procedure that person has permission to open. It then explains the steps and shows the supporting excerpt.
If the procedure doesn't say who approves the request, the assistant should acknowledge that gap. If it finds conflicting versions, it should flag the conflict instead of choosing one without explanation. Accounts receivable reviews the answer, and the document owner corrects the source.
Keep a human reviewer involved when decisions require comparing full documents or determining compliance. The Copilot Studio RAG guidance says this approach works better for retrieving specific facts than for those tasks.
How do you test the assistant before adopting it?
A pilot helps you check whether the assistant is more useful than your current search method. Showing a few good answers isn't enough.
- Gather real questions. Ask people who use the manuals for common and difficult questions. Include similar documents, questions the sources don't answer, and questions each department shouldn't be able to answer.
- Prepare expected answers. Someone familiar with the process records the expected answer and the current document that supports it. If there's no answer, mark the case so you can check whether the assistant recognizes the gap. Microsoft's guide to evaluating RAG applications recommends representative queries and, when possible, expected answers.
- Measure the current search method. Use the same questions with the search tool or folders the team already uses. Record whether people find the correct answer, how long it takes, and whether they can verify the source. Keep those results as a baseline.
- Test answers and permissions. Repeat the questions with the assistant and different user profiles. Record correct answers, unsupported answers, and cases where it acknowledges it doesn't know. Check that someone in sales can't obtain restricted information from human resources.
- Fix problems and measure again. Check whether each failure comes from a document, retrieval, or the written answer. Make the necessary updates and repeat the same tests. Compare the results with the previous search method and decide whether the improvement justifies keeping the assistant.
What controls do you need to protect and maintain documents?
Before connecting a folder, assign owners and check that the tool can enforce existing restrictions.
- User permissions: the assistant must filter documents before retrieving content. Azure AI Search documents access controls for individual documents and how permissions carry over when content is split into chunks. Verify how your chosen tool handles this.
- Current versions: assign an owner to each manual. That person should remove or mark older versions and check what appears when instructions conflict.
- Untrusted content: limit who can change sources and review suspicious changes. OWASP describes prompt injection attacks in which a manipulated document changes the assistant's response.
- Data and contracts: check where files are stored and processed, how long they're retained, and who can access them. For third-party manuals, check the contract and jurisdiction before adding them.
- Unsupported answers: define how to report them and who fixes them. If the assistant can't find an authorized, current source, it shouldn't present an answer as fact.
Should you use an existing tool or build an assistant?
The choice depends on the sources you need to connect, the permissions you must preserve, and the control your process requires.
Use what you already have if your search or document tool can find current excerpts with the right permissions. For one-off questions, a document chat may be enough, as long as the company authorizes uploading those files.
Try an automation platform if you need to connect repositories and organize a simple workflow, and the platform can enforce and verify permissions. Also check how it will update documents and what happens when a connection fails.
Consider custom development if your sources, permissions, or evaluation rules don't fit the other options. You can explore how RAG systems work before deciding which components you need.
Cost and timeline depend on the number and quality of documents, the connections, and the complexity of permissions. Testing, model usage, and who maintains the sources after the pilot also affect them.
You can't measure the system once and leave it alone. Documents change, and so do questions.
