At your company, the answers exist somewhere — in SharePoint, in old emails, in the wiki, in a binder kept by the colleague who leaves for holiday in two weeks. But people still ask in WhatsApp, because it’s faster. This page describes how your distributed company knowledge becomes a searchable answer source in Teams — with cited original sources, not a hallucinating black box.
Does this sound familiar?
- Employees post subject-matter questions in WhatsApp groups because they know someone there will answer quickly — not because the official documentation is bad, but because it’s impossible to find.
- Onboarding new colleagues consists largely of “Just ask Frank” — and Frank repeats the same explanation for the seventh time because it isn’t written down anywhere central.
- There’s a wiki, a SharePoint, a network-drive folder and a Confluence instance someone once set up — and each of them holds fragments of the truth, some current, some from 2019.
- When someone asks “What’s the process again for complaints over 5,000 euros?”, first someone searches internally (in vain), then the phone calls start, and in the end the answer may land in an email no one can ever find again.
- Sales leadership wants a chatbot that “finally gives answers”. IT doesn’t want yet another data graveyard. Management wants the knowledge to stay in the company when employees leave.
Why now — and not later
- Knowledge is leaving the building. Older colleagues retire, younger ones change employers every 3–5 years on average. Whatever isn’t centrally stored and findable is gone the day they leave.
- AI tools have arrived in the mid-market — mostly uncontrolled. Employees have long been copying company documents into private ChatGPT accounts because they need answers quickly. An orderly internal solution is the answer to a reality that is already happening.
- Microsoft 365 already ships with the building blocks. If you work in Microsoft 365 anyway, the technical distance to a Teams bot searching your SharePoint isn’t huge — but it isn’t zero either. At the latest, the discussion will happen at license renewal or around the question “Do we need Copilot?”. Better to do it properly once.
How it would look at your company
Step 1 — Review and sort the knowledge inventory (weeks 1–2)
Before we index anything, we clarify together: what lives where, what is official, what is outdated, who is allowed to see what — and who is not. This is the uncomfortable phase, but it’s also the most important one. An AI search over poorly organized content delivers bad answers, no matter how good the model is.
Stack: SharePoint Admin Center, Microsoft Graph, permission audit. Result: a list of sources to include in the search, plus a list of sources that need cleaning up first.
Step 2 — Set up the RAG architecture (weeks 3–5)
The pattern is called Retrieval-Augmented Generation: for every question, the system searches your company knowledge, retrieves the most relevant passages, and a language model formulates an answer from them — with a reference to the original source, so the person asking can read the full context. Not “the AI made this up”, but “this answer comes from this specific SharePoint document, as of its last update”.
Stack: Azure AI Search as the index, the Microsoft Graph API for access to SharePoint and OneDrive, Azure OpenAI Service (via Azure AI Foundry) for the language model — all in your own Azure tenant. Your data never leaves your Microsoft environment.
Step 3 — A Teams bot as the entry point (weeks 5–6)
Employees don’t ask in a new app they’d have to install. They ask where they already are — in Teams. We build a bot you can message like any colleague: “What’s the process again for complaints over 5,000 euros?” The answer arrives within two to five seconds, with a link to the original source. If the AI doesn’t have a confident answer, it says so — instead of guessing.
Stack: Microsoft Bot Framework, Teams app manifest, optionally Power Platform for simple integrations.
Step 4 — Respect permissions (weeks 4–6, in parallel)
This is where many AI projects fail: the search may only answer from documents the person asking is actually allowed to see. Anyone without access to the management folder must not get answers drawn from it — not even in summarized form. We set up the search so that it honors your SharePoint permissions rather than bypassing them.
Stack: Microsoft Graph with delegated permissions, Azure AI Search with security trimming.
Step 5 — Trial, feedback, expansion (weeks 6–10)
We start with a trial group of 10–20 people from two or three departments. They use the bot for three to four weeks, give feedback and flag bad answers. Based on that, we adjust the source selection, the prompts and the answer format. Only once the answers are useful in 80 percent of cases do we roll out more broadly.
What to watch out for along the way
- If someone sells you an AI search without first checking your SharePoint permissions — be careful. That is exactly the mistake that leads to “the management bonus” suddenly popping up as a search result for everyone. That’s not a model problem. That’s a permission problem.
- Ask for citations in the answers. A serious internal AI search always shows which source an answer comes from. A solution that just produces text without references is not verifiable — and therefore not trustworthy.
- Clarify where the data ends up. An “on-prem LLM” sounds safe, but is rarely realistic for a mid-market company to operate. Azure OpenAI in your own Azure tenant in an EU region is usually the pragmatic compromise: your data stays within your Microsoft environment, and your content is not used to train third-party models.
- Check whether it really has to be a custom bot — or whether Copilot is enough. Microsoft 365 Copilot can do much of what is described here out of the box. If your knowledge base sits cleanly in SharePoint and your permissions are right, Copilot is often the more honest answer than a custom build. But if you need to integrate specific sources outside M365 (ticketing systems, ERP, industry wikis), a custom bot becomes interesting.
What realistically changes afterwards
- Employees find answers to recurring subject-matter questions in Teams instead of in WhatsApp groups — or from colleagues who are already stretched thin.
- Onboarding gets easier: new hires can ask questions without feeling “stupid”, and get answers with references to the original source — which they can then read for themselves.
- Knowledge that previously lived only in individual heads gets documented more and more, because it becomes visible where the AI finds no answer — and that is exactly where the incentive to write something down emerges.
- The uncontrolled use of private ChatGPT accounts with company data declines, because there is a more convenient and legitimate alternative.
- Management gets an honest view of which topics are asked about most often — and with it an indicator of where the process or documentation gaps are.
What you contribute
- Access: an admin in your Azure and Microsoft 365 tenant who grants us targeted permissions. We work with service principals, not with permanent personal admin accounts.
- Stakeholder time: one experienced subject-matter person per relevant department who can judge which sources are authoritative and which are outdated — typically 2–3 hours per department during the review phase, then occasionally for trial feedback.
- Data protection officer and works council: an AI solution that can analyze employee questions needs a clean agreement. We provide the technical description; you take it through your codetermination and data protection processes.
- Trial group: 10–20 people willing to give honest feedback for three to four weeks — beyond just reporting “works” or “doesn’t work”.
Risks & when it does NOT fit
- If your SharePoint has grown wild and permissions are unclear, this is the wrong first step. Clean up first, then make it searchable. Otherwise you build a fast search over chaotic data — and the chaos becomes findable faster, not smaller.
- If the expectation is that AI will replace process knowledge that was never written down. A RAG search can only find what exists. If your relevant knowledge lives only in people’s heads, the first task is documentation, not AI.
- If the data protection framework can’t be settled. In sectors with special confidentiality requirements (tax firms, medical practice groups, critical infrastructure), the question of what an AI may see, and in which region, is not trivial. That belongs settled up front, not during the trial.
- If you plan to roll out Copilot soon anyway. Then the honest recommendation is sometimes: Copilot first, a custom build only once Copilot demonstrably falls short. We don’t recommend Copilot reflexively — but we don’t reflexively recommend against it either.
How the conversation starts
A 30-minute initial conversation, free of charge, by video or phone. What we clarify: where does your knowledge predominantly live today (SharePoint, wiki, tickets, email)? Which questions get asked over and over at your company? Have you already tried Copilot, or deliberately not? What is your data protection and codetermination situation? From that, we work out whether a custom RAG build, Copilot or a smaller solution is the right path.
Remote response is immediate during service hours. An initial conversation can typically be arranged within 3–5 working days; we confirm the next available slot as soon as you get in touch.
Frequently asked questions
Won’t the AI still hallucinate? Hallucinations mainly arise when a language model answers “from memory”, without a source. In the RAG pattern, the answer is formulated from your specific documents, with a reference to the source. Done well, the AI says “I can’t find anything on this in your sources” instead of guessing. It can never be ruled out entirely, but the risk drops considerably.
Will Microsoft or OpenAI see our company data? When you use Azure OpenAI in your own Azure tenant, the Microsoft enterprise terms apply: your data is not used to train models, and it stays in the Azure region you choose. That is not the same as an employee’s private ChatGPT account — and that is exactly why the in-house variant is much cleaner to set up from a data protection perspective.
What does ongoing operation cost? The bot itself isn’t the cost driver — the model calls and the search index in Azure are. For an 80-person company with moderate usage, ongoing Azure consumption typically runs between a few hundred and a little over a thousand euros per month — depending heavily on model choice and usage intensity. Before the rollout, we show you how to monitor and cap this yourself.
Can we extend this later to other sources — tickets, ERP, CRM? Yes, and that’s one of the reasons to choose an in-house build over Copilot alone. Via connectors or custom adapters, sources outside SharePoint can be integrated as well. But that only makes sense once the first area runs stably — otherwise you’re adding complexity before you’ve seen the first value.