A chatbot that really knows your company

Chatbots have a bad reputation, and for good reason. Most people have met one that after three questions wrote „I don't understand that”, or served up a menu tree that didn't contain what they were looking for. That bot wasn't thinking: it was matching keywords against a pre-written list.

Over the past few years, though, what you can expect from a system like this has changed. The difference isn't that „the technology got smarter”, it's where it takes the answer from.

The short version

• A keyword bot matches pre-written answers. A knowledge-base bot looks the answer up in your company's own documents and composes it from those.
• Quality depends first of all not on the model but on whether your documents are in order.
• A good bot says when it doesn't know the answer, and hands the case over to a human.
• It's worth it when there are a lot of repeating questions. At a few dozen enquiries a month, an answer template is cheaper.
• You have to measure: how many questions it answered on its own, and how often a human was needed. Without that you don't know whether it paid off.

The real difference between the two kinds of bot

The keyword solution

Here someone writes fifty questions and fifty answers in advance. The bot looks at which question the typed text most resembles, and sends the matching answer.

This works as long as the user asks roughly the way the author imagined. As soon as they phrase it differently, or ask two things at once, the system stops. And every new piece of information means extending the list by hand.

The knowledge-base solution

Here the bot doesn't work from pre-written answers but from the company's own material: price lists, contract templates, process descriptions, earlier client replies, product documentation.

When a question arrives, the system first looks up the relevant passages in those documents, then composes the answer from them. The industry calls this method retrieval augmented generation, RAG for short.

The practical difference shows in three things:

• You don't have to anticipate every question. If the information is in one of the documents, the bot can find it.
• It can be updated. If a price changes, you edit the document, not the bot.
• It can cite a source. It can tell you which part of which document it worked from.

What most people underestimate: the state of the documents

This is the point where most chatbot projects are decided, before anything technical happens at all.

The bot can only work from what you give it. If the company's knowledge lives in three people's heads, a Word file from 2019 and a long email thread, there's nothing to answer from. If, on the other hand, there is tidy, up-to-date material, the system becomes usable surprisingly fast.

In practice it usually looks like this:

• Inventory. What exists at all, and where.
• Selection. What is still valid. A wrong answer is often not invention but an old document that is no longer true.
• Tidying. One topic in one place. A price list that appears in ten places and contradicts itself is the biggest source of error.
• Filling the gaps. Whatever exists only in people's heads has to be written down. This is the most work, and it brings the most benefit, because it's valuable even without the bot.

This work can't be skipped, and it isn't worth skipping, because you come out ahead even if the bot never gets built: you'll have an organised knowledge base.

Handling hallucination

Language models occasionally write untrue things with confidence. This can't be eliminated entirely, but in a business setting it can be pushed down to a manageable level.

Tie the answer to a source. If the system may only work from the documents you supply, the chance of invention drops significantly. This is not the same as asking a general-purpose conversational model.

Allow it not to know. This sounds contradictory, but it's the most important setting. A good bot says „I have no information on that” and hands the case over. A bad bot answers at all costs.

Show the source. If the answer comes with a reference to the document it was built from, both the client and the colleague can check it.

Draw a line. There are topics the bot must not answer: custom pricing, contract amendments, legal questions, complaint handling. Those are worth routing to a human from the start.

When is it worth it, and when isn't it?

Introducing a chatbot isn't only a development cost, it's ongoing care too. So it's worth checking whether there's actually something for it to solve.

It's worth it if a lot of similar questions arrive every month. If a meaningful share of the team's time goes on explaining the same thing, that's a measurable loss. It's also worth it if the questions arrive outside working hours, or if internal knowledge is hard to search and colleagues keep asking each other.

It isn't worth it if there are few enquiries. At a few dozen questions a month, a well-written FAQ page and a handful of email templates give more, for less. It's also not worth it if every case is unique, or if the knowledge base doesn't exist and there's no intention of building one.

There's a middle option that rarely gets discussed: the bot for internal use. It doesn't talk to clients, it answers colleagues from the company's own material. The risk is lower, because a colleague notices a wrong answer, and the benefit is often greater.

How do you measure whether it works?

If you don't measure it, you'll only have a feeling about it. Four numbers are enough:

• Resolution rate. What percentage of conversations closed without needing to turn to a human.
• Handover rate. How often it handed the case over, and on what topics. That list shows where the knowledge base is thin.
• Response time and availability. How much sooner the client gets an answer than before.
• Feedback. A simple „was this helpful?” at the bottom of the answer reveals surprisingly much.

The first month's data almost always shows that two or three topics account for the majority of questions. Those are the ones to sort out first.

Data protection: before you upload anything

If the company's documents go into a system, a few questions have to be settled in advance.

Where is the data stored? In a European data centre, or outside it. This matters from a GDPR point of view.

What goes in at all? Clients' personal data typically doesn't belong in the knowledge base. General knowledge is the right content there: prices, processes, terms.

Is it used for training? With business services this can typically be switched off, or is off by default, but it's worth seeing it in the contract.

Who has access, and what do they see? With an internal bot it matters that the bot doesn't tell a colleague something they wouldn't otherwise be authorised to see.

Frequently asked questions

What does a company chatbot cost?

• It has two parts: one-off development and ongoing operation. Most of the development isn't taken up by the technology but by getting the knowledge base in order. For operation you pay for the language model according to how much it's used, plus maintenance of the system.

How long does it take to build?

• If the documents are tidy, a few weeks. If there's no knowledge base, assembling one becomes the larger half of the project, and that depends on the size of the company.

How does the bot know when something changes?

• You update the source document and the system re-reads it. There's no need to change the bot's settings.

What if it gives a client a wrong answer?

• That's why source citation, handover to a human and defining forbidden topics matter. Quotes, contract amendments and complaint handling are not dealt with by a bot.

Does it need our own language model?

• In a small business, almost never. The existing services are good enough and much cheaper. The difference comes from the company's own knowledge base, not from the model.

What's the difference between a chatbot and an AI agent?

• A chatbot answers. An agent also carries out a task: looks up a piece of data, prepares an email, fills in a form. The technological basis is the same, but with an agent the stakes of a mistake are higher, so it's worth tuning it to a narrower task.

In summary

A chatbot is worth as much as the knowledge behind it. The technology is rarely the bottleneck these days; tidy, up-to-date documentation very much is.

If you're wondering whether to start, begin with a simple measurement: collect the incoming questions for a month. If at the end of the list you see that three topics account for half the enquiries, there's something to work with. If every question is different, the bot isn't the right answer yet.