Answers · Entry 003
How do you keep an AI assistant from making things up about your business?
Short answer
Limit it to answering from a record you control, make it show the entry behind every answer, have it say plainly when nothing fits, and keep a person in charge of anything that leaves the building. Those rules make mistakes easy to catch, though some will still get through, so every question is logged for review.
Custom Solutions, Indianapolis · updated
Why general chatbots guess
A general chatbot is built to produce a likely-sounding reply. Ask it about your return policy and it will write a reasonable return policy, whether or not it matches yours.
The rules we build in
- The assistant answers only from your record. It does not fill gaps from the open internet or from general knowledge.
- Every answer shows the entry it came from, with who last changed that entry and when, so a staff member can check it in seconds.
- When no entry fits, the reply says "nothing in the record fits" and stops there.
- Replies to customers are drafts. A person reads, edits and sends them.
- Changes to the record go through a named approver, and the history is kept.
- Every question and answer is logged so you can review what people asked and what they got.
Questions to ask any vendor
Ask them to show you the source behind an answer during the demo. Ask what happens when the system doesn't know, and have them try it with a question it can't answer. Ask who can change the record and whether old versions are kept. Ask where the data is stored and how you get a full copy out.
Related questions
- Can it still be wrong?
- Yes. If an entry in the record is wrong, the answer built from it will be wrong too, and the model can occasionally misread an entry. Showing the source with every answer makes both kinds of mistake quick to check.
- Do customers talk to the assistant directly?
- Not by default. Staff use the chat window. Replies to customers are drafted for a person to check and send, and public answers on your website are written and approved ahead of time.