Customer-service chatbot
Your group is the delivery lead for this funded initiative.
The mandate
The board has approved $450K to deploy an AI customer-service chatbot across RetailFlow’s online channels. The case that won funding: customer satisfaction has fallen 78% to 68% and email response times sit at 26 hours. The chatbot is meant to deflect routine queries (order status, returns, sizing) and cut response times dramatically.
What the board thinks it bought: “a chatbot that answers customers, live in a quarter.”
Your job: deliver it without the predictable failures, and define what “good enough to go live” actually means.
Why this one is hard to deliver (not just buy)
- It is customer-facing, so a confident hallucination isn’t an internal embarrassment. It’s a wrong returns policy quoted to a real customer, in writing.
- The data looks ready but isn’t uniform: returns and sizing rules vary by product line and have exceptions that live in people’s heads, not systems.
- “It works in the demo” will be true within week two. That is the trap (Difference 2): the demo handles the easy 80%; the cost is in the angry-customer edge cases.
The data reality (ask Priya)
Conversation/transcript data is good, the best of the four initiatives. Priya’s honest read: “good data foundation, ~6-month implementation, proceed with confidence.” But “good foundation” does not mean “knows your policies.” Grounding the bot in current, correct policy documents (RAG) is where the real work is.
Who to interview, and the tension they bring
- Tom Walsh (Customer Service Manager): his team gets automated first. The evaluator’s-advantage and apprenticeship-pipeline questions land hardest here. Who reviews the bot’s answers, and does that role build or consume his juniors’ judgement?
- Priya Sharma (Data): feasibility and realistic “good enough” thresholds.
- Emma Rodriguez (Managing Director): what was promised to the board, and by when.
Your starting question for “good enough”
What deflection rate, and what maximum acceptable wrong-answer rate, makes this safe to put in front of customers? Where must a human stay in the loop before a response is sent?