An in-house AI sales assistant instead of a paid SaaS widget: 34 : 12 in a blind benchmark
A US outdoor-gear brand ran a paid SaaS chat widget on its storefronts — billed per store, and asking clarifying questions instead of answering them. The brand's Russian site added a hard data-residency constraint on top. I built an in-house chat that answers only from the brand's own catalogue and documents — retrieval-augmented generation (RAG) — running on Russian large language models, and proved it out with a blind head-to-head against the paid widget.
You pay per store — and the bot asks questions back instead of answering
The brand runs two Shopify storefronts plus a separate Russian site. The storefronts carried a paid SaaS chat widget billed per store — so the price multiplies by the number of storefronts and grows with the business.
The team's complaint was about quality. The widget didn't answer on the merits: asked a specific product-fit question, it fired back a clarifying question of its own — and when it did answer, it steered the customer toward whatever was in its own feed, past the job the customer had described.
A separate hard constraint applied to the Russian site: user data must not leave the country, which rules out foreign LLM providers entirely — both as a production backend and as a "temporary solution just for the pilot". That alone eliminates almost the entire market of off-the-shelf AI chat widgets.
So the brief was: an assistant that answers from the real catalogue and the real shipping and warranty terms, runs inside the required data perimeter, and is no worse than the paid widget — with "no worse" demonstrated by numbers.
Own corpus, own gateway, own server
A separate sub-project: a fact corpus, RAG retrieval over it, a FastAPI SSE gateway and an embeddable JS widget. Every answer is assembled from facts you can point at, and quality is verified by a test against a known groundtruth.
562 facts: catalogue, FAQ, shipping, warranty + phrasing lifted from real sales conversations
Embeddings over the corpus, selecting the facts relevant to the specific question
FastAPI + SSE, one contract across three LLM backends, IAM tokens, retries, safety valves
Streamed text, then a separate SSE frame carrying product cards with images
Conversations and leads to JSONL, a copy of every Q&A mirrored to a human on Telegram
The corpus comes from the catalogue and from real conversations
562 facts: product specs, FAQ, shipping and warranty terms. Plus — and this mattered most — 335 real conversations between the sales team and customers, 131 of them about stock. From those I took the answers along with the actual phrasing of the questions: how people really ask about size, weight, winter use and lead times. A bot trained on marketing copy simply doesn't recognise those questions.
Gateway: one contract, three LLM backends
An SSE API on top of RAG: the answer streams out, so the user reads it as it's generated. Behind a single contract sit three providers — YandexGPT in production, plus GigaChat and DeepSeek as alternatives for benchmarking and cheap iteration. Swapping backends doesn't touch the front end. The unglamorous parts all live in the gateway:
- IAM token refresh and retries that respect
Retry-After - a per-session lock — one user can't fire off five generations in parallel
- a budget circuit-breaker — token spend can't run away under abuse
- PII redaction before logging — phone numbers and emails never reach the conversation files
- anti-abuse via Cloudflare Turnstile
- speech recognition — the question can simply be dictated
Product cards as a separate frame
The text answer goes out first, so reading starts immediately. Right after it, the gateway pushes a separate SSE frame with cards for the products mentioned: image, name, link. The consultation stays readable, and there's a direct path from chat to cart. Lead capture rides along: once the conversation gets concrete, the contact is saved and handed to a human.
A catalogue that carries its photos
For a while the cards under the bot's answer showed a placeholder letter where the photo belonged. Two causes overlapped: the daily catalogue refresh overwrote the file with records that had no image field, and the static URL map had gone stale once the site's links changed. The refresh now pulls the image straight out of the same store API response — the first item in the gallery — and the host is checked by the same guard rail as the links. Every run writes a "with photo: N of M" counter to the log, and an index consistency test holds the fix in place.
The lead reaches sales with the conversation inside it
A manager calls back on the lead itself, so the conversation that produced it has to travel inside the lead. It used to live only in a rotating log, and by the time a real lead with a phone number arrived, the conversation behind it was already gone. Now the lead carries the whole session: dialogue and lead are joined on the session id, the report marks the dialogue with the name the customer left, and truncation is stated out loud — "showing 2 of 5 turns, 3 more didn't fit".
Quality checked by a test
The first version of the quality check used an LLM judge, and I threw it out. The blind judge systematically missed failures on exactly the category-level questions the sales team had complained about: the answer looked coherent, therefore "fine". I replaced it with a deterministic groundtruth test — for each question it's known which facts must appear in the answer, and the check runs against those. On top of that, 134 pytest tests, including grounding (the answer leans on the corpus, with every claim traceable to it) and anti-abuse.
The knowledge-gap detector stopped filing correct answers as "evasive". Shipping cost and the instalment rate are figures we're forbidden to quote in chat — a manager works them out — so those answers follow our own policy, and the old rule list would have had us repairing something that already worked. "No data" and "not specified" now count as a real hole in the corpus, while "a manager will calculate it" is checked against the topic of the question. The "evasive" section left the daily report altogether; "weight not specified" still counts as a hole.
The tests themselves needed work too. Seven of them failed with a type error and therefore verified nothing: the model stub still took two arguments after production grew a third — token spend accounting. That left the captcha, the daily budget and the replay protection uncovered. The same pass showed a test run eating 810 of the 1,000 daily calls meant for site visitors; local calls are now exempt from the shared ceiling.
Deterministic guards on top of the model
The three weeks after go-live went into answer accuracy and data integrity. The store sent its error reports as text and as voice notes — the voice notes are transcribed automatically and land in the same queue. Two of the errors would have cost a customer money. Asked about an inflatable tent, the bot led with a weak 2 PSI pump that will never inflate it. And it stated confidently that the clear end wall ships in the box, confusing it with the windows made of the same material a line above in the spec.
Both are now closed in code. The weak pump has dropped out of tent answers entirely; in the general pump list it carries a note about what it is for. The end wall got a lookup table covering the three models we hold data for, and everything else gets one fixed answer: "no data on file, a manager will confirm".
The takeaway worth carrying into the next project: editing the knowledge base alone did nothing here. The model kept inferring "included" from the neighbouring spec line, just as it had earlier with the stove safety rules. Where a wrong answer costs the customer money, the answer is settled in code.
The index knows exactly what was put into it
Six corpus files written specifically against the store's complaints — stock and dispatch, consultation manner, what's in the box, compatibility, use and care, delivery and payment — were missing from the allow-list of sources, and the builder skipped them silently. However many times the index was rebuilt, the bot knew nothing from those six files. The allow-list now includes them.
An honest limitation sits next to it: the index lives outside the repository, and a rebuild takes a person and a cloud key. Corpus edits reach production once that rebuild has run.
Canary rollout on a live site
The site is built in a site builder and edits go straight to production, so the rollout
was incremental. A backup was taken before touching anything. The snippet was appended to
the end of HEAD
with the previous markup left intact — verified by prefix comparison.
First one home page → check on the live site → then the remaining pages (about sixty of
them). The rollback is written up in three steps and sits next to the instructions.
Data residency by default
Gateway with SSE streaming, systemd unit on our own server
562 facts + embeddings, product cards on a separate index
Production LLM backend — data stays inside the required perimeter
Alternative backends behind the same contract: benchmarking and cheap iteration
Voice input — the question can simply be dictated
Anti-abuse without shoving a captcha in the customer's face
One snippet, embeds into a site-builder site with no access to sources
Every answer traced end to end: retrieval, model call, token spend
Services and timers on our own server, with alerts when one goes down
Grounding, anti-abuse, deterministic groundtruth instead of an LLM judge
A blind head-to-head against the paid widget
in-house engine / paid widget, with 3 ties
a question back instead of an answer: 21 of 49 for the widget, none for ours
on completeness — 3.98 versus 2.90
The benchmark methodology matters more than the headline number. 49 questions were synthesised from 174 real customer questions and laid out on a grid: 7 personas × 7 funnel stages — from "first time on the site" to "already decided, just checking the warranty". Every question went to both bots, the answers were shuffled, and judging was blind — the judge never knew which answer belonged to whom.
The decisive difference turned out to be willingness to answer. On 21 of the 49 questions the paid widget replied with a question of its own — precisely what the sales team had complained about. The in-house chat answered all 49 on the merits.
The chat is live on the production site. The feedback loop closed within a day: a remark from the business owner in the evening, two corrective commits by morning — the bot stopped inventing which items were popular and started reasoning from actual sales. That's arguably the best signal that the system is put together right: a fix shows up in production the next day.
A second effect is predictable cost. Instead of a subscription that multiplies by the number of storefronts, there's pay-per-use token spend plus our own server; there are no licences, and the spending ceiling is enforced by the budget circuit-breaker inside the gateway itself.
Where else the same methodology applies
Behind this case sits a standard problem: "advice inside a closed domain, where the answer is obliged to rest on facts". The same architecture works anywhere there's a proprietary body of knowledge and the cost of a wrong answer exceeds the cost of silence:
- → Stores with complex choices — equipment, machinery, gear: where a purchase needs explaining
- → B2B sites with long cycles — qualifying the lead in chat and handing it to sales with the conversation context attached
- → Internal support assistant — same corpus, different surface: suggesting the answer to an agent instead of making them search the policy docs
- → Any niche with data-residency rules — healthcare, finance, public sector: the in-country perimeter is assembled exactly the same way
- → Replacing any SaaS widget — when the subscription multiplies by locations, storefronts or brands, an in-house engine pays off on arithmetic
- An SSE gateway with one contract across several LLM providers — swapping backends never touches the front end
- The corpus recipe: catalogue + policies + real conversations
- A deterministic groundtruth test instead of an LLM judge — the judge misses exactly the failures you hired it for
- A blind A/B benchmark on questions synthesised from real ones: a persona × funnel-stage grid
- Canary rollout for the widget: backup → one page → check on the live site → the rest, with a three-step rollback
- Mirroring conversations to a human on Telegram for the first few weeks — so somebody sees what the bot is saying
If your chat widget asks questions back instead of answering — that's fixable
We start with the corpus and the verification method: what the bot is required to know, and how we'll prove it knows it. From prototype to go-live on a production site — four weeks, followed by ongoing work on reliability and answer accuracy.
Deep dive:Booking chatbot: why it asks instead of answering, and how to measure it
Related cases
Voice agent Vapi + YClients: booking without an admin
Picks up calls 24/7, books a slot in YClients, sends SMS confirmation. 14 scenarios; when the model isn't…
Triage classifier for inbound + booking into IDENT
Voice + chat with prioritization: acute pain / scheduled / prophylactic. FZ-152: data in RU, GigaChat without…
Auto-prompt for Ya.Maps / 2GIS reviews + AI replies
A workflow fires when a work order is closed in 1C. 5★ → link to Ya.Maps, ≤3★ → form to owner. AI drafts…
The 9,900 ₽ audit — with a concrete report and quote
I'll tell you what to deploy in your business first, what the payback looks like, and whether you need AI for the task at all (sometimes you don't).
Or just send your question — I reply within 2 hours