An in-house AI sales assistant instead of a paid SaaS widget: 34 : 12 in a blind benchmark
A US outdoor-gear brand ran a paid SaaS chat widget on its storefronts — billed per store, and asking clarifying questions instead of answering them. The brand's Russian site added a hard data-residency constraint on top. I built an in-house RAG chat on Russian LLMs and proved it out with a blind head-to-head against the paid widget.
You pay per store — and the bot asks questions back instead of answering
The brand runs two Shopify storefronts plus a separate Russian site. The storefronts carried a paid SaaS chat widget billed per store — so the price multiplies by the number of storefronts and grows with the business rather than with the value delivered.
The team's complaint wasn't the price, it was the quality. The widget didn't answer on the merits: asked a specific product-fit question, it fired back a clarifying question of its own — and when it did answer, it steered the customer toward whatever was in its own feed rather than toward what actually fit the job.
A separate hard constraint applied to the Russian site: user data must not leave the country, which rules out foreign LLM providers entirely — both as a production backend and as a "temporary solution just for the pilot". That alone eliminates almost the entire market of off-the-shelf AI chat widgets.
So the brief was: an assistant that answers from the real catalogue and the real shipping and warranty terms, runs inside the required data perimeter, and is no worse than the paid widget — with "no worse" demonstrated by numbers, not by vibes.
Own corpus, own gateway, own server
A separate sub-project: a fact corpus, RAG retrieval over it, a FastAPI SSE gateway and an embeddable JS widget. No "model magic" — every answer is assembled from facts you can point at, and quality is verified by a test rather than by intuition.
562 facts: catalogue, FAQ, shipping, warranty + phrasing lifted from real sales conversations
Embeddings over the corpus, selecting the facts relevant to the specific question
FastAPI + SSE, one contract across three LLM backends, IAM tokens, retries, safety valves
Streamed text, then a separate SSE frame carrying product cards with images
Conversations and leads to JSONL, a copy of every Q&A mirrored to a human on Telegram
The corpus comes from the catalogue and from real conversations
562 facts: product specs, FAQ, shipping and warranty terms. Plus — and this mattered most — over 300 real conversations between the sales team and customers. From those I took not just the answers but the actual phrasing of the questions: how people really ask about size, weight, winter use and lead times. A bot trained on marketing copy simply doesn't recognise those questions.
Gateway: one contract, three LLM backends
An SSE API on top of RAG: the answer streams out, so the user reads it as it's generated. Behind a single contract sit three providers — YandexGPT in production, plus GigaChat and DeepSeek as alternatives for benchmarking and cheap iteration. Swapping backends doesn't touch the front end. The unglamorous parts all live in the gateway:
- IAM token refresh and retries that respect
Retry-After - a per-session lock — one user can't fire off five generations in parallel
- a budget circuit-breaker — token spend can't run away under abuse
- PII redaction before logging — phone numbers and emails never reach the conversation files
- anti-abuse via Cloudflare Turnstile
- speech recognition — the question can be dictated instead of typed
Product cards as a separate frame
The text answer goes out first, so reading starts immediately. Right after it, the gateway pushes a separate SSE frame with cards for the products mentioned: image, name, link. The consultation doesn't turn into a wall of text, and there's a direct path from chat to cart. Lead capture rides along: once the conversation gets concrete, the contact is saved and handed to a human.
Quality checked by a test, not by an LLM judge
The first version of the quality check used an LLM judge, and I threw it out. The blind judge systematically missed failures on exactly the category-level questions the sales team had complained about: the answer looked coherent, therefore "fine". I replaced it with a deterministic groundtruth test — for each question it's known which facts must appear in the answer, and the check runs against those. On top of that, 76 pytest tests, including grounding (the answer leans on the corpus, not on the model's imagination) and anti-abuse.
Canary rollout on a live site
The site is built in a site builder and edits go straight to production, so the rollout
was incremental. A backup was taken before touching anything. The snippet was appended to
the end of HEAD
with the previous markup left intact — verified by prefix comparison, not by eyeballing it.
First one home page → check on the live site → then the remaining pages (about sixty of
them). The rollback is written up in three steps and sits next to the instructions.
Data residency by default, not as a compromise
Gateway with SSE streaming, systemd unit on our own server
562 facts + embeddings, product cards on a separate index
Production LLM backend — data stays inside the required perimeter
Alternative backends behind the same contract: benchmarking and cheap iteration
Voice input — the question can be dictated rather than typed
Anti-abuse without shoving a captcha in the customer's face
One snippet, embeds into a site-builder site with no access to sources
Grounding, anti-abuse, deterministic groundtruth instead of an LLM judge
A blind head-to-head against the paid widget
in-house engine / paid widget, with 3 ties
a question back instead of an answer: 21 of 49 for the widget, none for ours
on completeness — 3.98 versus 2.90
The benchmark methodology matters more than the headline number. 49 questions were synthesised from 174 real customer questions and laid out on a grid: 7 personas × 7 funnel stages — from "first time on the site" to "already decided, just checking the warranty". Every question went to both bots, the answers were shuffled, and judging was blind — the judge never knew which answer belonged to whom.
The decisive difference turned out to be willingness to answer, not intelligence. On 21 of the 49 questions the paid widget replied with a question of its own — precisely what the sales team had complained about. The in-house chat answered all 49 on the merits.
The chat is live on the production site. The feedback loop closed within a day: a remark from the business owner in the evening, two corrective commits by morning — the bot stopped inventing which items were popular and started reasoning from actual sales. That's arguably the best signal that the system is put together right: a fix shows up in production the next day, not in a vendor's next release.
A second effect is predictable cost. Instead of a subscription that multiplies by the number of storefronts, there's pay-per-use token spend plus our own server; there are no licences, and the spending ceiling is enforced by the budget circuit-breaker inside the gateway itself.
Where else the same methodology applies
This case isn't about "a chat on a website". It's the general problem of "advice inside a closed domain, where the answer must rest on facts rather than on the model". The same architecture works anywhere there's a proprietary body of knowledge and the cost of a wrong answer exceeds the cost of silence:
- → Stores with complex choices — equipment, machinery, gear: where a purchase needs explaining, not a price filter
- → B2B sites with long cycles — qualifying the lead in chat and handing it to sales with the conversation context attached
- → Internal support assistant — same corpus, different surface: suggesting the answer to an agent instead of making them search the policy docs
- → Any niche with data-residency rules — healthcare, finance, public sector: the in-country perimeter is assembled exactly the same way
- → Replacing any SaaS widget — when the subscription multiplies by locations, storefronts or brands, an in-house engine pays off on arithmetic, not on faith
- An SSE gateway with one contract across several LLM providers — swapping backends never touches the front end
- The corpus recipe: catalogue + policies + real conversations, not just marketing copy
- A deterministic groundtruth test instead of an LLM judge — the judge misses exactly the failures you hired it for
- A blind A/B benchmark on questions synthesised from real ones: a persona × funnel-stage grid
- Canary rollout for the widget: backup → one page → check on the live site → the rest, with a three-step rollback
- Mirroring conversations to a human on Telegram for the first few weeks — so somebody sees what the bot is saying
If your chat widget asks questions back instead of answering — that's fixable
We start with the corpus and the verification method, not with the model: what the bot is required to know, and how we'll prove it knows it. From prototype to go-live on a production site — roughly four weeks.
The 5,000 ₽ audit — with a concrete report and quote
I'll tell you what to deploy in your business first, what the payback looks like, and whether you need AI for the task at all (sometimes you don't).
Or just send your question — I reply within 2 hours