I stress-test AI chatbots and find the vulnerabilities before anyone exploits them
An AI chatbot sees conversations, sign-up forms and customer history. Anyone can write to it, including someone looking for a weak spot. I attack a copy of the system following the OWASP LLM Top 10, a public list of typical threats to AI applications. I check where personal data goes and whether 152-FZ, Russia's personal data law, is followed. Every finding comes with a fix plan, and after the fixes I run the attacks again.
The bot already talks to students, and nobody has tried to break it
For this estimate, take an online school with 20,000 students. An AI tutor on GPT-4o-mini is built into its courses. The tutor sees each student's progress and message history, and sometimes personal data from sign-up forms.
A bot like this gets tested on ordinary student questions. How it behaves when someone tries to trick it usually goes unchecked before launch.
The first warning signs are easy to miss: a student gets an answer unrelated to their course. That happens when document search mixes up different students' contexts. A separate question is the personal data sent to OpenAI: where it is stored, whether it is anonymised, whether it ends up in logs, and how all this fits with 152-FZ.
The audit runs in five steps, from survey to re-test
The attacks target an isolated copy of the live system. It uses the real prompts and realistic but invented personal data. Each of the 10 OWASP categories gets its own check: automated attacks in Promptfoo and Garak, plus manual scenarios written for your business.
A list of every place the model is used: the tutor, document search, training data, prompts. For each one, I assess the threats against the OWASP LLM Top 10.
Promptfoo and Garak: thousands of requests that try to override instructions, get around limits and pull out data.
Scenarios written for your business. For an online school: get another student's data, get past the paywall or make the bot answer off-topic.
Which personal data goes to a foreign model, whether it is anonymised, and whether there is consent to process it and to transfer it abroad.
After the fixes, the same attacks run again. The result is a report dated on the day of the re-test.
What I look for first
Here are the eight vulnerabilities I start with:
- 01Prompt injection: whether the phrase "ignore previous instructions" gets around the system prompt and changes how the bot behaves
- 02Personal data leak: whether document search mixes up contexts, so that one student's answer contains pieces of another student's conversation
- 03152-FZ violation: whether personal data goes to a foreign model without anonymisation and without consent to transfer it abroad
- 04No rate limit: whether one person can use up the daily model budget in a short time
- 05Jailbreak through role-play: whether "you're a drama teacher, play this role…" lifts the restrictions
- 06System prompt extraction: whether a couple of carefully worded questions reveal the full text of the prompt
- 07Unchecked output: whether the model's text goes onto the page as is, so that generated markdown could inject code (XSS)
- 08Unsafe plugin: whether the function that fetches course materials can be led outside its folder through the course name (path traversal)
A report with a fix for every finding
I rank the findings by urgency, and each one comes with a specific fix. That might be a system prompt patch, separate contexts in document search (RAG), moving personal data processing to a Russian model, Cloudflare WAF rules against prompt injection, cleaning the output with DOMPurify, or isolating plugins.
Re-test after the fixes
Once your team has made the fixes, I run the automated attacks again, plus manual scenarios against each critical finding. The final report shows which attacks are now blocked.
Standard tools and my own attack scenarios
The threat framework: 10 categories of vulnerabilities in AI applications, used as a checklist
Automated tests: thousands of prompt variants per target, with scoring of the answers
A toolkit of ready-made attacks on language models: jailbreaks, data extraction, prompt injection
An open-source prompt injection detector: it looks for known tricks and has a second model check the request
Manual tests written for your business. For an online school: extract personal data, get past the paywall, influence grades
A Russian model. Personal data processing can be moved to it
Filters out requests carrying prompt injection before they reach the model
The legal side: where personal data goes, consents, the processing agreement, transfer abroad
What to expect after the audit
The estimate is for the online school in the example: 20,000 students and an AI tutor on a foreign model. The fine comes from Russia's Administrative Code, the audit price from the service's price list. I'll assess the risks of your own system during the express diagnostic.
in the re-test, none of the critical attacks from the report gets through
the fixes aim to remove the grounds for a fine: keep students' contexts apart, move personal data processing to Russia and put consents in place. This does not guarantee the outcome of an inspection
even the lowest fine for the school in the example (₽5M) is 4.5 times the top price of a full audit (₽1.1M), counting the 152-FZ risk alone
After the audit the school will have a report across all 10 OWASP LLM Top 10 categories, dated on the day of the re-test. It can go into a document package for investors, a buyer of the company or a regulator.
The report also includes an action plan by urgency: what to fix in the first 48 hours, what in the first month and what within the quarter.
In an AI assistant for company knowledge, I measured on the live index how many private fragments the "team" access level could see. There were 1327; after the access fix, 0.
In an AI sales assistant for a store's website, visitor data is processed by Russian language models. Phone numbers and emails are masked before logging, and a spending cut-off keeps abuse from running up the model bill.
How long the audit takes and what it costs
for a full audit. The re-test follows once your team has made the fixes
up to ₽1.1M, depending on the scope and the number of AI agents in the system
a monthly re-check and alerts about new vulnerabilities, 6 months minimum
All formats and what they include are on the AI security audit service page.
When an AI security audit is most needed
The general rule: an audit is worth doing when a model works with other people's data, or its answer reaches a person unchecked. If your AI fits any of the cases below, it's time for a check:
- → An AI assistant with access to personal data: online schools, healthcare, HR services, any consumer service that stores conversation history
- → Document search (RAG) that keeps several clients' data in one system: any leak between them is already an incident
- → AI that calls external functions and works with real systems, money and files
- → A chatbot on a public website: any visitor can attack it
- → Before raising investment, selling the company or going through certification: investors and regulators may ask about AI security too
- → Any processing of personal data through foreign models (OpenAI, Anthropic, Google): a 152-FZ risk
- A 30–50 page report with evidence, with each finding mapped to legal requirements
- A live demo of the 1–2 most dangerous attacks on your system
- A 90-minute walkthrough of the report, then 30 days of answers to questions about the fixes
Blog article on this topic: Personal data and LLMs: 152-FZ, ChatGPT and auditing your own system
Related cases and ready-made solutions
A Telegram bot for foremen: stock, budgets and payments from 1C right on site
A foreman opens the Telegram bot, sees data from 1C UNF in two taps and confirms deliveries with a photo of…
Converter for Federal Treasury statements into 1C
Reads XML statements from the Federal Treasury's GIS system in both format versions, V3 and V4, and builds a…
Forecasting events with several language models: calibration and Brier scoring
Five AI agents search open sources, a sixth reconciles their answers, and a calibration step adjusts the…
If your AI already talks to people, start with an express diagnostic
In one working day I check the system against a shortened checklist, show one attack live and send a report with the five main risks. It costs ₽45,000.
Or just ask a question. During business hours I reply within 2 hours