From speech to a finished ad: transcription, translation, a new voice-over and lip-sync
One video ad turns into versions in 5 or more languages for different countries. Whisper transcribes the speech. GPT-4o translates the script and rewrites it for the market and the buyer. ElevenLabs recreates the speaker's voice with the right accent and reads the new script. The wav2lip model then matches the lip movement to the new soundtrack. One version used to take 4–6 hours of manual work; without lip-sync it now takes about 30 minutes.
Ads are needed in bulk, and one version made by hand takes half a day
In online advertising there are ad intelligence services (AdHeart, AdSpy, AdsLibrary, Anstrex) that collect other companies' ads and show thousands of videos that are working right now. Only the ideas can be borrowed from them. The narrator's voice, the background, the actor's face and the footage itself belong to their owners, so all of that has to be made from scratch.
It used to be done by hand. I'd watch someone else's video, transcribe the speech and give the text to a copywriter to rewrite for the offer I was advertising. Then I'd hire a voice actor on Fiverr, wait 1–3 days, edit in Final Cut and start over. That came to 4–6 hours of work per version, and that's without lip-sync.
With $2–5k a day going into A/B tests, how fast you can produce new videos matters a lot. When 8–10 ad sets run at once and each needs 20–30 fresh videos a week, manual work can't keep up.
Four steps: transcribe the speech, rewrite the script, voice it, match the lips
Whisper transcribes the original video
whisper-large-v3 on my own server, or OpenAI's whisper-1 through the API. It handles Russian, English, Spanish, Balkan and Turkic languages. The output is SRT subtitles with timecodes, so you can see which second each phrase falls on. Other companies' videos are transcribed the same way when I need to study the market.
GPT translates the script and rewrites it for the right buyer
GPT-4o does three jobs in one pass. First, it splits the transcript into blocks: the hook (the first 3 seconds), the buyer's problem, the proof and the call to action. Second, it translates into the target language (Russian, English, Serbian, Polish, Turkish) with local nuance, the way a native speaker would say it. Third, it rewrites the script for the offer, the buyer and the country, keeping the rhythm and the emotional beats of the original.
Each language and niche has its own prompts with ready "before → after" examples. The key is keeping the timecodes: every phrase must last as long as in the original, or the lips will drift out of sync. The output is 10–30 versions in each of 5+ languages, sorted by how well the model expects each one to land.
ElevenLabs recreates the voice with the right accent
ElevenLabs Multilingual v2 covers 29 languages in one model, and a cloned voice can speak any of them. This matters when one actress appears in the original video and versions are needed for different countries: her voice can be heard in Serbian and in Polish too.
There are two ways to do it. The first is voice cloning: a 30-second sample is enough for a good clone. The second is stock voices for quick A/B tests of timbre: male or female, age, tone. Stability, similarity and style settings are tuned for each niche. The output is an audio track the same length as the original, which matters for the next step.
wav2lip matches the lip movement
At this step the actress starts speaking the new language as if she had been filmed again. wav2lip takes the original video and the new audio track and redraws the mouth frame by frame so the lips move in time with the new speech. This needs a graphics card and hours of computing.
If the voice is off screen, ffmpeg simply replaces the audio track. If a person talks on camera, wav2lip or SadTalker matches the face. The result is a finished mp4 for ad platforms (FB Ads, TikTok, VK Ads, Yandex Direct).
What it runs on
- OpenAI Whisper API (whisper-1)
- whisper-large-v3 on my own server for large volumes
- Subtitle parsing for exact timecodes
- GPT-4o / Claude Sonnet 4.5
- Prompts with "hook → rewritten script" examples
- Answers in a strict JSON structure
- ElevenLabs Multilingual v2
- Voice cloning from a 30-second sample
- Voice settings: stability, similarity, style
- ffmpeg to replace the audio track
- wav2lip and SadTalker when a person talks on camera
- Python to run the steps, plus a task queue
What changed
per version of an ad (new audio, no lip-sync)
versions in each language; one actress covers 5+ countries with no reshoots
Whisper, GPT and ElevenLabs at the services' own rates, no lip-sync
The main gain is how fast ideas get tested. An A/B test of 20 opening lines (it used to be two) can be launched in one evening. The best combinations show up within the first 24–48 hours, and the weak ones are switched off before they spend the budget. Cost per install fell by 15–30% because the opening line matched the audience more closely.
The same speech synthesis outside advertising
The voice step also works on its own, without video. It has two uses: a spoken notice for telephony, and voice messages sent in a messenger on the manager's behalf through a business account. Both use the same voice, made the same way, so it sounds identical in every channel.
The opening of a call follows a fixed order
First the caller introduces themselves, then a short phrase says the call is being recorded, without giving a reason. The order is fixed in the voice-over script, so every call will open the same way.
The audio files are rebuilt from a script
The repository holds the synthesis script, and the files are recreated in seconds: two files, 4.0 and 3.2 seconds long. The change history keeps the wording of each phrase.
A voice message needs one specific file format
The messenger shows a file as a voice message only in one format. In any other format the recipient sees an ordinary attachment. So the format is set right at the output of the synthesis step.
Summaries with numbers stay as text
Voice carries short notices. Summaries with figures go out as text, where the reader can glance back at a number and reread it as often as needed.
Still open: outbound calls
The telephony provider can play an audio file only in the inbound-call setup. There is nowhere yet to put a prepared file into an outbound call, and I've asked their support about it.
A free replacement for paid synthesis, on your own hardware
The technology on this page includes a paid synthesis service, and clients often ask whether it can be done without a subscription. The model chosen as a replacement has an MIT licence: it supports Russian and clones a voice from a 10-second sample. The two best-sounding models were ruled out because their licences forbid commercial use. For now the choice exists only on paper: the working setup still uses the paid service.
The access key
The key for the paid service was reissued after the old one turned up in a log. The new key can only synthesise speech, every other function of the service is closed to it, and a spending limit is set explicitly.
Who it suits and who it doesn't
- · Ad agencies where 5 or more people make videos
- · Ad teams and online shops that need a lot of videos
- · Online shop marketing teams that regularly film videos told from the customer's point of view
- · Launches in 5 or more countries at once, with every language from one original script
- · Podcasts and online schools translating and dubbing short clips
- · Copying other people's videos outright: that infringes copyright
- · Niches with strict advertising rules (medicine, pharmaceuticals, finance): they need an advertising lawyer
- · Cloning someone's voice without their consent: that violates their rights
- · Long videos of 10 minutes or more: voice-over and editing cost about as much as a good human voice actor
A note on ethics. This setup is meant for multiplying your own ideas and original scripts. Other people's videos and voices can't be used without permission: that breaks copyright and the rights of the people themselves. I transcribe other companies' ads to understand the market, then write my own script and make my own audio and video.
Related cases
GMBSPYLAB: from the Facebook Ad Library to a Telegram feed
Async Python and Postgres 16. Every day 30–75k ads come in from the public Facebook Ad Library, and each…
End-to-end attribution for two Shopify stores: from click to order
My own service on the Python standard library receives Shopify order notifications and checks their signature…
Shopify orders land in the warehouse sheets by themselves
Order numbers, tracking numbers, statuses and cancellations are filled in every 30 minutes. An Apps Script…
Start with the 5,000 ₽ audit: a written report and a payback estimate
I'll tell you what to automate first, when it will pay for itself and whether your task needs AI at all. Sometimes it doesn't.
Or just ask a question. During business hours I reply within 2 hours