GMBSPYLAB: FB Ad Library → Telegram feed
Production pipeline collecting ads from the public FB Ad Library on Playwright and async Python. The interesting part is what happens after collecting: normalisation, perceptual dedup, queues and delivery. 30-75k cards a day across 30+ markets, delivered to Telegram with a separate topic per country.
Commercial ad-intelligence services: expensive, slow, missing the markets we need
Commercial ad-intelligence services (AdHeart, AdSpy, ad-library aggregators) charge $300-800/mo for a subscription and cover a limited set of countries. For niche markets (Tier-2 EU, LATAM, APAC) the data either does not exist or arrives 1-2 days late.
The Meta API via a dev app requires passing App Review — for a small team that means weeks of waiting for data that is already public. Meanwhile facebook.com/ads/library renders on the client, so a naive static scrape returns an empty page.
Goal — to collect a live stream of ads from 30+ countries in-house, into our own database, with filtering by format (image / video / PWA landing), how long the ad has been running, normalised tags, and posting to Telegram.
4-layer pipeline: collect → normalize → store → distribute
Playwright, but without DOM parsing
The listing renders on the client, so headless Chromium opens the page once — after that the pipeline works with the same response the browser itself receives and spends no time walking the tree. Parsing HTML on every card is tens of times more expensive.
Requests go out at ordinary human pace: a pool of sessions bound to a market, a floor rate of 2.5s between requests and a hard per-session ceiling. The point is to keep the stream predictable and sufficient for the daily volume.
The pipeline survives changes on the other side
The source is alive: its markup and response parameters change several times a day, and a naive parser silently returns zeros after every such change. Here the response schema is validated at every step, a mismatch re-initialises the session and the stream keeps running. If the source returns an error or slows down, the session backs off exponentially and comes back on its own. This happens dozens of times a day and none of it needs me.
Postgres 16 + arq + MinIO for media
SQLAlchemy 2 async + asyncpg + alembic migrations, indexes on advertiser-id × country × first-seen-date. arq via Redis for background tasks (media download, dedup). Images/videos → MinIO / R2 via aioboto3. Dedup via perceptual hash — 75k raw → 5-12k unique per day.
Telegram via aiogram 3 — two surfaces
v1 — Forum supergroup: createForumTopic
for each of the 30+ markets (EU-27 + UK + BR + TZ/MX/IN/CA/ZA/RS/...). Posting goes to a topic per country, so filtering by market is trivial.
v2 — Hashtag feed channel: single stream, tagging with
#GEO #PWA #VIDEO #1DAY #FBPAGE.
The activity-refresher cron bumps #NDAY
daily (shows "running for N days").
Technologies
- Playwright (Chromium headless)
- Session pool bound to a market
- Rate limiting + exponential backoff
- Response-schema validation at every step
- Python 3.12 + uv (package manager)
- SQLAlchemy 2 async + asyncpg
- PostgreSQL 16 + alembic migrations
- Pydantic v2 for all schemas
- arq + Redis 7 (background tasks)
- APScheduler (timer cron)
- MinIO / Cloudflare R2 (aioboto3)
- SQLite (cache, local state)
- aiogram 3 (Bot API)
- Forum supergroup with createForumTopic
- Hashtag-feed channel + N-day refresher
- Healthcheck → Discord/TG alerts
- Playwright-FB (primary source)
- ScrapeCreators (commercial backup source, non-EU)
- Meta Graph API (for verified IDs)
- EU DSA Repository (regulatory data)
- structlog (structured logs)
- 16/16 tests passing (pytest)
- Healthcheck probe + emergency halt
- Discord webhooks for status
The outcome
after dedup: 5-12k unique
EU-27 + UK + BR + 30 non-EU markets
/mo for the whole infrastructure — VPS + storage. Commercial services charge $300-800
Where it fits for clients: the same shape (collect → normalise → dedup → deliver) transfers to any source the client has lawful access to: your own Avito, WB Seller or Ozon Seller account, official exports, open ad libraries. On a call we work out the exact source and what the platform's own terms allow you to do with it.
Related cases
End-to-end attribution: from ad click to order
A stdlib middleware takes Shopify webhooks, verifies HMAC, fires an S2S postback to the tracker and a…
Paid Shopify orders → warehouse spreadsheets
Orders, tracking numbers, statuses and cancellations land by themselves every 30 minutes. An Advanced Sheets…
Shipping-partner file exchange: day file out, cost and tracking back
The day file builds itself from a date in the book and carries the book's colouring as genuine conditional…
The 9,900 ₽ audit — with a concrete report and quote
I'll tell you what to deploy in your business first, what the payback looks like, and whether you need AI for the task at all (sometimes you don't).
Or just send your question — I reply within 2 hours