GMBSPYLAB: from the Facebook Ad Library to a Telegram feed

Playwright without parsing the page

The ad listing is built in the browser, so a headless Chromium opens the page once. After that the system works with the same data the browser receives and spends no time going through the page structure card by card.

Requests go out at a measured pace: a pool of sessions, each tied to a market, at least 2.5 s between requests and a hard limit per session. The aim is a steady stream that covers the daily volume.

Changes on the source's side don't stop the collection

The source keeps changing: its markup and response parameters change often, and a simple parser quietly returns zeros after each change. Here the structure of the response is checked at every step. If it doesn't match, the session starts over and the stream keeps going. If the source returns an error or slows down, the session waits longer between attempts and recovers on its own. This happens regularly and needs no manual work.

Postgres 16, arq, and MinIO for media

SQLAlchemy 2 async, asyncpg and alembic migrations, with indexes on advertiser ID, country and first-seen date. arq on Redis runs background tasks (downloading media, removing duplicates). Images and videos go to MinIO / R2 through aioboto3. Near-duplicate images are removed by perceptual hash, which compares how images look: 30–75k raw cards become 5–12k unique ones a day.

Telegram via aiogram 3, in two formats

v1: a forum-style group. A topic is created with createForumTopic for each of the 30+ markets (the 27 EU countries, the UK, Brazil and others such as TZ, MX, IN, CA, ZA and RS). Each country's ads go to its own topic, so filtering by market is easy.

v2: a hashtag channel. One stream, tagged with #GEO #PWA #VIDEO #1DAY #FBPAGE. A daily scheduled job updates the #NDAY tag, which shows how many days the ad has been running.