Researching a competitor from open sources: what you can see from outside

Collection runs with one command

The 01-recon.sh script takes a domain and in one pass runs WHOIS, DNS, certificate parsing, a search of certificate history in the public certificate logs, a HEAD request, public scans, passive DNS and the web archive. Each step is saved as its own file in a working folder. You can come back to that file, attach it to a conclusion and compare it with a new run on the same domain a month later. Nobody can check an analysis that exists only in the analyst's terminal.

Opening the page is a separate step, on purpose

The first script doesn't open the page: WHOIS, DNS and archives come from outside sources, and the certificate and headers are read without loading the page. Opening the page itself is the only step that leaves a trace in the site owner's analytics, so it lives in a separate file, 02-live-click.sh, and runs only when I decide to run it. It captures exactly what the analysis is for:

• the offer and the price on the date of the analysis, which is what your own offer will be compared with
• contact forms: how many fields, which are required, what they promise after sending
• the outside services the page loads: analytics, chat, payments, CDN
• Set-Cookie and response headers, which identify the site engine quite precisely

The page is saved in full, prices included, as of the date of the analysis. Two dated copies turn "they were cheaper in spring" into a fact.

The platform is recognised by its fingerprints

A site rarely says what it's built on. But every platform leaves fingerprints, and they stay on the site when the text changes. The tracker-signatures.md reference is a "signal → platform" table that grows with every analysis:

• a response header unique to one platform: the strongest signal, which identifies the platform at once
• the naming and structure of cookies, which reveal both the site engine and the connected analytics
• file paths and build file names, which show the framework and how the front end is built
• the list of outside domains the page loads: CDN, analytics, chat, payments
• host names in the cloud provider's DNS, which show where everything is hosted and on what

The site's code shows how the product is built

A separate step reads the collected JavaScript and CSS to see how the product is built. Service addresses written into the code show which integrations are connected. Library versions show how fresh the front end is and how often it's updated. Internal names show how the data is organised. The step is cheap, often skipped, and it explains a great deal.

The rules for conclusions are kept in a separate document

Several parties stand behind a single site: the host, a CDN, a website builder or CMS, and the connected outside services. Each of them serves dozens of unrelated clients. So attribution-rules.md splits signals into two lists: those that actually prove something, and those that show up everywhere. A conclusion is drawn only from a unique signature.

No paid sources, by design

All collection relies on sources anyone can use: registry WHOIS servers, public DNS resolvers, public certificate logs, public page scans, passive DNS and the web archive. This limit is good for the method: it makes me rely on signals that anyone can check independently.

The method caught my own mistake

In the third analysis my first conclusion was wrong. Two different sites used the same outside service, and that looked like proof they were the same product. The signal was striking, repeatable and completely misleading. The error showed up one step later: the response headers carried a platform signature, and the second site had a different one. All the two sites shared was an outside service used by dozens of unrelated companies.

The rule that came out of it

Shared tools do not prove a shared team. That sentence went into attribution-rules.md together with a list of signals that look convincing and prove nothing:

• anycast addresses of a popular CDN: one shared pool for every customer on the plan
• the most popular registrar and its default privacy service: that only shows popularity
• a shared outside widget (chat, form, analytics): dozens of unrelated companies install them
• a standard landing page template and the usual cookie set of a popular framework

The resulting procedure is short: find a unique signature, identify the platform from it, confirm it with a second independent signal, and only then state a conclusion. Shared infrastructure goes into the write-up only as supporting evidence.

What I have after three analyses

A repository with the playbook, two reusable scripts and two reference documents that grow with each new analysis. There is also a delivery checklist with the fields that must be filled in before a conclusion counts as finished: what the product is, what the site runs on, which services are connected, and what exactly supports each point. The checklist exists for the same reason as the rules: so that the analyst's confidence never stands in for evidence.