Free tools / All tools / Technical scan
Free technical scan you can rerun on your own site
Our free technical scan is one script you download and run on your own computer, as often as you like. It reads robots.txt, the XML sitemap and llms.txt, then every page your sitemap lists, and checks nine things crawlers and assistants read: crawler policy, sitemap, llms.txt, canonical, sharing tags, structured data, navigation, language alternates and phone layout.
Download the script
bw-technical-scan.mjs, version 1.0.0, released 9 November 2026. 26,189 bytes. SHA-256 67b2c13d1b254fd9c8cfffc71105011a07f8bf5839f5a598b5c8f3c270b65521.
It needs Node.js 22 or later and nothing else. The phone layout render also needs Chrome or Edge installed on the same computer. There is no account, no email and no key; the script never sends results to us.
Run it
Scan one site and keep the full result:
node bw-technical-scan.mjs --out result.json https://www.example.comScan a site and its docs subdomain, render up to 12 pages each at phone width, and fail a build when any check fails:
node bw-technical-scan.mjs --phone --strict https://www.example.com https://docs.example.comCheck a local build before you publish it, judged against the address it will have:
node bw-technical-scan.mjs --map https://www.example.com=http://127.0.0.1:8080 https://www.example.comRun node bw-technical-scan.mjs --help for every option. A large site can be read as an even spread with --max-pages 200.
The nine checks
| Check | What passes | Why it is checked |
|---|---|---|
crawler_policy | robots.txt answers 200 as plain text, with at least one User-agent group and a Sitemap line. | Crawlers read this file first to learn what they may fetch and where the page list is. |
sitemap | A sitemap is found from robots.txt or /sitemap.xml, parses, lists pages on the same host, and every listed page answers 200 without noindex. | A sitemap that lists missing or noindex pages sends crawlers to pages that cannot be used. |
llms_txt | /llms.txt answers 200 with a "# " title on the first line, a "> " summary line and at least one link. | Assistants that read llms.txt expect the format published at llmstxt.org. |
canonical | Each page has exactly one canonical link, it is absolute, and it names the page itself. | Two addresses for one page split the signals a search engine or assistant collects about it. |
social_metadata | Each page has og:title, og:description, og:url, og:type, an absolute og:image and twitter:card. | Without them a shared link shows no title or image in chat apps, LinkedIn or X. |
structured_data | Each page has at least one JSON-LD block; every block parses and names a schema.org @context and @type. | A block that does not parse is ignored, so the facts in it never reach a crawler. |
navigation | Each page has a nav landmark holding at least three links to the same site. | Crawlers and screen readers find the main sections of a site through its navigation. |
language_alternates | Each page has hreflang="en" naming its canonical and hreflang="x-default". | The pair tells search engines which address to show by default and which language the page is in. |
phone_layout | Each page declares a width=device-width viewport. With --phone, the page also renders 390 pixels wide with no horizontal overflow. | Most first visits are on a phone; a page wider than the screen hides text off its right edge. |
Method
The script sends only GET requests, at most six at a time, each with the user agent BroadcastwellTechnicalScan/1.0.0 and a link back to this page. It changes nothing on the site. It follows robots.txt Sitemap lines on the same host first, then /sitemap.xml, and follows sitemap index files up to 25 files.
Page checks read the HTML the server sends, before any script runs, because that is what most crawlers read. Tags added later by JavaScript are not seen, and the result says so by failing the check. The phone render is the exception: it loads the page in a headless browser 390 pixels wide, waits for the load event and 0.8 seconds, then compares the page width with the screen width and names up to five elements that stick out.
Each run writes the date, the version, the options and every page's result to JSON, so a run before a fix and a run after it can be compared line by line.
What it does not check
It does not judge content quality, rankings, page speed or what any AI engine says about you. A pass means the tag or file is present and well formed, not that its content is right. Hreflang presence does not prove the alternates are correct for a site that serves several languages. To see what ChatGPT, Claude, Perplexity, Google AI Overviews and Google AI Mode actually say, start with our free tools.
Related free tools
Check one rule at a time with the robots.txt tester, the XML sitemap generator, the llms.txt checker, the social preview inspector and the AI access checker.
When you want us to do the work, every offer and its published price is on our offers page.