Skip to content

v1.1.0 — keep crawlers out of search

Latest

Choose a tag to compare

@jpbaking jpbaking released this 25 Jun 13:05

Keep the speed test out of search results and discourage crawlers.

This release adds a layered "do not index" posture so the self-hosted speed test never surfaces in search engines.

What's new

  • robots.txt served from the web root. Crawling is intentionally allowed so crawlers can fetch each page and honor its noindex directive — a blanket Disallow: / would stop them from ever seeing it, leaving externally linked URLs free to surface as bare results.
  • noindex, nofollow robots meta on the main UI (index.html) and the stability page (stability.html).
  • X-Robots-Tag: noindex header on the shareable result image (results/index.php) — the surface most likely to be hot-linked, and the only way to noindex a non-HTML response.
  • robots.txt is wired into both Dockerfiles and the entrypoint so it ships in the container web root.

The net effect: anything a search engine crawls gets dropped from its index. A Cloudflare-level layer (Bot Fight Mode / block AI crawlers) is recommended for bots that ignore these signals entirely.

Full changelog: v1.0.0...v1.1.0


🤖 Generated with Claude Code