About our crawler

If you found 13xBot in your server logs, someone ran a launch-readiness audit on your site. Here is exactly what it does.

How to identify it

The full User-Agent string is:

• 13xBot/0.1 (+https://get13x.dev/bot)

What it requests

• The URL that was submitted, plus up to 24 more pages linked from it, to a depth of two clicks. Never more than 25 pages in total.
• /robots.txt and /sitemap.xml.
• A handful of HEAD requests to verify that linked resources resolve: your og:image, your favicon, and any privacy or imprint page you link to.
• One request to a path that does not exist, to check that missing pages return a real 404 rather than a soft one.

Requests run at most five at a time and the whole audit stops after 40 seconds. It does not execute JavaScript, submit forms, follow links to other domains, or attempt to log in.

It respects robots.txt

Paths you disallow are not requested. A rule naming 13xbot takes precedence over your wildcard rule, so you can allow or block us specifically.

How to block it

Add this to your robots.txt:

• User-agent: 13xbot
• Disallow: /

Audits of your site will then report that crawling was disallowed instead of returning results.

What we store

The submitted URL, the findings, and technical facts about the site such as the framework we detected. We do not store page contents beyond the evidence attached to a finding. Reports are unlisted and marked noindex. See the privacy policy for the full picture, including how to have a report removed.

Questions or problems

If the crawler caused load you did not expect, or you want your domain excluded entirely, email hello@get13x.dev and we will add it to a blocklist.