About our checker's requests
When someone runs our free Sitemap checker or SEO checker on a site, the requests come from our own servers with this user agent:
WebLandscapesCheck/1.0 (+https://weblandscapes.com/tools/bot/)
What one check fetches
- The home page (to see which address the site prefers), and robots.txt.
- Up to 12 sitemap files, the first 128 KB of each.
- Up to 50 of the pages the sitemap lists, at most 6 at a time, reading the first 32 KB of each.
- For the SEO checker: the page asked for (the first 192 KB), its http:// and www versions, and up to 30 of its links, mostly as HEAD requests.
The same site can be checked at most 30 times an hour by all users together, and results are not used for anything but the report the person asked for.
How to opt out
To keep our checker off some pages, add a group for it to robots.txt; we skip every page it disallows:
User-agent: WebLandscapesCheck
Disallow: /
robots.txt and the sitemap files themselves are still read, because that is what the check reports on. To be left out entirely, write to contact@weblandscapes.com from an address at the domain, and we add it to our opt-out list.