ES Book a call Menu

Where the files go

Every file in the pack goes at the root of your site, so it answers at yoursite.com/robots.txt, not in a folder. How you put it there depends on your host. Below: the four steps, then the details for WordPress, Squarespace, Wix, Shopify, Cloudflare and ordinary hosting. Your pack's INSTALL.md has the same steps with your own addresses filled in.

1. Get the files

After paying you land on your order page. Its address is your download link for 30 days (90 on the $9.97 pack), and we email it to you too.

The order page: a Download all (ZIP) button, the eight files each with View and Download, the robots.txt choice, and the re-run button.
  1. Download all eight files as one ZIP.
  2. Or view and download each file on its own.
  3. Change your robots.txt choice for AI crawlers at any time; download robots.txt again afterwards.
  4. Re-run the crawl after you change pages or install the files: 10 re-runs are included.
llms.txt opened on the order page: the site's name, a one-line summary and a list of its pages with descriptions.
View shows a file right on the page. This is llms.txt for beecopaint.com, a site we run.

2. The root of your site

The root is the folder your home page lives in. A file put there answers at your domain followed by its name. Crawlers only look there: a robots.txt in /files/robots.txt is never read.

A web root folder holding index.html, robots.txt, sitemap.xml, llms.txt and llms-full.txt; each file answers at the domain followed by its name. public_html/ index.html robots.txt sitemap.xml llms.txt llms-full.txt https://yoursite.com/robots.txt https://yoursite.com/sitemap.xml https://yoursite.com/llms.txt https://yoursite.com/llms-full.txt same folder as your home page
Our drawing. The folder is often called public_html, htdocs, www, public or dist; on WordPress it is the one with wp-config.php in it.

3. Your host

WordPress

WordPress has made its own sitemap at /wp-sitemap.xml since version 5.5, and it generates a robots.txt by itself. Yoast SEO and Rank Math can also make llms.txt: in Yoast it is Yoast SEO > Settings > Site Features > AI tools > LLMS.txt, free; in Rank Math, the LLMS Txt module in its Dashboard. Keep theirs if you like; then use only our robots.txt, schema and fix list, and delete the line "Sitemap: https://yoursite.com/sitemap.xml" from our robots.txt. Yoast · Rank Math · WordPress 5.5

  1. Open your hosting account's file manager, or connect with SFTP.
  2. Go to the folder with wp-config.php and wp-content in it.
  3. Upload robots.txt, llms.txt and llms-full.txt, and sitemap.xml unless you keep your plugin's. Download any file you replace first, as a backup.
  4. Open yoursite.com/robots.txt. Most hosts serve a real file before WordPress sees the request; if you still see WordPress's version, a plugin is answering for it, so change robots.txt in that plugin's settings instead.

Squarespace

  • llms.txt: on Squarespace 7.1, open the SEO panel, then SEO Settings > LLMS.txt, paste the contents of our llms.txt and save. It is off until you do. There is one field, so llms-full.txt has nowhere to go. support.squarespace.com
  • robots.txt: every Squarespace site uses the same file and it can't be edited, so ours can't be installed. For AI crawlers, use Settings > Crawlers > "Block known artificial intelligence crawlers". Its list is longer than ours and not the same. support.squarespace.com
  • sitemap.xml: Squarespace makes its own at /sitemap.xml and it can't be edited. Ours lets you check that theirs lists the right pages.
  • The schema: Settings > Website Tools > Code Injection, the Header field, inside a script tag (Core, Plus and Advanced plans). support.squarespace.com

Wix

  • robots.txt: in your dashboard, SEO & GEO > Tools and settings > Robots.txt Editor > View File. Replace the text with ours, keep any line Wix says it needs, and save. Wix Customer Care doesn't help with changes made here. support.wix.com
  • sitemap.xml: Wix makes its own and lists it in robots.txt. Keep it.
  • llms.txt: we found no Wix help page about llms.txt. Check the SEO & GEO section of your dashboard; if it isn't offered, it can't go at the root.
  • The schema: in the home page's SEO settings, Structured Data Markup > + Add New Markup, paste the JSON (without the script tag) and Apply. Wix takes up to 5 blocks a page, each under 7,000 characters. support.wix.com

Shopify

All three changes are in your theme's code: Online Store > Themes > … > Edit code. Duplicate the theme first so you can go back. help.shopify.com

  • robots.txt: Add a new template > robots > Create template. Keep Shopify's default rules and add our extra lines (the Sitemap: line and the AI crawler group) below them. Shopify calls this an unsupported customization; deleting the template brings the default back. help.shopify.com
  • llms.txt: Shopify already answers /llms.txt for every store. To serve ours, add a templates/llms.txt.liquid template and paste our llms.txt into it. shopify.dev
  • sitemap.xml: Shopify makes its own. Keep it.
  • The schema: in layout/theme.liquid, inside the head. Many themes already output Organization data; check first, so you don't add a second block.

Cloudflare (Pages, Workers)

  1. Put robots.txt, sitemap.xml, llms.txt and llms-full.txt in the folder that ends up at the root of your build (often public/ or static/).
  2. Deploy as usual.
  3. If Cloudflare's managed robots.txt is on (Security Settings > Bot traffic > Configure AI bot policies), Cloudflare adds its own lines in front of your file. Keep one source of AI rules: either that setting or our AI crawler group. developers.cloudflare.com

Ordinary hosting (cPanel, FTP, Netlify, GitHub Pages, S3)

  1. Open the file manager or connect with FTP or SFTP.
  2. Find the folder with your home page (index.html or index.php) and upload the four files there.
  3. Netlify, GitHub Pages and Vercel: put them in the folder that gets published and deploy. S3: upload them to the bucket's root with Content-Type text/plain for the .txt files and application/xml for the sitemap.

4. The schema

schema.jsonld describes your business to search engines. Open it, replace every TODO with the real value or delete that line, then paste it into your home page's head section like this:

<head>
  …
  <script type="application/ld+json">
  { "@context": "https://schema.org", "@type": "LocalBusiness", "name": "…" }
  </script>
</head>

Then test the page with Google's Rich Results Test. search.google.com/test/rich-results

5. Check it worked

Open each address in your browser. You should see the plain text of the file. If you see your home page or an error page, the file isn't at the root yet.

A browser window open at yoursite.com/llms.txt showing plain text that starts with a # line: the file is in place. https://yoursite.com/llms.txt # Your Business > One line about what you do. ## Services - [Your service](https://yoursite.com/service/): … plain text = in place
Our drawing of what a correctly installed llms.txt looks like in a browser.

Then go back to your order page and press Re-run: the new crawl should find your robots.txt and sitemap. Submit the sitemap to Google in Search Console. support.google.com

Before you buy: what the preview shows

The preview's head: pages read, the site's facts, and a table of which files the site's host can take.
  1. How many pages we read and how many can go in the sitemap and llms.txt.
  2. Which files your host can take, before you pay.
The robots.txt choice and the line-by-line change it makes to the live file.
  1. Pick what AI crawlers are asked to do.
  2. See exactly which lines are added (+) to your current robots.txt.
The buy box: the two packs, the host and the buy button.
  1. $4.97 for up to 50 pages, or $9.97 for up to 250 pages and 90 days.
  2. Your host, so INSTALL.md has the right steps (you can change it later).
  3. Buy opens the payment window on the same page.
The payment window: the site and the price, the email for the download link, the ownership box, and the space where the ways to pay appear.
  1. The email the download link goes to.
  2. Confirm you own or manage the site.
  3. Card, Apple Pay, Google Pay or Link appear here once the box is ticked. Card details are handled by Stripe; we never see your card.

Run the free preview

Get a quote

Leave your name and email, and a phone number or a note if you like. We'll get back to you soon.

Prefer email? Contact@WebLandscapes.com

Book a call

Fifteen minutes, free. Pick a time below; the invitation comes from Google Calendar.

Pick a time

1 Pick a day

Checking the calendar…

Rather talk now? Call +1 786 608 8489 or email Contact@WebLandscapes.com.