Skip to content

Commit

Permalink
Merge pull request #59 from yamanidev/fix-typo
Browse files Browse the repository at this point in the history
Fixed typo
  • Loading branch information
alextim authored Sep 10, 2023
2 parents 46e9193 + 53074af commit a256232
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion packages/astro-robots-txt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ The _robots.txt_ file informs search engines which pages on your website should

For Astro project you usually create the _robots.txt_ in a text editor and place it to the `public/` directory.
In that case you must manually synchronize `site` option in _astro.config.\*_ with `Sitemap:` record in _robots.txt_.
It brakes DRY principle.
It breakes DRY principle.

Sometimes, especially during development, it's necessary to prevent your site from being indexed. To achieve this you need to place the meta tag `<meta name="robots" content="noindex">` into the `<head>` section of your pages or add `X-Robots-Tag: noindex` to the HTTP response header, then add the lines `User-agent: *` and `Disallow: \` to _robots.txt_.
Again you have to do it manually in two different places.
Expand Down

0 comments on commit a256232

Please sign in to comment.