Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add noindex, follow meta tags to search page. #1615

Closed
2 tasks
benlk opened this issue Jan 15, 2019 · 0 comments · Fixed by #1674
Closed
2 tasks

Add noindex, follow meta tags to search page. #1615

benlk opened this issue Jan 15, 2019 · 0 comments · Fixed by #1674
Labels
priority: high Either blocks work on a priority-normal task or a solution here informs other work. status: needs tests type: feature request
Milestone

Comments

@benlk
Copy link
Collaborator

benlk commented Jan 15, 2019

Why

Because with the right search query and aggressive linking to that query, SEO spammers can use a site's search results page to put their message in Google using that site's page rank. In the reported case, the search results page is being used to get a scam phone number into Google's results.

Here's a page of google results for the search query that the scammer is using:

screen shot 2019-01-15 at 12 00 07 pm

And here's their analytics dashboard:

screen-shot-2019-01-02-at-10-56-20-am

To state the problem plainly:

  1. A spammer puts their desired payload into the search box, and presses search: The URL now contains the spammer's query
  2. The spammer takes the URL for the search page with their query and smears it around the internet, using that URL to drive search crawlers so that the desired text ranks highly in Google
  3. The spammer profits when someone calls their phone number for support for a bitcoin wallet problem, probably.

See also https://yoast.com/blocking-your-sites-search-results/

Brought up in https://secure.helpscout.net/conversation/754261795/3067?folderId=1211651

The solution

The long-term solution in Largo, which should be implemented as soon as possible.

  • add an option, checked by default, to "Prevent search engines from indexing your site's internal search pages. This is recommended because the pages have no internal value"
  • display the <meta name="robots" content="noindex, nofollow"> tag on search pages in the header

The short-term solutions:

  • add Yoast SEO plugin?
  • manually edit robots.txt
@benlk benlk added priority: high Either blocks work on a priority-normal task or a solution here informs other work. type: feature request status: needs tests labels Jan 15, 2019
@benlk benlk added this to the 0.6.2 milestone Jan 15, 2019
benlk added a commit that referenced this issue Apr 20, 2019
This resolves #1615, where I had originally written to use an option for this. Instead, based on research, it is recommended to avoid indexing these pages. See https://yoast.com/blocking-your-sites-search-results/ .
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: high Either blocks work on a priority-normal task or a solution here informs other work. status: needs tests type: feature request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant