#

webcrawlers

Here is 1 public repository matching this topic...

slemarchand / no-robots

🚫🤖 Override /robots.txt to disallow all web crawlers, regardless settings stored in the database. Compatible with Liferay 7.0, 7.1, 7.2, 7.3 and 7.4.

web osgi robots-txt crawlers liferay liferay-portal liferay-dxp liferay-7 liferay71 liferay70 liferay72 liferay-71 liferay-72 liferay73 webcrawlers liferay-73 liferay-74 liferay74 liferay-70

Updated Sep 4, 2021
Java

Improve this page

Add a description, image, and links to the webcrawlers topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the webcrawlers topic, visit your repo's landing page and select "manage topics."