Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请教下爬虫的频率在哪设置的吗 #78

Open
mashpolo opened this issue Feb 21, 2020 · 1 comment
Open

请教下爬虫的频率在哪设置的吗 #78

mashpolo opened this issue Feb 21, 2020 · 1 comment

Comments

@mashpolo
Copy link
Contributor

最近一直搜索失败,看了下日志,发现不管是从baidu还是从360搜索,都要求了验证码,导致获取不到正确的页面,看起来像是爬虫太频繁了导致的。
具体返回的结果是

        如果经常出现此页面,请把您的IP和反馈意见<a href="http://info.so.com/feedback.html?add=ip%3A%20185.194.148.243" target="_blank">提交</a>给我们,我们会尽快处理您的反馈。</p>
        <div class="why">
            <strong>为什么会出现验证码?</strong>
            出现验证码表示您所在的网络可能存在异常,同IP短时间内大量发送请求,被服务器判断为异常请求。<br/>
            需要您输入验证码,以确认本次的访问是正常请求,而非机器自动请求,即可正常使用360搜索。
      </div>
    </div>
@howie6879
Copy link
Owner

howie6879 commented Feb 21, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants