Skip to content

Releases: scrapy-plugins/scrapy-zyte-smartproxy

v1.6.0

27 May 14:18
Compare
Choose a tag to compare
  • Enable crawlera on demand by setting CRAWLERA_FORCE_ENABLE_ON_HTTP_CODES

v1.5.1

22 May 00:05
Compare
Choose a tag to compare
  • Remove username and password from settings since it's removed from crawlera.
  • Include affected spider in logs.
  • Handle situations when crawlera is restarted and reply with 407's for a few minutes
    by retrying the requests with a exponential backoff system.

v1.5.0

23 Jan 16:24
49e9b8a
Compare
Choose a tag to compare
  • Correctly check for bans in crawlera (Jobs will not get banned on non ban 503's).
  • Exponential backoff when crawlera doesn't have proxies available.
  • Fix dont_proxy=False header disabling crawlera when it is enabled.

v1.4.0

20 Sep 18:43
13a893f
Compare
Choose a tag to compare
  • Remove crawlera X-headers when crawlera is disabled
  • Introduction of DEFAULT_CRAWLERA_HEADERS settings

v1.2.0

24 Oct 10:33
Compare
Choose a tag to compare
  • Recommend middleware order to be 610 to run before RedirectMiddleware.
  • Change default download timeout to 190s or 3 minutes 10 seconds (instead of 1800s or 30 minutes).
  • Test and advertize Python 3 compatiblity.
  • New crawlera/request and crawlera/request/method/* stats counts.
  • Clear Scrapy DNS cache for proxy URL in case of connection errors.
  • Distribute plugin as universal wheel.