Releases: scrapy-plugins/scrapy-zyte-smartproxy
Releases · scrapy-plugins/scrapy-zyte-smartproxy
v1.6.0
- Enable crawlera on demand by setting
CRAWLERA_FORCE_ENABLE_ON_HTTP_CODES
v1.5.1
- Remove username and password from settings since it's removed from crawlera.
- Include affected spider in logs.
- Handle situations when crawlera is restarted and reply with 407's for a few minutes
by retrying the requests with a exponential backoff system.
v1.5.0
- Correctly check for bans in crawlera (Jobs will not get banned on non ban 503's).
- Exponential backoff when crawlera doesn't have proxies available.
- Fix
dont_proxy=False
header disabling crawlera when it is enabled.
v1.4.0
- Remove crawlera X-headers when crawlera is disabled
- Introduction of DEFAULT_CRAWLERA_HEADERS settings
v1.2.0
- Recommend middleware order to be 610 to run before
RedirectMiddleware
.
- Change default download timeout to 190s or 3 minutes 10 seconds (instead of 1800s or 30 minutes).
- Test and advertize Python 3 compatiblity.
- New
crawlera/request
and crawlera/request/method/*
stats counts.
- Clear Scrapy DNS cache for proxy URL in case of connection errors.
- Distribute plugin as universal wheel.