https://github.com/kennethreitz/grequests
virtualenv -p /usr/local/bin/python2 grequests.py2.env
source grequests.py2.env/bin/activate
pip install grequests
pip install -r requirements.py2.txt
virtualenv -p /usr/local/bin/python3 grequests.py3.env
source grequests.py3.env/bin/activate
pip install grequests
pip install -r requirements.py3.txt
- test environment: Mac Pro macOS High Sierra
- test contents: http://www.baidu.com domain 200 concurrent requests
# python2 environment
(grequests.py2.env) ➜ grequests_project python tests/get_baidu_requests.py
6.46457982063
(grequests.py2.env) ➜ grequests_project python tests/get_baidu_grequests.py
0.7383248806
# python3 environment
(grequests.py3.env) ➜ grequests_project python tests/get_baidu_requests.py
5.181720018386841
(grequests.py3.env) ➜ grequests_project python tests/get_baidu_grequests.py
0.8184318542480469
- test environment: Mac Pro macOS High Sierra
- test contents: https://www.baidu.com domain 200 concurrent requests
# python2 environment
(grequests.py2.env) ➜ grequests_project python tests/get_baidu_requests.py
12.3944959641
(grequests.py2.env) ➜ grequests_project python tests/get_baidu_grequests.py
2.24422097206
# python3 environment
(grequests.py3.env) ➜ grequests_project python tests/get_baidu_requests.py
12.188169956207275
(grequests.py3.env) ➜ grequests_project python tests/get_baidu_grequests.py
2.3108882904052734
The efficiency of grequests
is an order of magnitude higher than requests
HTTPS's request is twice as slow as HTTP