timeout not working?

[dropwhile]

It seems that timeouts are not working when the host cannot be reached. It appears that the standard socket timeout timeframe is being used (I get similar timings with timeouts via telnet to the same non listening host/port for example).

>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> import requests
>>> import time
>>> def callwithtime(callable, *args, **kwds):
...     t1 = time.time()
...     try:
...         callable(*args, **kwds)
...     except:
...         pass
...     t2 = time.time()
...     print '%s took %0.3f ms' % (callable.func_name, (t2-t1)*1000.0)
... 
>>> callwithtime(requests.get, 'http://1.1.1.1/foo.txt')
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): 1.1.1.1
^Cget took 10215.639 ms
>>> callwithtime(requests.get, 'http://1.1.1.1/foo.txt', timeout=1)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): 1.1.1.1
WARNING:requests.packages.urllib3.connectionpool:Retrying (0 attempts remain) after connection broken by 'error(60, 'Operation timed out')': /foo.txt
get took 75865.529 ms
>>> callwithtime(requests.get, 'http://1.1.1.1/foo.txt', timeout=0.1)
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): 1.1.1.1
WARNING:requests.packages.urllib3.connectionpool:Retrying (0 attempts remain) after connection broken by 'error(60, 'Operation timed out')': /foo.txt
get took 75441.546 ms

[kennethreitz]

This should be causing a ConnectionError, but instead it just hangs. Very odd.

[dropwhile]

Does a timeout value need to be passed to HTTPConnection when you create one?

https://github.com/kennethreitz/requests/blob/a76f4925b512b481375f0886a5b675d3e8522b34/requests/packages/urllib3/connectionpool.py#L151

When I create an httpconnect object without passing timeout to it, it uses the default timeout for a new socket.
You are setting the timeout here: https://github.com/kennethreitz/requests/blob/a76f4925b512b481375f0886a5b675d3e8522b34/requests/packages/urllib3/connectionpool.py#L214

But that is after the connect attempt has already been made, but before reading. So it is more of a read timeout than a connection timeout.

Maybe pass a timeout value on the _new_conn creation of an HTTPConnection?

https://github.com/kennethreitz/requests/blob/a76f4925b512b481375f0886a5b675d3e8522b34/requests/packages/urllib3/connectionpool.py#L151

>>> h = httplib.HTTPConnection('1.1.1.1', 80, timeout=1)
>>> try:
...   h.request('GET', '/')
... except Exception as e:
...   print e
... 
## short wait here
timed out
>>> 
>>> h = httplib.HTTPConnection('1.1.1.1', 80)
>>> try:
...   h.request('GET', '/')
... except Exception as e:
...   print e
... 
### very long wait here
[Errno 51] Network is unreachable

related code in python httplib module where timeout is set if passed to class on init.

[ruudud]

This seems to be partially fixed in urllib3/urllib3#17.

But when calling urlopen in the Request class here, there is a bug in the urlopen function that does not consider the timeout parameter when throwing the exception, causing a TypeError.

I´ve created an issue and a pull request (urllib3/urllib3#23) to get this fixed.

[ruudud]

..and now it is merged in. Is there a strategy for merging in new changes from urllib3?

[kennethreitz]

just updated it :)

the 'make deps' command pulls it in, for now. I'm technically a fork though, so use sparingly.

[ruudud]

Aha, I see. Missed that detail. (-:

Tested the timeout parameter now, and it is working for me.

[kennethreitz]

Excellent! I'll try to roll a release tonight.

[dropwhile]

great! :D

[kennethreitz]

Released!

[pikeas]

I'm experiencing this issue myself, but only when a DNS lookup is needed.

Using cactus's code above:

callwithtime(requests.get, 'http://1.1.1.1/') # 75 seconds
callwithtime(requests.get, 'http://1.1.1.1/', timeout=1) # 1 second
callwithtime(requests.get, 'http://NOTREAL/') # 30 seconds
callwithtime(requests.get, 'http://NOTREAL/', timeout=1) # 13 seconds, INCORRECT

The two requests requiring DNS lookups also output:
WARNING:requests.packages.urllib3.connectionpool:Retrying (0 attempts remain) after connection broken by 'gaierror(8, 'nodename nor servname provided, or not known')': /

[wanaryytel]

I'm seeing the same behaviour on v2.13.0.

[Lukasa]

@wanaryytel Please do not comment on old issues (this issue was originally opened in 2011, more than five years ago) to suggest you are seeing vague problems like "timeouts not working". The last comment here was made almost five years ago, and any problem you are seeing now is going to be entirely unrelated to this one.

If you are having an issue, either look for newer issues that replicate your problem or open a new one.

I should note, by the way, that "I am seeing the same problem" is using a few more words to say "+1", and is not hugely helpful. I highly recommend being a bit clearer about what problem you are seeing, and how you are triggering it. The more information you can give us, the more likely it is that you will see a resolution to your problem.