-
Notifications
You must be signed in to change notification settings - Fork 348
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proper way to handle closed connections? #467
Comments
@jared-mackey A couple of questions:
@teamon I could be wrong but it seems like this probably isn't related to Tesla so if you'd rather close this discussion then we can transfer it to the finch repo. |
I'll update the versions for the one I am able to verify on and report back. |
@keathley I have updated to the latest versions from hex for Mint, Finch, Tesla and Nimblepool and the issue has persisted. I can't seem to reproduce it locally but the gist of what we are doing is this: Inside a Phoenix request we start a task that makes an API call over to the have I been pwned API. We do not await the task inside the Phoenix request. (We are not linking the task, so it should carry on in the background). |
After seeing some trace data on these it is clear the request is not being sent based on the duration of the trace being sub millisecond. So I don't think it is the same as the issue in mint previously reported. To work around this, I am going to add a retry middleware to our Telsa pipeline. |
Closing this one, do let me know if there is anything actionable on tesla side. |
I'm seeing a case where socket closed is returned but the request IS sent, so my retry middleware is causing the request to happen multiple times. |
We are also seeing these "socket closed" errors. I can't reproduce them locally very reliably but they happen frequently in production for us. Interestingly, they seem to happen more often on clients that have relatively low traffic. It's enough of an issue that we will probably have to move away from Finch until it gets fixed. |
@Billzabob I was able to work around this via a retry middleware. It’s not ideal, but did solve the problem for me. plug Tesla.Middleware.Retry,
delay: 10,
max_retries: 2,
should_retry: &should_retry?/1 and defp should_retry?({:ok, %{status: status}}) when status in 400..599, do: true
defp should_retry?({:ok, _}), do: false
defp should_retry?({:error, _}), do: true Customize to your specific needs, of course. 😁 |
We'll give that a try. Any chance that setting the |
@Billzabob Hi mate, just checking in, did you find a solution for this issue? We're seeing pretty much the same thing as you are, also only in production servers. I was thinking what might be the route cause, can you tell me what operating system are you using, Erlang/Elixir version etc.? We're on Alma Linux, Elixir 1.15.7, Erlang 26.1.2. What I concluded that it's the Tesla Finch adapter doing this kind of funky translation of the |
@salex89 # In our application.ex
pool_opts = %{default: [pool_max_idle_time: 60_000]}
children = [
# ...
{Finch, name: PlaidClient, pools: pool_opts},
# ...
] It still feels like there's an underlying bug but this workaround works pretty well. |
@Billzabob Interesting, thanks. We'll try that. Did you, perhaps, try with configurations like |
those pool_opts seem to be working great for me. Great find! Agreed seems like an underlying bug. |
I am also running into these errors. Polling an endpoint every X seconds and then randomly, Do we have any idea what the root cause of the Mint error is? |
Hi, first thanks for the awesome library. Love this project and all the work put into it to make it great. Thank you so much. (Ditto for Mint and Finch 😁)
Ok, onto the meat of the issue. When using the finch adapter we are seeing mint socket closed errors and was wondering the best way to handle these. The issue is two part though:
Regardless of the cause, would a retry middleware be the best approach for handling these in general? Or would trying to handle this in either Tesla, Finch or Mint be best? I would likely lean towards a retry middleware for a quick fix, but is that the most ideal solution long term?
I would be happy to help dive into the appropriate place to resolve this if it is determined it should be handled in one of these libs.
Thanks!
/cc @keathley -- might be relevant to you as well for Finch. 😁
The text was updated successfully, but these errors were encountered: