Concurrent eager loading failing under Passenger

[joegsn]

I'm fairly certain I'm missing something, but I'm not sure what at this point.

Please note: Commentary here is specific to running within Passenger:
I have an eager query which loads a lot of associations, and wanted to see if I could speed it up via some parallelization (async seems like it would fit the bill just fine). So, I added this to my initialization:

DB.extension :async_thread_pool
Sequel::Model.plugin :concurrent_eager_loading, always: true

and went to go run the big query, and it just stopped after querying the top-level item. I did some experimenting, and it seems that the concurrent (async) loading starts once you have 2 associations being loaded (which makes sense).

Then I tried running the same query manually via irb and found that it worked flawlessly.

This confused me, so I set about making a small example to see if I could figure out why it was occurring, but all I've been able to accomplish so far is come up with a small (relatively) example with which I can recreate the issue.

(Had to add .txt to the end for github to let me upload)
config.ru.txt
Gemfile.txt

In my typical habits, I use bundler, and then run:

bundle install
bundle exec passenger start -p 9292

The example will create a small database (test.db) and starts passenger up.
Accessing /eager succeeds just fine.
Accessing /eager-concurrent fails. Log output shows that it queried the first table, but nothing else.

Since none of this happens when using irb, I have to assume it's Passenger specific, but I'm at a loss at the moment. Perhaps there's another statement that needs to be run when Passenger is in this spawning method (like DB.disconnect). Any help would be appreciated.

[jeremyevans]

If you are using Passenger, you definitely want to disconnect connections before Passenger forks child processes, so if you aren't doing that, you should. Looking at the config.ru, you are calling DB.disconnect in each worker process, instead of in the parent process before forking. Try adding DB.disconnect at the very bottom of config.ru. Unfortunately, beyond that I don't have any ideas. I don't use Passenger myself (I use either Puma or Nginx/Unicorn depending on the application).

I tried your config.ru with Puma, and requests for both /eager and /eager-concurrent result in the same queries. With /eager-concurrent, a separate connection is used for one of the queries (you can more easily see that by using DB.log_connection_info = true.

If you do find out what the issue is with Passenger, please report back here.

[joegsn]

You got me thinking about the spawn mode for Passenger because of that. It does seem to be specific to Passenger's "smart" spawning mode which does some forking instead of multiple ruby instances in order to improve loading performance. Their "direct" spawning mode (similar to Puma) doesn't encounter this issue.

After realizing that, I was suspecting that it might have been related to the extension :async_thread_pool, since it must have some sort of data, and perhaps that isn't fork-safe.

I removed my inclusion of the line

DB.extension :async_thread_pool

from the top where all the other extensions/plugins were setup and put it down in the Passenger event block:

if defined?(PhusionPassenger)
  PhusionPassenger.on_event(:starting_worker_process) do |forked|
    if forked
      # We're in smart spawning mode; behave nice.
      DB.disconnect
    else
      # We're in conservative spawning mode. We don't need to do anything.
    end
    # make sure to load the async thread pool extension finally
    DB.extension :async_thread_pool
  end
else
  # make sure to load the async thread pool extension in non-passenger.
  DB.extension :async_thread_pool
end

This seems to work.

Unless there's a reason I shouldn't do this, I'm going to run with it.

Thanks!

[jeremyevans]

That sounds good. Looks like this behavior is expected. The async thread pool sets up threads when initialized, if you fork afterwrad, those threads go away as forking only forks the current thread. I should update the async_thread_pool documentation to specify that. I'll take care of that today.

[joegsn]

Works for me. Glad that it's a valid solution.