Add connection pooling

[kevinnoel-be]

Description:
Reuse of connections created per database (configured or discovered) vs current behavior to create & close connection per database on each scrape.

Link to tracking Issue:
#30831

Testing:
Updated unit & integration tests. Also, ran locally multiple scenario:

• no feature gate specified (default): current behavior maintained, connections created/closed on each database per scrape
• feature gate connection pool enabled, no connection pool config specified (default): reduction of the number of connections created/closed
• feature gate connection pool enabled, connection pool config tweaked: connections created on first scrape & closed when configured lifetime reached or collector shutdown

Documentation:

• change log
• readme for the feature gate & related optional configurations linked to this feature

Note
Checking internally for getting the CLA signed

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: kevinnoel-be / name: Kevin N. (380908b)

[djaglowski]

LGTM, just one nit

[djaglowski]

@kevinnoel-be, please sign the CLA

[kevinnoel-be]

@kevinnoel-be, please sign the CLA

As mentioned in the description, working on it internally to get it signed, please bear with us 😅

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[kevinnoel-be]

@djaglowski Thanks for you patience, the CLA is now signed!

[djaglowski]

Thanks @kevinnoel-be. Code looks generally good to me but I think some recent changes upstream have broken compilation: client_factory.go:42:18: cfg.NetAddr undefined

[kevinnoel-be]

@djaglowski My bad, didn't check much on my side. Ran tests + integration tests locally before (properly) pushing again this time 😅

[djaglowski]

My bad, didn't check much on my side.

No worries. There's a decent gap between local tooling and CI, and lots of unrelated activity causing minor conflicts, so it can be very difficult.

Looks like one more check: Generated code is out of date, please run "make generate" and commit the changes in this PR.

[kevinnoel-be]

Looks like one more check: Generated code is out of date, please run "make generate" and commit the changes in this PR.

Uh, I guess there's an order to things. I did run gci which changed those imports, maybe needs to be fixed in some template to avoid this? I'll take a look

[djaglowski]

Hmm, I think this may be related to recent changes in out generated test code. I ran make generate locally on this branch and get the same incorrect format. cc: @atoulme

@@ -9,10 +9,12 @@ import (
 	"github.com/stretchr/testify/require"
 	"go.opentelemetry.io/collector/component"
 	"go.opentelemetry.io/collector/component/componenttest"
-	"go.opentelemetry.io/collector/confmap/confmaptest"
+
 	"go.opentelemetry.io/collector/consumer/consumertest"
 	"go.opentelemetry.io/collector/receiver"
 	"go.opentelemetry.io/collector/receiver/receivertest"
+
+	"go.opentelemetry.io/collector/confmap/confmaptest"
 )
 
 // assertNoErrorHost implements a component.Host that asserts that there were no errors.
Generated code is out of date, please run "make generate" and commit the changes in this PR.

link to failure here

[atoulme]

You installed mdatagen from core - try make generate in contrib. I'll harmonize the differences between the repos asap.

That or we still have issues with fmt after generation, I thought we had that figured out.

[djaglowski]

Thanks @atoulme. Unfortunately I ran make generate in contrib (and did again just now with the same result).

➜  opentelemetry-collector-contrib (config-db-pool) make generate
cd cmd/mdatagen && go install .
/Applications/Xcode.app/Contents/Developer/usr/bin/make for-all CMD="go generate ./..."
running go generate ./... in root
running go generate ./... in ./cmd/configschema
...
running go generate ./... in ./receiver/postgresqlreceiver
...
running go generate ./... in ./testbed
running go generate ./... in ./testbed/mockdatasenders/mockdatadogagentexporter
➜  opentelemetry-collector-contrib (config-db-pool) git diff                                                                                                                             ✭
diff --git a/receiver/postgresqlreceiver/generated_component_test.go b/receiver/postgresqlreceiver/generated_component_test.go
index 42ffa42608..9b09d44ee2 100644
--- a/receiver/postgresqlreceiver/generated_component_test.go
+++ b/receiver/postgresqlreceiver/generated_component_test.go
@@ -9,10 +9,12 @@ import (
        "github.com/stretchr/testify/require"
        "go.opentelemetry.io/collector/component"
        "go.opentelemetry.io/collector/component/componenttest"
-       "go.opentelemetry.io/collector/confmap/confmaptest"
+
        "go.opentelemetry.io/collector/consumer/consumertest"
        "go.opentelemetry.io/collector/receiver"
        "go.opentelemetry.io/collector/receiver/receivertest"
+
+       "go.opentelemetry.io/collector/confmap/confmaptest"
 )

[atoulme]

You might want to rebase off the latest main, at least with #31520 the generated files are impacted.

[djaglowski]

I rebased locally and unfortunately get the same result

[kevinnoel-be]

AFAIS the order we see in CI and locally when running generate seems to correspond with what is in the template. I had the "good" idea to run gci target at some point after the generate one which is why this check is failing (as the template doesn't seem to have the "proper" import order and gci didn't ignore the generated file).

TL;DR I'll rebase on Monday after a make generate to fix this PR. To avoid any similar issue I guess the generated template should follow the imposed/configured imports order.

[kevinnoel-be]

Rebased, ran make generate and didn't had any differences this time. Seems this got fixed in the template (spacing)