Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/redisreceiver] add scraper shutdown #17491

Merged
merged 4 commits into from
Jan 12, 2023

Conversation

matianjun1
Copy link
Contributor

Description:

The redis client has a connPool. When using redisreceiver with receiver_creator, the old client won't be closed. That may lead a memory leak.

Link to tracking Issue:

Testing:

receivers:
  nop:
  receiver_creator:
    watch_observers: [host_observer]
    receivers:
      redis/on_host:
        # If this rule matches an instance of this receiver will be started.
        rule: type == "port" && port == 6379 && is_ipv6 == true
      resource_attributes:
        service.name: redis_on_host

processors:
  nop:

exporters:
  nop:

service:
  pipelines:
    extensions: [host_observer]
    metrics/nop:
      receivers: [nop]
      processors: [nop]
      exporters: [nop]
    metrics/redis:
      receivers: [receiver_creator]
      processors: [nop]
      exporters: [nop]

Documentation:

github.com/go-redis/redis/v7@v7.4.1/redis.go

// Close closes the client, releasing any open resources.
//
// It is rare to Close a Client, as the Client is meant to be
// long-lived and shared between many goroutines.
func (c *baseClient) Close() error {
	var firstErr error
	if c.onClose != nil {
		if err := c.onClose(); err != nil {
			firstErr = err
		}
	}
	if err := c.connPool.Close(); err != nil && firstErr == nil {
		firstErr = err
	}
	return firstErr
}

@matianjun1 matianjun1 requested a review from a team January 10, 2023 11:53
@github-actions github-actions bot added the receiver/redis Redis related issues label Jan 10, 2023
@runforesight
Copy link

runforesight bot commented Jan 10, 2023

Foresight Summary

    
Major Impacts

build-and-test-windows duration(3 seconds) has decreased 44 minutes 7 seconds compared to main branch avg(44 minutes 10 seconds).
View More Details

⭕  build-and-test-windows workflow has finished in 3 seconds (44 minutes 7 seconds less than main branch avg.) and finished at 12th Jan, 2023.


Job Failed Steps Tests
windows-unittest-matrix -     🔗  N/A See Details
windows-unittest -     🔗  N/A See Details

✅  telemetrygen workflow has finished in 50 seconds (2 minutes 4 seconds less than main branch avg.) and finished at 12th Jan, 2023.


Job Failed Steps Tests
build-dev -     🔗  N/A See Details
publish-latest -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  tracegen workflow has finished in 1 minute 11 seconds (1 minute 21 seconds less than main branch avg.) and finished at 12th Jan, 2023.


Job Failed Steps Tests
build-dev -     🔗  N/A See Details
publish-latest -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  check-links workflow has finished in 50 seconds (1 minute 12 seconds less than main branch avg.) and finished at 12th Jan, 2023.


Job Failed Steps Tests
changed files -     🔗  N/A See Details
check-links -     🔗  N/A See Details

✅  changelog workflow has finished in 2 minutes 6 seconds and finished at 12th Jan, 2023.


Job Failed Steps Tests
changelog -     🔗  N/A See Details

✅  prometheus-compliance-tests workflow has finished in 3 minutes 43 seconds (4 minutes 26 seconds less than main branch avg.) and finished at 12th Jan, 2023.


Job Failed Steps Tests
prometheus-compliance-tests -     🔗  ✅ 21  ❌ 0  ⏭ 0    🔗 See Details

✅  load-tests workflow has finished in 7 minutes 49 seconds (7 minutes 21 seconds less than main branch avg.) and finished at 12th Jan, 2023.


Job Failed Steps Tests
loadtest (TestTraceAttributesProcessor) -     🔗  ✅ 3  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestIdleMode) -     🔗  ✅ 1  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceNoBackend10kSPS|TestTrace1kSPSWithAttrs) -     🔗  ✅ 8  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetric10kDPS|TestMetricsFromFile) -     🔗  ✅ 6  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetricResourceProcessor|TestTrace10kSPS) -     🔗  ✅ 12  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceBallast1kSPSWithAttrs|TestTraceBallast1kSPSAddAttrs) -     🔗  ✅ 10  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestBallastMemory|TestLog10kDPS) -     🔗  ✅ 19  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details

 build-and-test workflow has finished in 18 minutes 55 seconds (30 minutes 40 seconds less than main branch avg.) and finished at 12th Jan, 2023.


Job Failed Steps Tests
unittest-matrix (1.19, extension) N/A  ✅ 528  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, internal) N/A  ✅ 636  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, internal) N/A  ✅ 636  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, extension) N/A  ✅ 528  ❌ 0  ⏭ 0    🔗 See Details
correctness-metrics N/A  ✅ 2  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, processor) N/A  ✅ 1481  ❌ 0  ⏭ 0    🔗 See Details
correctness-traces N/A  ✅ 17  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, processor) N/A  ✅ 1481  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-0) N/A  ✅ 2565  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, receiver-0) N/A  ✅ 2565  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, exporter) N/A  ✅ 2464  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, exporter) N/A  ✅ 2464  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, other) N/A  ✅ 4438  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, other) N/A  ✅ 4438  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.18, receiver-1) N/A  ✅ 1884  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-1) N/A  ✅ 1884  ❌ 0  ⏭ 0    🔗 See Details
integration-tests N/A  ✅ 55  ❌ 0  ⏭ 0    🔗 See Details

🔎 See details on Foresight

*You can configure Foresight comments in your organization settings page.

@hughesjj
Copy link
Contributor

Other than the changelog (see .chloggen/TEMPLATE.yaml, copy it to a relevant .yaml in .chloggen with a descriptive name) it LGTM

@@ -25,6 +25,8 @@ type client interface {
// line delimiter
// redis lines are delimited by \r\n, files (for testing) by \n
delimiter() string
// close
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please avoid comments that don't provide any value

@@ -50,3 +52,8 @@ func (c *redisClient) delimiter() string {
func (c *redisClient) retrieveInfo() (string, error) {
return c.client.Info("all").Result()
}

// close.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// close.

@dmitryax dmitryax added the ready to merge Code review completed; ready to merge by maintainers label Jan 12, 2023
@dmitryax dmitryax merged commit 447a32a into open-telemetry:main Jan 12, 2023
@plantfansam plantfansam mentioned this pull request Jul 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready to merge Code review completed; ready to merge by maintainers receiver/redis Redis related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants