Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Connectors page breaks when one connect cluster unavailable #3977

Closed
4 tasks done
stalbot15 opened this issue Jun 26, 2023 · 1 comment · Fixed by #4061
Closed
4 tasks done

Connectors page breaks when one connect cluster unavailable #3977

stalbot15 opened this issue Jun 26, 2023 · 1 comment · Fixed by #4061
Assignees
@iliax
Labels
scope/backend status/accepted An issue which has passed triage and has been accepted type/bug Something isn't working
Milestone
0.8

Comments

@stalbot15
Copy link

stalbot15 commented Jun 26, 2023 •
edited
Loading

Issue submitter TODO list

  • I've looked up my issue in FAQ
  • I've searched for an already existing issues here
  • I've tried running master-labeled docker image and the issue still persists there
  • I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

If any configured Kafka Connect Cluster is unavailable, the entire Connectors page fails to load. After a few retries, the uncaught exception causes the rendering to break entirely.

Expected behavior

The page should render and any unreachable/unavailable connect clusters should simply be omitted from the list of connectors.

We use one connect cluster per connector, and thus have many clusters that we would like to monitor in the UI. If one cluster is unavailable, we still want to be able to see the rest without having to remove the offending cluster from the configuration.

Your installation details

App version: 0.7.0

Basic configurations, with an intentionally unavailable connect address: http://not.available:8083

Steps to reproduce

  1. Run image docker run -p 8080:8080 -e DYNAMIC_CONFIG_ENABLED=true provectuslabs/kafka-ui:master
  2. Configure valid kafka cluster (I did this via the UI, but you could pass in however you like)
  3. Configure invalid kafka connect
    1. Name: test
    2. Address: http://not.available.com:8083
  4. Navigate to the Connectors page

Screenshots

No response

Logs

ERROR [reactor-http-epoll-2] o.s.b.a.w.r.e.AbstractErrorWebExceptionHandler: [5eaf4cf2-86]  500 Server Error for HTTP GET "/api/clusters/test/connectors"
org.springframework.web.reactive.function.client.WebClientRequestException: Failed to resolve 'not.available.com' [A(1)] after 2 queries 
        at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:136)

        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
        *__checkpoint ⇢ Request to GET http://not.available.com:8083/connectors [DefaultWebClient]
        *__checkpoint ⇢ Handler com.provectus.kafka.ui.controller.KafkaConnectController#getAllConnectors(String, String, ConnectorColumnsToSortDTO, SortOrderDTO, ServerWebExchange) [DispatcherHandler]
        *__checkpoint ⇢ com.provectus.kafka.ui.config.CorsGlobalConfiguration$$Lambda$1304/0x00000008017f1878 [DefaultWebFilterChain]
        *__checkpoint ⇢ com.provectus.kafka.ui.config.CustomWebFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ com.provectus.kafka.ui.config.ReadOnlyModeFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ AuthorizationWebFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ ExceptionTranslationWebFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ LogoutWebFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ ServerRequestCacheWebFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ ReactorContextWebFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ HttpHeaderWriterWebFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain]
        *__checkpoint ⇢ org.springframework.web.filter.reactive.ServerHttpObservationFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ HTTP GET "/api/clusters/test/connectors" [ExceptionHandlingWebHandler]
Original Stack Trace:
                at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:136)
                at reactor.core.publisher.MonoErrorSupplied.subscribe(MonoErrorSupplied.java:55)
                at reactor.core.publisher.Mono.subscribe(Mono.java:4485)
                at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)
                at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:222)
                at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:222)
                at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:222)
                at reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93)
                at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onError(MonoFlatMapMany.java:204)
                at reactor.core.publisher.SerializedSubscriber.onError(SerializedSubscriber.java:124)
                at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.whenError(FluxRetryWhen.java:225)
                at reactor.core.publisher.FluxRetryWhen$RetryWhenOtherSubscriber.onError(FluxRetryWhen.java:274)
                at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onError(FluxContextWrite.java:121)
                at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.maybeOnError(FluxConcatMapNoPrefetch.java:326)
                at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.onNext(FluxConcatMapNoPrefetch.java:211)
                at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107)
                at reactor.core.publisher.SinkManyEmitterProcessor.drain(SinkManyEmitterProcessor.java:471)
                at reactor.core.publisher.SinkManyEmitterProcessor$EmitterInner.drainParent(SinkManyEmitterProcessor.java:615)
                at reactor.core.publisher.FluxPublish$PubSubInner.request(FluxPublish.java:602)
                at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.request(FluxContextWrite.java:136)
                at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.request(FluxConcatMapNoPrefetch.java:336)
                at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.request(FluxContextWrite.java:136)
                at reactor.core.publisher.Operators$DeferredSubscription.request(Operators.java:1717)
                at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onError(FluxRetryWhen.java:192)
                at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:201)
                at reactor.netty.http.client.HttpClientConnect$MonoHttpConnect$ClientTransportSubscriber.onError(HttpClientConnect.java:311)
                at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:201)
                at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onError(DefaultPooledConnectionProvider.java:162)
                at reactor.netty.internal.shaded.reactor.pool.AbstractPool$Borrower.fail(AbstractPool.java:475)
                at reactor.netty.internal.shaded.reactor.pool.SimpleDequePool.lambda$drainLoop$9(SimpleDequePool.java:429)
                at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onError(FluxDoOnEach.java:186)
                at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:201)
                at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnectionAllocator$PooledConnectionInitializer.onError(DefaultPooledConnectionProvider.java:560)
                at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:241)
                at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:315)
                at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:106)
                at reactor.core.publisher.Operators.error(Operators.java:198)
                at reactor.core.publisher.MonoError.subscribe(MonoError.java:53)
                at reactor.core.publisher.Mono.subscribe(Mono.java:4485)
                at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)
                at reactor.netty.transport.TransportConnector$MonoChannelPromise.tryFailure(TransportConnector.java:587)
                at reactor.netty.transport.TransportConnector.lambda$doResolveAndConnect$11(TransportConnector.java:394)
                at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
                at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
                at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
                at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
                at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
                at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:110)
                at io.netty.resolver.InetSocketAddressResolver$2.operationComplete(InetSocketAddressResolver.java:86)
                at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
                at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
                at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
                at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
                at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
                at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
                at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118)
                at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1096)
                at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(DnsResolveContext.java:1035)
                at io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:422)
                at io.netty.resolver.dns.DnsResolveContext.onResponse(DnsResolveContext.java:655)
                at io.netty.resolver.dns.DnsResolveContext.access$500(DnsResolveContext.java:66)
                at io.netty.resolver.dns.DnsResolveContext$2.operationComplete(DnsResolveContext.java:482)
                at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
                at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
                at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
                at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
                at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
                at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625)
                at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105)
                at io.netty.resolver.dns.DnsQueryContext.trySuccess(DnsQueryContext.java:232)
                at io.netty.resolver.dns.DnsQueryContext.finish(DnsQueryContext.java:224)
                at io.netty.resolver.dns.DnsNameResolver$DnsResponseHandler.channelRead(DnsNameResolver.java:1364)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
                at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
                at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
                at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
                at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
                at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
                at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
                at io.netty.channel.epoll.EpollDatagramChannel.processPacket(EpollDatagramChannel.java:662)
                at io.netty.channel.epoll.EpollDatagramChannel.recvmsg(EpollDatagramChannel.java:697)
                at io.netty.channel.epoll.EpollDatagramChannel.access$300(EpollDatagramChannel.java:56)
                at io.netty.channel.epoll.EpollDatagramChannel$EpollDatagramChannelUnsafe.epollInReady(EpollDatagramChannel.java:536)
                at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
                at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
                at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
                at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
                at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
                at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.net.UnknownHostException: Failed to resolve 'not.available.com' [A(1)] after 2 queries 
        at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1088)
        at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(DnsResolveContext.java:1035)
        at io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:422)
        at io.netty.resolver.dns.DnsResolveContext.onResponse(DnsResolveContext.java:655)
        at io.netty.resolver.dns.DnsResolveContext.access$500(DnsResolveContext.java:66)
        at io.netty.resolver.dns.DnsResolveContext$2.operationComplete(DnsResolveContext.java:482)
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
        at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
        at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625)
        at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105)
        at io.netty.resolver.dns.DnsQueryContext.trySuccess(DnsQueryContext.java:232)
        at io.netty.resolver.dns.DnsQueryContext.finish(DnsQueryContext.java:224)
        at io.netty.resolver.dns.DnsNameResolver$DnsResponseHandler.channelRead(DnsNameResolver.java:1364)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at io.netty.channel.epoll.EpollDatagramChannel.processPacket(EpollDatagramChannel.java:662)
        at io.netty.channel.epoll.EpollDatagramChannel.recvmsg(EpollDatagramChannel.java:697)
        at io.netty.channel.epoll.EpollDatagramChannel.access$300(EpollDatagramChannel.java:56)
        at io.netty.channel.epoll.EpollDatagramChannel$EpollDatagramChannelUnsafe.epollInReady(EpollDatagramChannel.java:536)
        at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:833)

Additional context

No response
@stalbot15 stalbot15 added status/triage Issues pending maintainers triage type/bug Something isn't working labels Jun 26, 2023
@github-actions
Copy link

Hello there stalbot15! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀

@Haarolean Haarolean added scope/backend status/accepted An issue which has passed triage and has been accepted and removed status/triage Issues pending maintainers triage labels Jun 27, 2023
@iliax iliax linked a pull request Jul 25, 2023 that will close this issue
BE: KC: Suppress unavailable connects #4061
Merged
13 tasks
@iliax iliax self-assigned this Jul 25, 2023
@Haarolean Haarolean added this to the 0.8 milestone Aug 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@iliax iliax

Labels
scope/backend status/accepted An issue which has passed triage and has been accepted type/bug Something isn't working
Projects
None yet
Milestone
0.8
Development

Successfully merging a pull request may close this issue.

BE: KC: Suppress unavailable connects provectus/kafka-ui
3 participants
@iliax @Haarolean @stalbot15