Failed to create subscription

[xiaobing0508]

Failed to create consumer for topic partition-3.
Other partition is normal.
pulsar-broker.log:
16:27:13.669 [pulsar-io-4-38] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [product/thor/persistent/data_out-partition-3] Cursor data_out_g_flt_xgks_test recovered to position 99681:294407

16:27:13.669 [pulsar-io-4-38] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [product/thor/persistent/data_out-partition-3] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 114, 111, 100, 117, 99, 116, 47, 116, 104, 111, 114, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 100, 97, 116, 97, 95, 111, 117, 116, 45, 112, 97, 114, 116, 105, 116, 105, 111, 110, 45, 51], pulsar/cursor=[100, 97, 116, 97, 95, 111, 117, 116, 95, 103, 95, 102, 108, 116, 95, 120, 103, 107, 115, 95, 116, 101, 115, 116], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds

16:27:13.684 [BookKeeperClientWorker-OrderedExecutor-77-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [product/thor/persistent/data_out-partition-3] [data_out_g_flt_xgks_test] Updating cursor info ledgerId=99725 mark-delete=99681:294407

16:27:13.687 [bookkeeper-ml-scheduler-OrderedScheduler-4-0] WARN org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [product/thor/persistent/data_out-partition-3] Failed to update cursor metadata for data_out_g_flt_xgks_test due to version conflict org.apache.pulsar.metadata.api.MetadataStoreException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /managed-ledgers/product/thor/persistent/data_out-partition-3/data_out_g_flt_xgks_test

16:27:13.687 [bookkeeper-ml-scheduler-OrderedScheduler-4-0] WARN org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [product/thor/persistent/data_out-partition-3] Failed to update consumer data_out_g_flt_xgks_test
org.apache.bookkeeper.mledger.ManagedLedgerException$BadVersionException: org.apache.pulsar.metadata.api.MetadataStoreException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /managed-ledgers/product/thor/persistent/data_out-partition-3/data_out_g_flt_xgks_test

16:27:13.688 [bookkeeper-ml-scheduler-OrderedScheduler-4-0] WARN org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [product/thor/persistent/data_out-partition-3] Failed to open cursor: ManagedCursorImpl{ledger=product/thor/persistent/data_out-partition-3, name=data_out_g_flt_xgks_test, ackPos=99681:294407, readPos=99681:294408}

16:27:13.688 [bookkeeper-ml-scheduler-OrderedScheduler-4-0] WARN org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://product/thor/data_out-partition-3] Failed to create subscription for data_out_g_flt_xgks_test: org.apache.pulsar.metadata.api.MetadataStoreException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /managed-ledgers/product/thor/persistent/data_out-partition-3/data_out_g_flt_xgks_test

16:27:13.688 [bookkeeper-ml-scheduler-OrderedScheduler-4-0] ERROR org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://product/thor/data_out-partition-3] Failed to create subscription: data_out_g_flt_xgks_test error: java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException$BadVersionException: org.apache.pulsar.metadata.api.MetadataStoreException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /managed-ledgers/product/thor/persistent/data_out-partition-3/data_out_g_flt_xgks_test

16:27:13.688 [bookkeeper-ml-scheduler-OrderedScheduler-4-0] WARN org.apache.pulsar.broker.service.ServerCnx - [/172.16.224.102:13000][persistent://product/thor/data_out-partition-3][data_out_g_flt_xgks_test] Failed to create consumer: consumerId=3, java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException$BadVersionException: org.apache.pulsar.metadata.api.MetadataStoreException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /managed-ledgers/product/thor/persistent/data_out-partition-3/data_out_g_flt_xgks_test

[shibd]

Could you give us more detail or reproduce steps?

[xiaobing0508]

I don't know what happened,but I create new subscription without that wrong.

[xiaobing0508]

I use pulsar-perf to consume partition-3 with that subscription name.

12:28:12.313 [pulsar-timer-5-1] INFO org.apache.pulsar.client.impl.ConnectionHandler - [persistent://product/thor/data_out-partition-3] [data_out_g_flt_xgks] Reconnecting after connection was closed

12:28:12.316 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x3f2e5cc7, L:/192.168.191.53:58748 - R:192.168.191.53:6650]] Connected to server

12:28:12.321 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x57acae78, L:/192.168.191.53:36470 - R:192.168.191.55/192.168.191.55:6650]] Connected to server

12:28:12.323 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://product/thor/data_out-partition-3][data_out_g_flt_xgks] Subscribing to topic on cnx [id: 0x57acae78, L:/192.168.191.53:36470 - R:192.168.191.55/192.168.191.55:6650], consumerId 3

12:28:12.341 [pulsar-client-io-1-1] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0x57acae78, L:/192.168.191.53:36470 - R:192.168.191.55/192.168.191.55:6650] Received error from server: java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException$BadVersionException: org.apache.pulsar.metadata.api.MetadataStoreException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /managed-ledgers/product/thor/persistent/data_out-partition-3/data_out_g_flt_xgks

12:28:12.342 [pulsar-client-io-1-1] WARN org.apache.pulsar.client.impl.ConsumerImpl - [persistent://product/thor/data_out-partition-3][data_out_g_flt_xgks] Failed to subscribe to topic on 192.168.191.55/192.168.191.55:6650

12:28:12.342 [pulsar-client-io-1-1] WARN org.apache.pulsar.client.impl.ConnectionHandler - [persistent://product/thor/data_out-partition-3] [data_out_g_flt_xgks] Could not get connection to broker: java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException$BadVersionException: org.apache.pulsar.metadata.api.MetadataStoreException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /managed-ledgers/product/thor/persistent/data_out-partition-3/data_out_g_flt_xgks -- Will try again in 2.98 s

[xiaobing0508]

What should I do?

[shibd]

Emmm, I'm not really sure how it happened. The subscribe(data_out_g_flt_xgks_test ) can be delete? You can try run the following command to unsubscribe first and then subscribe.

pulsar-admin topics unsubscribe [options] persistent://tenant/namespace/topic

[xiaobing0508]

I can't unsubscribe,Reason is Subscription not found.

[github-actions]

The issue had no activity for 30 days, mark with Stale label.

[github-actions]

The issue had no activity for 30 days, mark with Stale label.