[Enhancement]: Upgrade Pulsar to 3.x

[haorenfsa]

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

Upgrade Pulsar to 3.x

Why is this needed?

• Enables Milvus to be run in with Pulsar in OpenShift Make Docker images non-root, by default, and OpenShift compliant apache/pulsar#13376
• Fix a stability issue of pulsar BadVersionException , unable to consume topic, failing to reconnect apache/pulsar#14779

Anything else?

ref:

• Cannot run Milvus in our Openshift cluster with (runsAsUser, runAsGroup, fsGroup) - How to remove them??  zilliztech/milvus-helm#43
• fix: fix permissions on /milvus for OpenShift compatibility #30775

dependency:

• enhance: upgrade pulsar-client-go to 0.11.1 #36435

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[rere950303]

@haorenfsa thanks for your works,
Do you have any plans to increase the pulsar version to 3.0.1? I would like to change the pulsar version used in milvus 2.3.12 to 2.10.0 because of volume directive issues in dockerfile. I don't think there will be any big issues when referring to the minor version release note. Are there any expected issues? I would appreciate it if you could comment briefly.

[xiaofan-luan]

@haorenfsa thanks for your works, Do you have any plans to increase the pulsar version to 3.0.1? I would like to change the pulsar version used in milvus 2.3.12 to 2.10.0 because of volume directive issues in dockerfile. I don't think there will be any big issues when referring to the minor version release note. Are there any expected issues? I would appreciate it if you could comment briefly.

So which version you are targeting to use? 2.10 or 3.0.1? We would recommend to use 3.0.1 since we test it thoroughly

[rere950303]

@xiaofan-luan thanks for answer,
In milvus 2.3.12 version, I was originally going to use the pulsar 2.10.0 version.
do you mean that there is no issue with using the pulsar 3.0.1 version???

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[xiaofan-luan]

keep this.
@haorenfsa is there a deadline for this?

[haorenfsa]

@xiaofan-luan One must first upgrade pulsar to some other 2.x version before one can upgrade to 3.0.1.
@LoveEachDay will write a doc about the upgrading route, before we change the default version to 3.0.1 in helm chart.
/assign @LoveEachDay

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[haorenfsa]

/reopen

[sre-ci-robot]

@haorenfsa: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[haorenfsa]

keep alive

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[haorenfsa]

milvus-operator now supports pulsar v3 zilliztech/milvus-operator#182 (comment)
but only for new milvus instance. Manifest example is in https://github.com/zilliztech/milvus-operator/blob/main/config/samples/milvus_pulsar_v3.yaml.

The upgrade plan from v2 is still under design.

[haorenfsa]

/assign

[haorenfsa]

#36435

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[haorenfsa]

keep alive

[xiaofan-luan]

can we do it in Milvus 2.5?

[haorenfsa]

Yes

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.