Allow setting Expiration Policy on Subscriptions created via Pub Sub Binder

[jmitash]

When Pub Sub subscriptions are created, if no expiration policy is specified they expire by default in 31 days. In the setup I am working with, it is common for there to be little activity for long periods of times on some streams. For some of these streams, we spin up consumers reactively once a batch of messages are queued in the subscription (as opposed to having them idle year-round). However, because the subscription is quiet for so long, sometimes it is automatically deleted in between batches. Since there is no way to set the expiration policy on the Binder's auto-created subscriptions, I must create them externally.

I would like to be able to configure the expiration policy for auto-created subscriptions so I don't have to rely on external setup.

If desired, I am happy to contribute this change. I have already prepared it for my own use: main...jmitash:spring-cloud-gcp:feat/pubsub-binder-expiration-config.

[meltsufin]

@jmitash Thanks for bringing this up. I didn't realize there was this 31 day expiration default. What's the error message that you get when this happens?

I think this is a very reasonable feature request. @burkedavison WDYT?

[jmitash]

According to the docs, basically if anything is connecting to the subscription, it counts as activity and resets the deletion timer:

If Pub/Sub detects subscriber activity or if you update any of the subscription properties, the subscription deletion clock restarts. Examples of subscriber activities include open connections, active pulls, or successful pushes.

So I don't think people would see an error because the subscription should not be expired while still in active use. But I'm not 100% certain of that because I'm not sure how exactly they count those things (e.g. what makes a pull "active"? do only new connections count?).

In my case, these streams that are deleted are related to a job initiated by a manual CSV upload. The job is only run a few times a year. So instead of leaving our SCS applications running idle 99% of the time, we set them up so that they start off scaled to zero until there is a backlog of messages. This is done by a different tool which uses the Google Cloud Monitoring API, so it does not count as subscription activity. Because the normal state is not running at all, nothing is pulling when the subscription is deleted, the subscriptions reach the 31 day expiration and get deleted. But because once the application does start up and creates all the subscriptions again, there is no error, but the messages that should have gone to that subscription before then are lost.

[burkedavison]

SGTM

[meltsufin]

@jmitash Thanks for the explanation. If you are comfortable opening a pull request for this, we will take a look.

[burkedavison]

Fixed in #2897