Pin Cloud Pub/Sub dependencies to minor versions

[kamalaboulhosn]

In an attempt to minimize issues caused by bad minor releases in dependencies and to make the rollback procedure for customers easier, we should pin dependencies to a minor version. When new minor versions are released, we should increase the pinned minor version to match the new version in a PR and release a new version of the Pub/Sub client library. If there is no new code in the library, but there are new dependency minor version increments, we should release a new version at least once a month. It is fine to release at a faster cadence than that if desired. This is how things work in the Java library and in fact, the minor version increase PRs are automatically generated. Perhaps thy could be auto generated in Ruby as well.

[dazuma]

@kamalaboulhosn This technique is not common in the Ruby ecosystem. Can you point us to us to specific problems that have resulted from bad minor releases in dependencies or have required rollbacks, so we can understand the issue? (Feel free to send to g/ruby-cloud-eng internally if you cannot share externally.)

[kamalaboulhosn]

@dazuma We have seen issues with this in other languages like Python. We just had an issue this week that was caused by a bad change in an underlying dependency. Previously, we have seen this happen in node.js as well. For customers, this type of issue can be quite difficult to understand.

If we pin the Cloud Pub/Sub client library to a minor version and ensure that we have a release of the client library to increase version numbers, then the path to rollback to a working version of the underlying dependency is much clearer; one just needs to roll back the Pub/Sub client library. It will save back and forth with the customer and also improve MTTR for customer pipelines.

[dazuma]

@kamalaboulhosn This will cause problems in Ruby because Ruby apps use global transitive dependency resolution across all libraries they use. We generally depend on semver semantics to ensure that there is a solution to this resolution. Therefore, if one library breaks semver by pinning to a minor version, it precludes use of any other libraries that happen to depend on another minor version or otherwise declare dependencies that similarly contradict semver. That is, it nullifies the main defense we have against the diamond dependency problem. This is why pinning to minor versions is uncommon, and indeed generally discouraged, in the Ruby community.

We need to discuss this within the client libraries team as a whole, because if we do it for Pubsub, it will impact how we manage dependencies across all our clients.
@quartzmo please hold off on implementing for now.

[dazuma]

For Ruby, if an underlying dependency is broken (and it is outside Google's control to fix it) then users have the ability to pin it themselves (temporarily until a permanent fix is in place) in their own Gemfile. That is the normal practice for Ruby developers, and it would have been the normal Ruby solution for the issue cited above. They do not need downstream libraries attempting to do it a priori for them (and risking doing it differently from other downstream libraries).

[kamalaboulhosn]

It looks like we are going to explore other options for dealing with this issue. Closing.