ci(jenkins): release process

[v1v]

Highlights

• enable ITs: async when running as a PR or sync when running in a branch/tag
• Continuous delivery when running the pipeline either in a branch or tag. master branch refers to latest while others will name the image with their names.
• Script to bump versions for the Node.js agent.

Issues

• Depends on:
  • feat: enable opbeans customisation apm-integration-testing#556
  • release process for opbeans apm-pipeline-library#271
  • ci(jenkins): trigger opbeans release process apm-agent-nodejs#1553

Tasks

• Versioning
• Release automation
• ITs validation should be synchronous when running a release.

[watson]

I don't think you need to run npm install again as long as you just ensure to build a new Docker image as the version dependency is already specified as a range. If we do that you also don't risk upgrading to a version with a higher major component as that will be a breaking change.

[v1v]

I don't think you need to run npm install again as long as you just ensure to build a new Docker image as the version dependency is already specified as a range. If we do that you also don't risk upgrading to a version with a higher major component as that will be a breaking change.

The idea of this particular approach is to ensure the version for the apm-agent-nodejs which has been shipped got an opbeans docker image with it. Therefore we got reproducibility, using ranges might be good for certain cases, but for this particular one, there is a requirement to ensure all the releases for all the apm-agents got a specific opbeans docker image matching with the same version.

The next follow-ups will consist of running some functional tests to validate whether those dependencies are not breaking anything. The one which will do that is the:

• https://github.com/elastic/apm-integration-testing/blob/master/.ci/scripts/opbeans.sh

It does run something but requires to add more test coverage for those use cases.

At the moment, the automation is in place, we need to add more test coverage to be in a more healthy place.

Not sure if I answered your concerns, please let me know If I need to clarify it.

[watson]

What I mean is that I think it's redundant. If I understand the process of how a new Docker image is being build, the 3 steps (npm install ..., git add ..., and git commit ...) doesn't do any difference in how the Docker image turns out as long as the Docker image is rebuilt, it will be built with a fresh npm install and hence get the latest version allowed in the range specified in package.json.

I would always recommend doing a manual upgrade between each major version of the agent, as it most likely involves manual checking and upgrading of code due to the breaking changes.

But maybe I misunderstood how the Docker images are built?

[kuisathaverat]

On the Docker, file installs the version that it is configured in the package.json, this means you have to manually edit this file before making an Opbeans release to set a fixed version for the Node.jas agent, the automated process that @v1v is putting in place make this change and trigger a push to build a new Docker image.

[watson]

In this repository, we don't have a checked-in package-lock.json file, so as long as you're running npm install from a freshly cloned version of the repo, you will get the latest version of the agent within the range specified in package.json. So if we specify ^3.0.0 in package.json, we'll get the latest 3.x version.

You can verify this by running npm ls elastic-apm-node after finishing the npm install. That will show the actual version that has been installed.

[watson]

Just had a chat with @v1v. While the bump-version.sh script would technically work without manually running npm install elastic-apm-node@${AGENT_VERSION} it would only work for installing whatever is the latest version at the time the script runs.

There can be race conditions where two versions of the Node agent has been published and then later this script is being run twice, both times installing the latest of those two versions. That's obviously not a good thing, so being very specific about the version like the script is now is a good idea.

However, for this to be 100% dependent, you also need to change the version defined in package.json to no longer be a range:

-    "elastic-apm-node": "^3.0.0",
+    "elastic-apm-node": "3.0.0",

By removing the ^ sign, you ensure that only the version specified will be installed. If you don't do this, running npm install elastic-apm-node@3.1.0 would just change ^3.0.0 to ^3.1.0 which isn't that dependable and might create weird issues down the line.

[watson]

For details of what these prefixes to the version (like ^) means in package.json, see: https://docs.npmjs.com/files/package.json#dependencies