Add Elasticsearch helper binaries to release #5501

FlamingSaint · 2024-05-30T03:38:00Z

Which problem is this PR solving?

Resolves [Feature]: Add Elasticsearch helper binaries to GitHub Releases #5499

Description of the changes

Add the es helpers to scripts/package-deploy.sh

How was this change tested?

Checklist

I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
I have signed all commits
I have added unit tests for the new functionality
I have run lint and test steps successfully
- for jaeger: make lint test
- for jaeger-ui: yarn lint and yarn test

Signed-off-by: FlamingSaint <raghuramkannan400@gmail.com>

codecov · 2024-05-30T03:44:39Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.19%. Comparing base (629c47e) to head (fdf2d45).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5501      +/-   ##
==========================================
+ Coverage   96.18%   96.19%   +0.01%     
==========================================
  Files         327      327              
  Lines       16012    16012              
==========================================
+ Hits        15401    15403       +2     
+ Misses        436      435       -1     
+ Partials      175      174       -1

Flag	Coverage Δ
badger_v1	`8.05% <ø> (ø)`
badger_v2	`1.93% <ø> (ø)`
cassandra-3.x-v1	`16.43% <ø> (ø)`
cassandra-3.x-v2	`1.85% <ø> (ø)`
cassandra-4.x-v1	`16.43% <ø> (ø)`
cassandra-4.x-v2	`1.85% <ø> (ø)`
elasticsearch-7-v1	`18.89% <ø> (?)`
elasticsearch-8-v1	`19.07% <ø> (?)`
elasticsearch-8-v2	`19.07% <ø> (?)`
elasticsearch-8.x-v2	`?`
grpc_v1	`9.48% <ø> (ø)`
grpc_v2	`7.53% <ø> (ø)`
kafka	`9.78% <ø> (ø)`
opensearch-1-v1	`18.94% <ø> (?)`
opensearch-1.x-v1	`?`
opensearch-2-v1	`18.94% <ø> (?)`
opensearch-2-v2	`18.92% <ø> (?)`
opensearch-2.x-v1	`?`
opensearch-2.x-v2	`?`
unittests	`93.89% <ø> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

yurishkuro · 2024-05-30T04:39:19Z

this workflow does not run in the CI, so how was this tested?

FlamingSaint · 2024-05-30T16:23:08Z

I am not sure how to test this. The only solution I can come up with is to make a test release and see if it those es-helpers are included in the tar (Right now they would be included in the same tar). Hopefully there is a better way to test this ?

yurishkuro · 2024-05-30T16:29:37Z

run scripts/package-deploy.sh script with redirecting logs to a file and upload the log file here

FlamingSaint · 2024-05-30T16:50:23Z

logfile.log

Signed-off-by: FlamingSaint <raghuramkannan400@gmail.com>

FlamingSaint · 2024-05-30T22:39:08Z

How do I proceed to test this ?

yurishkuro · 2024-05-30T23:23:28Z

I think we introduced a bug recently to the Makefile, change it to

diff --git a/Makefile b/Makefile
index 7055dad4..ff5440b5 100644
--- a/Makefile
+++ b/Makefile
@@ -95,7 +95,7 @@ test-and-lint: test fmt lint

 .PHONY: echo-version
 echo-version:
-       @echo "GIT_CLOSEST_TAG=$(GIT_CLOSEST_TAG)"
+       @echo "$(GIT_CLOSEST_TAG)"

 .PHONY: echo-all-pkgs
 echo-all-pkgs:

and you will need to run make build-all-platforms before the packaging script.

FlamingSaint · 2024-05-30T23:36:27Z

There seems to be an error when I run make build-all-platforms .

fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
GOOS=linux GOARCH=amd64 make _build-platform-binaries
make[1]: Entering directory '/home/raghuram/OpenSource/jaeger'
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath   -o ./cmd/agent/agent-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/agent
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath  -tags ui -o ./cmd/all-in-one/all-in-one-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/all-in-one
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath   -o ./cmd/collector/collector-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/collector
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath  -tags ui -o ./cmd/query/query-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/query
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath   -o ./cmd/ingester/ingester-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/ingester
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath  -tags ui -o ./cmd/jaeger/jaeger-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/jaeger
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath   -o ./cmd/remote-storage/remote-storage-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/remote-storage
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -o ./examples/hotrod/hotrod-linux-amd64 ./examples/hotrod/main.go
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" -o ./cmd/tracegen/tracegen-linux-amd64 ./cmd/tracegen/
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" -o ./cmd/anonymizer/anonymizer-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/anonymizer/
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -o ./plugin/storage/es/esmapping-generator-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/esmapping-generator/
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -o ./cmd/es-index-cleaner/es-index-cleaner-linux-amd64 ./cmd/es-index-cleaner/
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -o ./cmd/es-rollover/es-rollover-linux-amd64 ./cmd/es-rollover/
make _build-platform-binaries-debug GOOS=linux GOARCH=amd64 DEBUG_BINARY=1
make[2]: Entering directory '/home/raghuram/OpenSource/jaeger'
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -gcflags="all=-N -l"  -o ./cmd/agent/agent-debug-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/agent
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -gcflags="all=-N -l"  -o ./cmd/collector/collector-debug-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/collector
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -gcflags="all=-N -l" -tags ui -o ./cmd/query/query-debug-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/query
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -gcflags="all=-N -l"  -o ./cmd/ingester/ingester-debug-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/ingester
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -gcflags="all=-N -l"  -o ./cmd/remote-storage/remote-storage-debug-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/remote-storage
fatal: No names found, cannot describe anything.
CGO_ENABLED=0 installsuffix=cgo go build -trimpath -gcflags="all=-N -l" -tags ui -o ./cmd/all-in-one/all-in-one-debug-linux-amd64 -ldflags "-X github.com/jaegertracing/jaeger/pkg/version.commitSHA=665385b6975b0506ab2be0fcd77a7e48d8b3300e -X github.com/jaegertracing/jaeger/pkg/version.latestVersion= -X github.com/jaegertracing/jaeger/pkg/version.date=2024-05-30T22:29:36Z" ./cmd/all-in-one
make[2]: Leaving directory '/home/raghuram/OpenSource/jaeger'
make[1]: Leaving directory '/home/raghuram/OpenSource/jaeger'
make _prepare-winres-helper NAME="Jaeger Agent"            PKGPATH="cmd/agent"
make[1]: Entering directory '/home/raghuram/OpenSource/jaeger'
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
echo $VERSIONINFO | goversioninfo -o="cmd/agent/resource.syso" -
2024/05/31 05:02:21 Could not parse the .json file: invalid character ',' looking for beginning of value
make[1]: *** [Makefile:356: _prepare-winres-helper] Error 2
make[1]: Leaving directory '/home/raghuram/OpenSource/jaeger'
make: *** [Makefile:342: _prepare-winres] Error 2

yurishkuro · 2024-05-31T05:39:23Z

What does this command output for you? git rev-parse --is-shallow-repository
If it prints true, can you make your repo non-shallow?

FlamingSaint · 2024-05-31T06:20:39Z

yurishkuro · 2024-05-31T20:09:17Z

That's odd, my understanding fatal: No names found, cannot describe anything. was logged by git when it's a shallow copy.

yurishkuro · 2024-05-31T20:09:51Z

so make echo-version doesn't print anything for you?

FlamingSaint · 2024-05-31T22:49:27Z

Yes, Apart form the usual fatal messages
.

FlamingSaint · 2024-06-01T06:20:36Z

GIT_SHALLOW_CLONE := $(shell git rev-parse --is-shallow-repository)
# Some of GitHub Actions workflows do a shallow checkout without tags. This avoids logging warnings from git.
GIT_CLOSEST_TAG=$(shell if [ "$(GIT_SHALLOW_CLONE)" = "true" ]; then git describe --abbrev=0 --tags; else echo 0.0.0; fi)

I changed the value of GIT_SHALLOW_CLONE to true and the fatal errors where not there anymore. This is very weird considering that git rev-parse --is-shallow-repository is false.

scripts/package-deploy.sh

FlamingSaint · 2024-06-01T17:02:14Z

I think we introduced a bug recently to the Makefile, change it to

diff --git a/Makefile b/Makefile
index 7055dad4..ff5440b5 100644
--- a/Makefile
+++ b/Makefile
@@ -95,7 +95,7 @@ test-and-lint: test fmt lint

 .PHONY: echo-version
 echo-version:
-       @echo "GIT_CLOSEST_TAG=$(GIT_CLOSEST_TAG)"
+       @echo "$(GIT_CLOSEST_TAG)"

 .PHONY: echo-all-pkgs
 echo-all-pkgs:

and you will need to run make build-all-platforms before the packaging script.

Shall I include this change as well ?

Signed-off-by: FlamingSaint <raghuramkannan400@gmail.com>

yurishkuro · 2024-06-01T18:12:29Z

yes

yurishkuro · 2024-06-01T18:13:34Z

Thanks

## Which problem is this PR solving? - Resolves jaegertracing#5499 ## Description of the changes - Add the es helpers to scripts/package-deploy.sh ## How was this change tested? - ## Checklist - [x] I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md - [x] I have signed all commits - [ ] I have added unit tests for the new functionality - [x] I have run lint and test steps successfully - for `jaeger`: `make lint test` - for `jaeger-ui`: `yarn lint` and `yarn test` --------- Signed-off-by: FlamingSaint <raghuramkannan400@gmail.com> Signed-off-by: Vamshi Maskuri <gwcchintu@gmail.com>

## Which problem is this PR solving? - A bug was introduced in #5501 where the wrong binary path was used ## Description of the changes - Use the right path - Allow overriding `tar` command since on MacOS the `--sort` option is not supported (so I had to use `gtar` to test) ## How was this change tested? These steps succeeded locally: ``` $ make build-all-platforms $ TARCMD=gtar bash scripts/package-deploy.sh ``` Signed-off-by: Yuri Shkuro <github@ysh.us>

FlamingSaint added 2 commits May 30, 2024 09:03

Add ElasticSearch helper binaries to release

3e9dd5a

Signed-off-by: FlamingSaint <raghuramkannan400@gmail.com>

Remove extra space

94a6703

Signed-off-by: FlamingSaint <raghuramkannan400@gmail.com>

FlamingSaint requested a review from a team as a code owner May 30, 2024 03:38

FlamingSaint requested a review from joe-elliott May 30, 2024 03:38

yurishkuro added the changelog:bugfix-or-minor-feature label May 30, 2024

FlamingSaint added 2 commits May 31, 2024 03:56

Create a separate tools tar for es-tools

da239ca

Signed-off-by: FlamingSaint <raghuramkannan400@gmail.com>

Fix indentation

665385b

Signed-off-by: FlamingSaint <raghuramkannan400@gmail.com>

Merge branch 'jaegertracing:main' into es_release

94e3f06

yurishkuro reviewed Jun 1, 2024

View reviewed changes

scripts/package-deploy.sh Outdated Show resolved Hide resolved

yurishkuro reviewed Jun 1, 2024

View reviewed changes

scripts/package-deploy.sh Outdated Show resolved Hide resolved

Make suggested changes

fdf2d45

Signed-off-by: FlamingSaint <raghuramkannan400@gmail.com>

yurishkuro approved these changes Jun 1, 2024

View reviewed changes

yurishkuro merged commit ca50f53 into jaegertracing:main Jun 1, 2024
40 checks passed

FlamingSaint deleted the es_release branch June 3, 2024 21:40

yurishkuro mentioned this pull request Jun 11, 2024

Fix binary path in package-deploy.sh #5561

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Elasticsearch helper binaries to release #5501

Add Elasticsearch helper binaries to release #5501

FlamingSaint commented May 30, 2024

codecov bot commented May 30, 2024 •

edited

Loading

yurishkuro commented May 30, 2024

FlamingSaint commented May 30, 2024 •

edited

Loading

yurishkuro commented May 30, 2024

FlamingSaint commented May 30, 2024 •

edited

Loading

FlamingSaint commented May 30, 2024

yurishkuro commented May 30, 2024

FlamingSaint commented May 30, 2024

yurishkuro commented May 31, 2024

FlamingSaint commented May 31, 2024

yurishkuro commented May 31, 2024

yurishkuro commented May 31, 2024

FlamingSaint commented May 31, 2024

FlamingSaint commented Jun 1, 2024

FlamingSaint commented Jun 1, 2024

yurishkuro commented Jun 1, 2024

yurishkuro commented Jun 1, 2024

Add Elasticsearch helper binaries to release #5501

Add Elasticsearch helper binaries to release #5501

Conversation

FlamingSaint commented May 30, 2024

Which problem is this PR solving?

Description of the changes

How was this change tested?

Checklist

codecov bot commented May 30, 2024 • edited Loading

Codecov Report

yurishkuro commented May 30, 2024

FlamingSaint commented May 30, 2024 • edited Loading

yurishkuro commented May 30, 2024

FlamingSaint commented May 30, 2024 • edited Loading

FlamingSaint commented May 30, 2024

yurishkuro commented May 30, 2024

FlamingSaint commented May 30, 2024

yurishkuro commented May 31, 2024

FlamingSaint commented May 31, 2024

yurishkuro commented May 31, 2024

yurishkuro commented May 31, 2024

FlamingSaint commented May 31, 2024

FlamingSaint commented Jun 1, 2024

FlamingSaint commented Jun 1, 2024

yurishkuro commented Jun 1, 2024

yurishkuro commented Jun 1, 2024

codecov bot commented May 30, 2024 •

edited

Loading

FlamingSaint commented May 30, 2024 •

edited

Loading

FlamingSaint commented May 30, 2024 •

edited

Loading