Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Try fix smoke tests (#5889 -> v2) #5891

Merged
merged 1 commit into from
Aug 13, 2024
Merged

Try fix smoke tests (#5889 -> v2) #5891

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .azure-pipelines/steps/install-docker-compose-v1.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
parameters:
- name: isLinux
type: boolean
default: false

- name: 'dockerComposePath'
type: string
default: 'C:/docker-compose/docker-compose.exe'

steps:
- ${{ if eq(parameters.isLinux, true) }}:
- bash: |
sudo mkdir -p "$(dirname "${{ parameters.dockerComposePath }}")"
sudo curl -SL https://github.com/docker/compose/releases/download/1.29.2/docker-compose-linux-x86_64 -o ${{ parameters.dockerComposePath }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticing on this one that the release versions between linux (v1.29.2) differ from windows (v1.29.1)
Unsure if it matters, just pointing out.

sudo chmod 755 ${{ parameters.dockerComposePath }}
displayName: Download docker-compose
- ${{ else }}:
- powershell: |
$dir= (Split-Path -parent "${{ parameters.dockerComposePath }}")
mkdir -f -p $dir
# GitHub now requires TLS1.2. In PowerShell, run the following
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest "https://github.com/docker/compose/releases/download/1.29.1/docker-compose-windows-x86_64.exe" -OutFile "${{ parameters.dockerComposePath }}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super nit pick, but the file name is install-docker-compose-v1.yml, but these steps seem to only download the installer.

Copy link
Member

@lucaspimentel lucaspimentel Aug 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait, it's just a stand-alone .exe? never mind, then!

displayName: Download docker-compose
38 changes: 25 additions & 13 deletions .azure-pipelines/steps/run-snapshot-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ parameters:
- name: 'apiKey'
type: string
default: ''

- name: 'dockerComposePath'
type: string
default: 'docker-compose'

steps:
- template: ./clean-docker-containers.yml
Expand Down Expand Up @@ -56,7 +60,7 @@ steps:
displayName: Set env-specific variables

- bash: |
docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) run --rm $(START_TEST_AGENT_TARGET)
${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) run --rm $(START_TEST_AGENT_TARGET)
env:
dockerTag: $(dockerTag)
DD_LOGGER_DD_API_KEY: ${{ parameters.apiKey }}
Expand All @@ -65,54 +69,54 @@ steps:

- script: |
echo "Starting snapshot session"
docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) --fail "http://localhost:8126$(START_ENDPOINT)"
${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) --fail "http://localhost:8126$(START_ENDPOINT)"
displayName: start snapshot session
env:
DD_LOGGER_DD_API_KEY: ${{ parameters.apiKey }}

- bash: |
docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) run -e dockerTag=$(dockerTag) ${{ parameters.target }}
${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) run -e dockerTag=$(dockerTag) ${{ parameters.target }}
env:
dockerTag: $(dockerTag)
DD_LOGGER_DD_API_KEY: ${{ parameters.apiKey }}
displayName: docker-compose run ${{ parameters.target }}
displayName: ${{ parameters.dockerComposePath }} run ${{ parameters.target }}

- script: |
echo "Dumping traces"
docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) -o /debug_snapshots/${{ parameters.snapshotPrefix }}_traces.json "http://localhost:8126$(TRACE_DUMP_ENDPOINT)"
${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) -o /debug_snapshots/${{ parameters.snapshotPrefix }}_traces.json "http://localhost:8126$(TRACE_DUMP_ENDPOINT)"

echo "Dumping stats"
docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) -o /debug_snapshots/${{ parameters.snapshotPrefix }}_stats.json "http://localhost:8126$(STATS_DUMP_ENDPOINT)"
${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) -o /debug_snapshots/${{ parameters.snapshotPrefix }}_stats.json "http://localhost:8126$(STATS_DUMP_ENDPOINT)"

echo "Dumping all requests"
docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) -o /debug_snapshots/${{ parameters.snapshotPrefix }}_requests.json "http://localhost:8126$(REQUESTS_DUMP_ENDPOINT)"
${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) -o /debug_snapshots/${{ parameters.snapshotPrefix }}_requests.json "http://localhost:8126$(REQUESTS_DUMP_ENDPOINT)"
displayName: dump snapshots
env:
DD_LOGGER_DD_API_KEY: ${{ parameters.apiKey }}

- ${{ if eq(parameters.isLinux, true) }}:
- bash: |
echo "Verifying snapshot session (fail on mis-match)"
docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) --w '\nGetting a 400 means there is a diff in snapshots. You can diff the files with the artifacts generated. You can also run the tests locally. Follow the doc in /docs/development/CI/RunSmokeTestsLocally\n' --fail "http://localhost:8126$(VERIFY_ENDPOINT)"
${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) --w '\nGetting a 400 means there is a diff in snapshots. You can diff the files with the artifacts generated. You can also run the tests locally. Follow the doc in /docs/development/CI/RunSmokeTestsLocally\n' --fail "http://localhost:8126$(VERIFY_ENDPOINT)"
displayName: check snapshots
env:
DD_LOGGER_DD_API_KEY: ${{ parameters.apiKey }}
- ${{ else }}:
- bash: |
echo "Verifying snapshot session (fail on mis-match)"
docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) --fail "http://localhost:8126$(VERIFY_ENDPOINT)"
${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) exec -T $(TEST_AGENT_TARGET) $(CURL_COMMAND) --fail "http://localhost:8126$(VERIFY_ENDPOINT)"
displayName: check snapshots
env:
DD_LOGGER_DD_API_KEY: ${{ parameters.apiKey }}

- script: docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) logs $(TEST_AGENT_TARGET)
- script: ${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) logs $(TEST_AGENT_TARGET)
displayName: dump docker-compose logs for $(TEST_AGENT_TARGET)
env:
DD_LOGGER_DD_API_KEY: ${{ parameters.apiKey }}
condition: succeededOrFailed()
continueOnError: true

- script: docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) down
- script: ${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) down
displayName: docker-compose stop services
env:
DD_LOGGER_DD_API_KEY: ${{ parameters.apiKey }}
Expand All @@ -122,7 +126,7 @@ steps:
# Run crash tests
- ${{ if eq(parameters.isLinux, true) }}:
- bash: |
LOGS=$(docker-compose -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) run -e dockerTag=$(dockerTag) -e CRASH_APP_ON_STARTUP=1 -e COMPlus_DbgEnableMiniDump=0 ${{ parameters.target }})
LOGS=$(${{ parameters.dockerComposePath }} -f $(COMPOSE_PATH) -p ddtrace_$(Build.BuildNumber) run -e dockerTag=$(dockerTag) -e CRASH_APP_ON_STARTUP=1 -e COMPlus_DbgEnableMiniDump=0 ${{ parameters.target }})
echo $LOGS

# check logs for evidence of crash detection in the output
Expand All @@ -144,4 +148,12 @@ steps:
- script: |
sudo chmod -R 644 tracer/build_data/dumps/* || true
displayName: Make dumps uploadable to AzDo
condition: succeededOrFailed()
condition: succeededOrFailed()

- script: |
docker network prune -f
displayName: Clean up docker networks
condition: succeededOrFailed()
continueOnError: true


45 changes: 38 additions & 7 deletions .azure-pipelines/ultimate-pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5454,6 +5454,7 @@ stages:
variables:
targetShaId: $[ stageDependencies.merge_commit_id.fetch.outputs['set_sha.sha']]
targetBranch: $[ stageDependencies.merge_commit_id.fetch.outputs['set_sha.branch']]
dockerComposePath: 'C:/docker-compose/docker-compose.exe'
jobs:
- template: steps/update-github-status-jobs.yml
parameters:
Expand All @@ -5469,6 +5470,10 @@ stages:
vmImage: windows-2022

steps:
- template: steps/install-docker-compose-v1.yml
parameters:
dockerComposePath: $(dockerComposePath)

- template: steps/clone-repo.yml
parameters:
targetShaId: $(targetShaId)
Expand All @@ -5492,7 +5497,7 @@ stages:
displayName: create test data directories

- bash: |
docker-compose -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
$(dockerComposePath) -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
--build-arg DOTNETSDK_VERSION=$(dotnetCoreSdkLatestVersionShort) \
--build-arg RUNTIME_IMAGE=$(runtimeImage) \
--build-arg PUBLISH_FRAMEWORK=$(publishFramework) \
Expand All @@ -5509,9 +5514,10 @@ stages:
target: 'nuget-smoke-tests.windows'
snapshotPrefix: "smoke_test"
isLinux: false
dockerComposePath: $(dockerComposePath)

- bash: |
docker-compose -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
$(dockerComposePath) -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
--build-arg DOTNETSDK_VERSION=$(dotnetCoreSdkLatestVersionShort) \
--build-arg RUNTIME_IMAGE=$(runtimeImage) \
--build-arg PUBLISH_FRAMEWORK=$(publishFramework) \
Expand All @@ -5528,6 +5534,7 @@ stages:
target: 'nuget-dddotnet-smoke-tests.windows'
snapshotPrefix: "smoke_test"
isLinux: false
dockerComposePath: $(dockerComposePath)

- publish: tracer/build_data
artifact: _$(System.StageName)_$(Agent.JobName)_logs_$(System.JobAttempt)
Expand All @@ -5544,6 +5551,7 @@ stages:
variables:
targetShaId: $[ stageDependencies.merge_commit_id.fetch.outputs['set_sha.sha']]
targetBranch: $[ stageDependencies.merge_commit_id.fetch.outputs['set_sha.branch']]
dockerComposePath: 'C:/docker-compose/docker-compose.exe'
jobs:
- template: steps/update-github-status-jobs.yml
parameters:
Expand All @@ -5559,6 +5567,10 @@ stages:
vmImage: windows-2022

steps:
- template: steps/install-docker-compose-v1.yml
parameters:
dockerComposePath: $(dockerComposePath)

- template: steps/clone-repo.yml
parameters:
targetShaId: $(targetShaId)
Expand All @@ -5579,7 +5591,7 @@ stages:
displayName: Create test data directories

- bash: |
docker-compose -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
$(dockerComposePath) -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
--build-arg DOTNETSDK_VERSION=$(dotnetCoreSdkLatestVersionShort) \
--build-arg RUNTIME_IMAGE=$(runtimeImage) \
--build-arg PUBLISH_FRAMEWORK=$(publishFramework) \
Expand All @@ -5595,6 +5607,7 @@ stages:
target: 'dotnet-tool-smoke-tests.windows'
snapshotPrefix: "smoke_test"
isLinux: false
dockerComposePath: $(dockerComposePath)

- publish: tracer/build_data
artifact: _$(System.StageName)_$(Agent.JobName)_logs_$(System.JobAttempt)
Expand All @@ -5611,6 +5624,7 @@ stages:
variables:
targetShaId: $[ stageDependencies.merge_commit_id.fetch.outputs['set_sha.sha']]
targetBranch: $[ stageDependencies.merge_commit_id.fetch.outputs['set_sha.branch']]
dockerComposePath: 'C:/docker-compose/docker-compose.exe'
jobs:
- template: steps/update-github-status-jobs.yml
parameters:
Expand All @@ -5626,6 +5640,10 @@ stages:
vmImage: windows-2022

steps:
- template: steps/install-docker-compose-v1.yml
parameters:
dockerComposePath: $(dockerComposePath)

- template: steps/clone-repo.yml
parameters:
targetShaId: $(targetShaId)
Expand All @@ -5646,7 +5664,7 @@ stages:
displayName: Create test data directories

- bash: |
docker-compose -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
$(dockerComposePath) -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
--build-arg DOTNETSDK_VERSION=$(dotnetCoreSdkLatestVersionShort) \
--build-arg RUNTIME_IMAGE=$(runtimeImage) \
--build-arg PUBLISH_FRAMEWORK=$(publishFramework) \
Expand All @@ -5662,6 +5680,7 @@ stages:
target: 'smoke-tests.windows'
snapshotPrefix: "smoke_test"
isLinux: false
dockerComposePath: $(dockerComposePath)

- publish: tracer/build_data
artifact: _$(System.StageName)_$(Agent.JobName)_logs_$(System.JobAttempt)
Expand All @@ -5678,6 +5697,7 @@ stages:
variables:
targetShaId: $[ stageDependencies.merge_commit_id.fetch.outputs['set_sha.sha']]
targetBranch: $[ stageDependencies.merge_commit_id.fetch.outputs['set_sha.branch']]
dockerComposePath: 'C:/docker-compose/docker-compose.exe'
jobs:
- template: steps/update-github-status-jobs.yml
parameters:
Expand All @@ -5694,6 +5714,10 @@ stages:
vmImage: windows-2022

steps:
- template: steps/install-docker-compose-v1.yml
parameters:
dockerComposePath: $(dockerComposePath)

- template: steps/clone-repo.yml
parameters:
targetShaId: $(targetShaId)
Expand All @@ -5717,7 +5741,7 @@ stages:
displayName: Create test data directories

- bash: |
docker-compose -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
$(dockerComposePath) -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
--build-arg DOTNETSDK_VERSION=$(dotnetCoreSdkLatestVersionShort) \
--build-arg RUNTIME_IMAGE=$(runtimeImage) \
--build-arg PUBLISH_FRAMEWORK=$(publishFramework) \
Expand All @@ -5733,6 +5757,7 @@ stages:
target: 'dd-dotnet-smoke-tests.windows'
snapshotPrefix: "smoke_test"
isLinux: false
dockerComposePath: $(dockerComposePath)

- publish: tracer/build_data
artifact: _$(System.StageName)_$(Agent.JobName)_logs_$(System.JobAttempt)
Expand All @@ -5749,6 +5774,7 @@ stages:
variables:
targetShaId: $[ stageDependencies.merge_commit_id.fetch.outputs['set_sha.sha']]
targetBranch: $[ stageDependencies.merge_commit_id.fetch.outputs['set_sha.branch']]
dockerComposePath: 'C:/docker-compose/docker-compose.exe'
jobs:
- template: steps/update-github-status-jobs.yml
parameters:
Expand All @@ -5761,9 +5787,13 @@ stages:
variables:
smokeTestAppDir: "$(System.DefaultWorkingDirectory)/tracer/test/test-applications/regression/AspNetCoreSmokeTest"
pool:
vmImage: windows-2022
name: azure-windows-scale-set

steps:
- template: steps/install-docker-compose-v1.yml
parameters:
dockerComposePath: $(dockerComposePath)

- template: steps/clone-repo.yml
parameters:
targetShaId: $(targetShaId)
Expand All @@ -5782,7 +5812,7 @@ stages:
path: $(smokeTestAppDir)/artifacts

- bash: |
docker-compose -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
$(dockerComposePath) -f docker-compose.windows.yml -p $(DockerComposeProjectName) build \
--build-arg DOTNETSDK_VERSION=$(dotnetCoreSdkLatestVersionShort) \
--build-arg RUNTIME_IMAGE=$(runtimeImage) \
--build-arg PUBLISH_FRAMEWORK=$(publishFramework) \
Expand All @@ -5799,6 +5829,7 @@ stages:
target: 'tracer-home-smoke-tests.windows'
snapshotPrefix: "smoke_test"
isLinux: false
dockerComposePath: $(dockerComposePath)

- publish: tracer/build_data
artifact: _$(System.StageName)_$(Agent.JobName)_logs_$(System.JobAttempt)
Expand Down
Loading