[build] Build tracer with ReadyToRun #5962
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| #!/bin/bash | ||
| # | ||
| # This scripts downloads the necessary binaries to be used for the AWS Lambda Layer. | ||
| # This artifacts include: Tracer and ClrProfiler | ||
| # | ||
|
|
||
| set -eo pipefail | ||
|
|
||
| #Create a directory to store the files | ||
| target_dir=artifacts | ||
| mkdir -p $target_dir | ||
|
|
||
| # if [ -n "$CI_COMMIT_TAG" ] || [ -n "$DOTNET_PACKAGE_VERSION" ]; then | ||
| # echo "Downloading artifacts from Github" | ||
| # VERSION=${DOTNET_PACKAGE_VERSION:-${CI_COMMIT_TAG##v}} # Use DOTNET_PACKAGE_VERSION if it exists, otherwise use CI_COMMIT_TAG without the v | ||
|
|
||
| # for SUFFIX in "" ".arm64"; do | ||
| # curl --location --fail \ | ||
| # --output $target_dir/datadog-dotnet-apm-${VERSION}${SUFFIX}.tar.gz \ | ||
| # "https://github.com/DataDog/dd-trace-dotnet/releases/download/v${VERSION}/datadog-dotnet-apm-${VERSION}${SUFFIX}.tar.gz" | ||
| # done | ||
|
|
||
| # echo -n $VERSION > $target_dir/version.txt | ||
| # exit 0 | ||
| # fi | ||
|
|
||
| branchName="refs/heads/$CI_COMMIT_BRANCH" | ||
|
|
||
| echo "Looking for azure devops PR builds for branch '$branchName' for commit '$CI_COMMIT_SHA' to start" | ||
|
|
||
| # We should _definitely_ have the build by now, so if not, there probably won't be one | ||
| # Check for PR builds first (as more likely to be "full" builds) | ||
| allBuildsForPrUrl="https://dev.azure.com/datadoghq/dd-trace-dotnet/_apis/build/builds?api-version=7.1&definitions=54&\$top=100&queryOrder=queueTimeDescending&reasonFilter=pullRequest" | ||
| buildId=$(curl -sS $allBuildsForPrUrl | jq --arg version $CI_COMMIT_SHA --arg branch $CI_COMMIT_BRANCH '.value[] | select(.triggerInfo["pr.sourceBranch"] == $branch and .triggerInfo["pr.sourceSha"] == $version) | .id' | head -n 1) | ||
|
|
||
| if [ -z "${buildId}" ]; then | ||
| echo "No PR builds found for commit '$CI_COMMIT_SHA' on branch '$branchName'. Checking for standalone builds..." | ||
| allBuildsForBranchUrl="https://dev.azure.com/datadoghq/dd-trace-dotnet/_apis/build/builds?api-version=7.1&definitions=54&\$top=10&queryOrder=queueTimeDescending&branchName=$branchName&reasonFilter=manual,individualCI" | ||
| buildId=$(curl -sS $allBuildsForBranchUrl | jq --arg version $CI_COMMIT_SHA '.value[] | select(.sourceVersion == $version and .reason != "schedule") | .id' | head -n 1) | ||
| fi | ||
|
|
||
| if [ -z "${buildId}" ]; then | ||
| echo "No build found for commit '$CI_COMMIT_SHA' on branch '$branchName' (including PRs)" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Found build with id '$buildId' for commit '$CI_COMMIT_SHA' on branch '$branchName'" | ||
|
|
||
| architectures=("x64" "arm64") | ||
| for architecture in "${architectures[@]}"; do | ||
| echo "Looking for artifacts for architecture '$architecture'" | ||
|
|
||
| artifacts=("linux-tracer-home-linux-$architecture-r2r" "linux-universal-home-linux-$architecture") | ||
|
|
||
| # Now try to download the artifacts from the build | ||
| for artifactName in "${artifacts[@]}"; do | ||
| artifactsUrl="https://dev.azure.com/datadoghq/dd-trace-dotnet/_apis/build/builds/$buildId/artifacts?api-version=7.1&artifactName=$artifactName" | ||
|
|
||
| # Keep trying to get the artifact for 30 minutes | ||
| downloadUrl="" | ||
| TIMEOUT=1800 | ||
| STARTED=0 | ||
| until (( STARTED == TIMEOUT )) || [ ! -z "${downloadUrl}" ] ; do | ||
| echo "Checking for '$artifactName' at '$artifactsUrl'..." | ||
| # If the artifact doesn't exist, .resource.downloadUrl will be null, so we filter that out | ||
| downloadUrl=$(curl -s $artifactsUrl | jq -r '.resource.downloadUrl | select( . != null )') | ||
| sleep 100 | ||
| (( STARTED += 100 )) | ||
| done | ||
| (( STARTED < TIMEOUT )) | ||
|
|
||
| if [ -z "${downloadUrl}" ]; then | ||
| echo "No downloadUrl found after 30 minutes for commit '$CI_COMMIT_SHA' on branch '$branchName'" | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Downloading '$artifactName' from '$downloadUrl'..." | ||
| curl -o $target_dir/artifacts.zip "$downloadUrl" | ||
| unzip $target_dir/artifacts.zip -d $target_dir/$architecture | ||
| mv $target_dir/$architecture/$artifactName/* $target_dir/$architecture | ||
| rm -rf $target_dir/artifacts.zip | ||
| rmdir $target_dir/$architecture/$artifactName | ||
| done | ||
| done | ||
|
|
||
| ls -l $target_dir | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -639,23 +639,50 @@ async Task DownloadWafVersion(string libddwafVersion = null, string uncompressFo | |
| .EnableNoRestore() | ||
| .CombineWith(targetFrameworks, (p, framework) => p | ||
| .SetFramework(framework) | ||
| .SetOutput(MonitoringHomeDirectory / framework)) | ||
| ); | ||
| }); | ||
|
|
||
| Target PublishManagedTracerR2R => _ => _ | ||
| .Unlisted() | ||
| .After(CompileManagedSrc) | ||
| .Executes(() => | ||
| { | ||
| var targetFramework = TargetFramework.NET6_0; | ||
|
|
||
| // Needed as we need to restore with the RuntimeIdentifier | ||
| DotNetRestore(s => s | ||
| .SetProjectFile(Solution.GetProject(Projects.DatadogTraceMsBuild)) | ||
|
There was a problem hiding this comment. Do you really need this at all? 🤔 I'm about 90% sure you don't need this dll at all, and should be stripping it out of the layer. @tonyredondo can you confirm? There was a problem hiding this comment. It's stripped down later, but I want to make sure I build as close as possible to the current process, don't want to diverge much in case in the future we need to use some r2r binaries for other platforms like Azure! There was a problem hiding this comment. Just for info, I believe this is literally only used if you're using the tracer as part of your build process, so I think there's basically no chance you'll need it 😃 |
||
| .SetPublishReadyToRun(true) | ||
| .SetRuntime(RuntimeIdentifier) | ||
| ); | ||
|
|
||
| DotNetPublish(s => s | ||
| .SetProject(Solution.GetProject(Projects.DatadogTraceMsBuild)) | ||
| .SetConfiguration(BuildConfiguration) | ||
| .SetTargetPlatformAnyCPU() | ||
| .SetPublishReadyToRun(true) | ||
| .SetRuntime(RuntimeIdentifier) | ||
| .SetSelfContained(false) | ||
| .SetFramework(targetFramework) | ||
| .SetOutput(MonitoringHomeDirectory / targetFramework) | ||
| ); | ||
| }); | ||
|
|
||
| Target PublishNativeSymbolsWindows => _ => _ | ||
| .Unlisted() | ||
| .OnlyWhenStatic(() => IsWin) | ||
| .After(CompileTracerNativeSrc, PublishManagedTracer) | ||
| .Executes(() => | ||
| { | ||
| foreach (var architecture in ArchitecturesForPlatformForTracer) | ||
| { | ||
| var source = NativeTracerProject.Directory / "bin" / BuildConfiguration / architecture.ToString() / | ||
| $"{NativeTracerProject.Name}.pdb"; | ||
| var dest = SymbolsDirectory / $"win-{architecture}" / Path.GetFileName(source); | ||
| CopyFile(source, dest, FileExistsPolicy.Overwrite); | ||
| } | ||
| }); | ||
|
|
||
| Target PublishDdDotnetSymbolsWindows => _ => _ | ||
| .Unlisted() | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI this will 30mins for the first artifact and then 30 mins for the next one too. That's prob not what we really want, but it's not a big deal, and prob not worth fixing frankly! :D
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is fine, I tested multiple times, and some artifacts were not available, so I think this is OK