CICD failure - devops/scripts/construct_tre_url.sh: No such file or directory #3488
Comments
|
HI @azuretresussex thanks for creating this. I'll try test out the pipelines today, but might be Monday before the wider team can respond. As you say it looks like the additions here: microsoft/AzureTRE-Deployment@3de7802 might have introduced a bug. If you are able can you try modifying: to I'm hoping that resolves things as in the Deployment repo the AzureTRE repo sites in a subdirectory called AzureTRE. @LizaShak @yuvalyaron @tamirkamara are you able to take a look? |
|
Thanks for responding so quickly Marcus. :) I modified that line of code as suggested, but it didn’t work unfortunately. That script isn’t present in the AzureTRE-Deployment repo. By adding pwd and ls -alR commands to .github/actions/devcontainer_run_command/action.yml in the Deployment repo, I can verify that, as the devops/scripts directory exists in the AzureTRE repo only.
I assume this wouldn't be due to the name of our copy of AzureTRE-Deployment repo, (microsoft-azure-tre)? |
|
No, won't be the name. So the AzureTRE upstream repo gets checked out in the dev container build as a If you open the deployment repo in vscode and then in the dev container you will see it. Does look like a path issue. What's in the directory one level up from |
|
I don't think I can go up the directory structure on the GH runner. If I do cd .. it does nothing, which makes sense.
|
|
Odd, can you do an Our Israeli colleagues are back in on Sunday and may be able to provide some insight. |
|
Run pwd ./.devcontainer: ./.devcontainer/scripts: ./.git: ./.git/branches: ./.git/hooks: ./.git/info: ./.git/logs: ./.git/logs/refs: ./.git/logs/refs/heads: ./.git/logs/refs/remotes: ./.git/logs/refs/remotes/origin: ./.git/objects: ./.git/objects/0c: ./.git/objects/10: ./.git/objects/18: ./.git/objects/19: ./.git/objects/2a: ./.git/objects/2b: ./.git/objects/2d: ./.git/objects/3e: ./.git/objects/46: ./.git/objects/48: ./.git/objects/49: ./.git/objects/4c: ./.git/objects/57: ./.git/objects/58: ./.git/objects/59: ./.git/objects/5a: ./.git/objects/5b: ./.git/objects/5f: ./.git/objects/60: ./.git/objects/66: ./.git/objects/69: ./.git/objects/6f: ./.git/objects/79: ./.git/objects/80: ./.git/objects/83: ./.git/objects/84: ./.git/objects/86: ./.git/objects/88: ./.git/objects/8e: ./.git/objects/8f: ./.git/objects/98: ./.git/objects/9e: ./.git/objects/a3: ./.git/objects/a5: ./.git/objects/a7: ./.git/objects/a9: ./.git/objects/af: ./.git/objects/b3: ./.git/objects/b8: ./.git/objects/bf: ./.git/objects/d4: ./.git/objects/e1: ./.git/objects/e8: ./.git/objects/ea: ./.git/objects/f6: ./.git/objects/f9: ./.git/objects/fa: ./.git/objects/fb: ./.git/objects/ff: ./.git/objects/info: ./.git/objects/pack: ./.git/refs: ./.git/refs/heads: ./.git/refs/remotes: ./.git/refs/remotes/origin: ./.git/refs/tags: ./.github: ./.github/actions: ./.github/actions/devcontainer_run_command: ./.github/linters: ./.github/scripts: ./.github/workflows: ./templates: ./templates/workspace_services: ./templates/workspaces: |
|
Ok, strange, it seems the AzureTRE repository hasn't been downloaded. Can you check the Build Dev Container step in GitHub actions. This step in the Dockerfile should download it - https://github.com/microsoft/AzureTRE-Deployment/blob/de9750a2cd5cee1358be084c8234bcc57cde4315/.devcontainer/Dockerfile#L103 |
|
Seems to be OK... Apologies for the huge paste!! 3m 11s #2 [internal] load .dockerignore #3 [internal] load metadata for mcr.microsoft.com/vscode/devcontainers/python:dev-3.8-bullseye #5 [internal] load build context #4 [ 1/24] FROM mcr.microsoft.com/vscode/devcontainers/python:dev-3.8-bullseye@sha256:9e58e3083a525aa1d70d6f0945451caa8feafbb91a74c83aaad44844486981db #5 [internal] load build context #4 [ 1/24] FROM mcr.microsoft.com/vscode/devcontainers/python:dev-3.8-bullseye@sha256:9e58e3083a525aa1d70d6f0945451caa8feafbb91a74c83aaad44844486981db #6 [ 2/24] COPY .devcontainer/scripts/non-root-user.sh /tmp/ #7 [ 3/24] RUN bash /tmp/non-root-user.sh "vscode" "1001" "123" #8 [ 4/24] RUN su vscode -c "umask 0002 && . /usr/local/share/nvm/nvm.sh && nvm install lts/* 2>&1" #9 [ 5/24] COPY .devcontainer/scripts/terraform.sh /tmp/ #10 [ 6/24] RUN bash /tmp/terraform.sh "1.4.5" /usr/bin #11 [ 7/24] COPY .devcontainer/scripts/docker-client.sh /tmp/ #12 [ 8/24] RUN /tmp/docker-client.sh vscode #13 [ 9/24] RUN apt-get update && apt-get install -y ca-certificates curl gnupg lsb-release --no-install-recommends && curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg && echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null && apt-get update && apt-get install -y docker-ce="5:23.0.3-1 #14 [10/24] RUN if [ "false" = "true" ]; then apt-get update && apt-get install -y libaugeas0 --no-install-recommends && python3 -m venv /opt/certbot/ && /opt/certbot/bin/pip install --no-cache-dir --upgrade pip && /opt/certbot/bin/pip install --no-cache-dir certbot && apt-get clean -y && rm -rf /var/lib/apt/lists/* ; fi #15 [11/24] COPY .devcontainer/scripts/porter-v1.sh /tmp/ #16 [12/24] RUN export PORTER_VERSION=v1.0.11 PORTER_TERRAFORM_MIXIN_VERSION=v1.0.2 PORTER_AZ_MIXIN_VERSION=v1.0.1 PORTER_AZURE_PLUGIN_VERSION=v1.2.0 PORTER_HOME=/home/vscode/.porter/ && /tmp/porter-v1.sh #17 [13/24] COPY .devcontainer/scripts/azure-cli.sh /tmp/ #18 [14/24] RUN export AZURE_CLI_VERSION=2.37.0-1 #19 [15/24] RUN curl -L --fail -o /usr/local/bin/yq "https://github.com/mikefarah/yq/releases/download/v4.33.3/yq_linux_amd64" && chmod +x /usr/local/bin/yq #20 [16/24] RUN npm install -g pajv@1.2.0 #21 [17/24] RUN if [ "false" = "false" ]; then apt-get update && apt-get install --no-install-recommends -y git && apt-get clean -y && rm -rf /var/lib/apt/lists/* ; fi #22 [18/24] RUN echo "export HISTFILE=$HOME/commandhistory/.bash_history" >> "$HOME/.bashrc" && echo "export PROMPT_COMMAND='history -a'" >> "$HOME/.bashrc" && mkdir -p "$HOME/commandhistory" && touch "$HOME/commandhistory/.bash_history" #23 [19/24] COPY ./.devcontainer/scripts/gh.sh /tmp/ #24 [20/24] RUN if [ "false" = "true" ]; then /tmp/gh.sh; fi #25 [21/24] COPY .devcontainer/scripts/install-azure-tre-oss.sh .devcontainer/devcontainer.json /tmp/ #26 [22/24] RUN oss_version_in_json=$(grep -oP '(?<="OSS_VERSION": ")[^"]*' /tmp/devcontainer.json) && /tmp/install-azure-tre-oss.sh "$OSS_VERSION:-$oss_version_in_json" "/home/vscode/AzureTRE" && sudo chown -R vscode /home/vscode/AzureTRE #27 [23/24] WORKDIR //home/vscode/AzureTRE/cli #28 [24/24] RUN make install-cli && echo -e "\n# Set up tre completion\nsource <(_TRE_COMPLETE=bash_source tre)" >> #29 exporting to image #30 exporting cache |
|
This issue is a result of calling scripts in Github Actions that aren't available yet. The TRE OSS is downloaded only inside the devcontainer so referencing its scripts isn't possible outside. |
|
@azuretresussex can you give this fix a try and let us know if it works for you? |
|
Hi @tamirkamara Thank you for looking at this. With those extra files, and a slight modification to the path in two places in deploy_tre.yml, the issue I had in Deploy management was resolved.
The Storage Account for tfstate and the ACR were deployed OK, however, during ACR Login, I received the following:
Any ideas? Thanks! |
|
|
Hi @tamirkamara, @marrobi I made changes to deploy_tre.yml whilst I was trying to fix this myself. I wasn't whether the pwd on the runner would find the scripts.
I didn't try re-running the Workflow, however I have just re-tried now, and it is OK the second time (without any change from the last time when it failed.) I also just found mention of this known issue here (merci, @chboudry !) Success!
Many thanks for your help! |
Describe the bug
Deploying AzureTRE using provided deploy_tre.yml fails during 'Deploy Management'.
Steps to reproduce
When 'Deploy Management' runs, it fails as follows:
Prepare all required actions
Getting action download info
Run ./.github/actions/devcontainer_run_command
Run azure_env="AzureCloud"
Run source devops/scripts/construct_tre_url.sh
/home/runner/work/_temp/15921fb8-6b7f-497f-88ed-b3e0210eea03.sh: line 1: devops/scripts/construct_tre_url.sh: No such file or directory
Error: Process completed with exit code 1.
If I download the AzureTRE archive directly and open it, it does indeed include the above file, so I assume it is the path. However, not sure how to resolve this issue. Any help would be appreciated.
The text was updated successfully, but these errors were encountered: