Skip to content

Commit

Permalink
Merged Main (#1999)
Browse files Browse the repository at this point in the history
* Fix Guacamole refresh token (#1785)

* Fixing Guacamole refresh token

* Fix aad tenant bug

Co-authored-by: Anat Balzam <anatbalzam@microsoft.com>

* Add Contributor to the Role permissions (#1781)

* Put it back to Owner during investigation

* Attempt to find correct permissions

* User Access Administrator

Co-authored-by: Marcus Robinson <marrobi@microsoft.com>

* Ignore Only Root index.html (#1800)

* fixes #1775

* remove unwanted cli prefix

* only ignore root index.html

* Gitea/Guacamole should be able to access AAD_TENANT_ID (#1798)

* Add auth-tenant-id to ws keyvault

* Bump versions

* linting

* re-instating the deploy/destroy files

* Linting

* Update deploy.sh

* Update destroy.sh

* shared services in pr bot and split out in tests (#1813)

* Resource Processor: Configure logging handler per process (#1784)

* Update TRE Developer doc for API (#1801)

* E2E work with scope_id from workspace properties (#1797)

* E2E work with local API

* Added scope identifier uri into tests

* Hangiver from previous method

* Try adding a sleep into the endpoint dns

* Bump the version

* Wait for the private endpoint

* Bump version

* Forgot to wait on teh sleep

* Bumped

* refactoring bug

* Purge Protection

* Bump version

* PR Comments

* More PR tweaks

* typo

* shell check comment

* Remove purge protection

* _get_app_auth_info

* Update docs on running End-to-end tests locally (#1829)

* Fix check order in pr-bot (#1850)

Only check user permissions if a command is detected
to avoid adding 'sorry, not allowed' comments in response to comments
that aren't commands

* Next available IP range calculator only considers active workspaces (#1849)

* Increase Azure CLI version (#1864)

* update all versions to 0.3 (#1754)

* Fix Firewall Logging (#1870)

* switch firewall away from dedicated log tables

* update TF lock

* fix liniting issue with firewall.tf

* Change how access properties in get_scope (#1882)

* added missing param for invoke-action (#1906)

* added missing param

* api version

* Add Bicep tools to devcontainer (#1848)

Co-authored-by: Marcus Robinson <marrobi@microsoft.com>

* E2E tests: Fix shared service and performance tests  (#1860)

* Fix tests

* WIP

* WIP: add a command to build a user resource

* Fix performance test

* fix gucacamole dev vm

* removed unused import

* Fix shared services test

* fix user resources command

* Revert Makefile changes

* fix tabs

* Update templates/workspaces/base/terraform/variables.tf

Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>

* Add .terraform in .dockerignore files (#1872)

* Bump pyjwt from 2.3.0 to 2.4.0 in /api_app (#1913)

* Add resource id var to shared services. (#1914)

* Add resource id var to shared services.

* Update gitea version.

* Fix linter version.

Co-authored-by: Liza Shakury <lizashakury@Lizas-MacBook-Pro.local>

* add tflint config (#1919)

* Update httpx package (#1917)

* update httpx package

* bump version number of API

* Improve documentation for Resource Processor (#1827)

* Re-host Nexus on vm (#1584)

* Initial commit

* Replaced webapp with vm

* Amended docker start commands

* Amended firewall

* Add nexus config to persistent volume

* Add private dns zone

* Corrected rg var

* Added Nexus letsencrypt cert gen

* Fixed linting

* Changed terraform.lock.hcl to previous version

* Removed leftover debug

* Typo fix

Co-authored-by: Stuart Leeks <stuart@leeks.net>

* File path amend

Co-authored-by: Stuart Leeks <stuart@leeks.net>

* Fix for cloudapp DNS resolution errors

* Docker running on Nexus VM

* Documented Letsencrypt process

* Permissions fix

* Typo fix

Co-authored-by: Stuart Leeks <stuart@leeks.net>

* Typo fix

Co-authored-by: Stuart Leeks <stuart@leeks.net>

* Typo fix

Co-authored-by: Stuart Leeks <stuart@leeks.net>

* Formatting changes

* Added reference to letsencrypt doc

* Added new page reference

* Moved password generation for nexus to tf

* Write script to fs first before execution

* Password reset finally working

* Make config nexus script runnable from any dir

* Added basic status info

* Fix recursive file loop

* Typo fix

* Updated docs

* renamed env file

* Fix typo

* Added new nexus fqdn to user resources

* Add vnet link to workspaces

* Bump versions

* Removed nexus properties file

* Updated execution permissions

* Get cert in tf

* Added az cli get cert

* Amended prune job

* Added msi id to login

* Amended msi and exported cert pwd

* Jetty configuration

* Escape jetty vars

* Password script fixes

* Amended networking to use module

* Use https in config script

* Removed res proc location variable

* Potential linting fix

* Linting fixes

* Linting directive positioning

* Gitea version bump

* Terraform format

* Reorder linting to workaround superlinter bug with Terraform

* Added nexus-cert to build and caching of letsencrypt

* Adopted new shared service deploy method

* Added cron job to renew nexus cert

* Removed location references

* And another

* Removed location refs and added az cli

* Fixed nexus-cert kv permissions

* Corrected outputs directory

* Fixed shared service deployment steps

* Updated docs and removed renew prompt

* version bump

* Increase bundle versions

* remote location from variables files

* Removed shared service make

* Removed docker prune

* Bash headers

* Layer clean

* Reduce layer

* Testing without kv role assignment

* Removed kv role assignment

* Adding firewall rule to allow letsencrypt from RP

* Genericised cert service and added letsencrypt action

* Fixed auth hook

* Removed make commands

* Certbot in bundle container

* Tidied naming

* Python base image

* Generate action successful

* Inject cert name to nexus bundle

* Implemented app gateway start/stop

* Separated cloudinit yaml into scripts

* Fixed new line issue

* Fixed bash casing

* Added local nexus repo config

* Added retry logic to config repos

* gitea bump

* Fixed status code

* terraform linting

* Added docs

* Lint fix

* Update docs/tre-developers/letsencrypt.md

* Update docs/tre-admins/setup-instructions/configuring-shared-services.md

* Update docs/tre-developers/letsencrypt.md

* Update docs/tre-developers/letsencrypt.md

* Update docs/tre-admins/setup-instructions/configuring-shared-services.md

Co-authored-by: Marcus Robinson <marrobi@microsoft.com>

* Fix firewall conflict

* Added note to docs for cert kv conflicts

* Renamed sonatype-nexus to nexus for new version

* Added old nexus service code

* Lint fix

* Renamed folder to be obvious as the nexus-vm

* Added docs for upgrade path

* Added data.azurerm rg core

* linting

* bash linting

* Require workspace of 0.2.14 or above

* Moved new version notes to section below config steps

* Removed give new cert name

* RP cert permissions

* tf format

* Added required params for certs and nexus tempalte schema

* Added cert import permissions

* Added certs delete permission

* App gateway az login

* Version bumps

* tf fmt

* Added missing az cred params to certs

* Add purge permission

* Bump tf versions to 3.4.0 & set purge to false

* Removed unsupported property from new provider

* Moved nexus private zone to core

* Amended location var

* Amended zone location

* Added upgrade flag for tf

* Remove tf lock

* Added new tf key

* Added key into uninstall

* Resolve firewall rule conflicts

* Var reference fix

* Fix for potential @ symbol in nexus admin password causing curl bug

* Added nexus_version variable to user resources for back compat

* Added docs for nexus_version

* downgrade superlinter

* revert superlinter to v4

* Remove lint aws plugin block

* Use superlinter latest

* Manually set tflint path

Co-authored-by: oliver7598 <oliver.a@live.co.uk>
Co-authored-by: Stuart Leeks <stuart@leeks.net>
Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>
Co-authored-by: ross-p-smith <rosmith@microsoft.com>
Co-authored-by: Jamie D <daltskin@hotmail.com>
Co-authored-by: Stuart Leeks <stuartle@microsoft.com>
Co-authored-by: marrobi <marrobi@microsoft.com>

* Mandatory client-secret when creating a workspace (#1924)

* Mandatory client_secret when creating workspace

* Debugging settings

* azure rm version

* Update templates/workspaces/base/.env.sample

Co-authored-by: Marcus Robinson <marrobi@microsoft.com>

* Update templates/workspaces/base/.env.sample

Co-authored-by: Marcus Robinson <marrobi@microsoft.com>

* Update templates/workspaces/base/terraform/variables.tf

Co-authored-by: Marcus Robinson <marrobi@microsoft.com>

* disable app service's ftp (#1930)

* Airlock resources - tf scripts (#1843)

* Airlock resources - tf scripts

* reusing the existing sb + adding network rules
bug fixes

* Make etag required in API documentaiton, remove custom check (#1932)

* Make etag required in API documentaiton, remove custom check

* Update _version.py

* tests + remove string

Co-authored-by: sharon <sharon.hart@microsoft.com>

* Reimage Resource Processor Automatically (#1929)

* reimage resource processor automatically

* resource processor vm user docker permissions

* update hcl

* initial swa deploy

* Tag tre core services (#1916)

* tag core resources

* Setting workspace_owner_object_id when creating workspaces (#1928)

* Wait for DNS

* workspace_owner_object_id

* Bumped version

* Added extra value for local testing

* Pass through workspace_owner_object_id

* workspace_identifier_uri

* typo in test name

* Kept extra check for 202

* Optimize Guacamole docker image (#1933)

* Optimize Guacamole docker image

* remove comments

* pin tomcat version

* lint changes

* fix

* ignore linter error about the user

* fix user

* update versions

Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>

* Upgrade azurerm provider version to 3.5.0 (#1947)

* E2E on main run in sequence (#1945)

* Allow e2e tests to run in sequence from main

* fix

* actions lint fix

* add missing markers to pytest.ini

* fix selector string

* update timeout

* fix prbot e2eTestsCustomSelector param (#1959)

* Airlock processor - function app based - Base (#1950)

* Cost Report - Tag Gitea shared service (#1941)

* Tag gitea shared service

* Fix format

* Upgrade mysql

* Ignore tflint error due to a bug in it.

* CR changes

* Move tflint ignore comment.

* Fix TF fmt.

* Fix TF lint.

* Upgrade gitea version

Co-authored-by: Liza Shakury <lizashakury@Lizas-MacBook-Pro.local>

* update-ws-svc-fw-rule-name-ad (#1957)

* azurerm_app_service_plan is deprecated and we should use azurerm_service_plan (#1958)

* azurerm_app_service_plan ->azurerm_service_plan

* Migrate resources

* Linter

* tflint

* azurerm_app_service_plan ->azurerm_service_plan

* Migrate resources

* Linter

* tflint

* Handle empty state

* Don't migrate Terraform (#1977)

* [cost] Tag firewall and nexus shared services. (#1979)

* Tag firewall and nexus

* Update gitea version.

* Update nexus porter

* Update nexus parameters

* Update nexus variables

Co-authored-by: Liza Shakury <lizashakury@Lizas-MacBook-Pro.local>

* Create Application Administrator (#1975)

* Create Application Administrator

* Change sp name to have TRE in it

* Cleanup bundle dockerfiles (#1969)

* cleanup bundle dockerfiles

* lint fixes

* pin apt-get versions

* fix version

* bump version

* Register VM Bundle for E2E tests (#1987)

* Publish before Register (#1988)

* Publish before Register

* TF_VAR_application_admin_client_id

* Registering a user_resource needs the Workspace Service Name (#1989)

* WORKSPACE_SERVICE_NAME

* run command needs WORKSPACE_SERVICE_NAME

* And added to the docker run

* add missing dockerfile.tmpl references (#1990)

Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>

* Create user_resource in e2e tests (#1952)

* Create user_resource in e2e tests

* Testing user Resource

* Purge Protection

* Added guacamole-azure-windowsvm fpr e2e

* Register the Windows VM

* TF_VAR_application_admin_client_id (#1993)

* Missing Inputs (#1994)

* Use different identity to create applications (#1976)

* Use different identity to create applications

* Bump version

* Mandatory application_admin_client_id

* Missing porter parameters

* [cost] Tag Guacamole Workspace Service in Terraform (#1971)

tag Guacamole workspace service for cost feature

Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>
Co-authored-by: Tamir Kamara <26870601+tamirkamara@users.noreply.github.com>
Co-authored-by: Guy Bertental <gubert@microsoft.com>

* config.sample -> config.source

* gitignore config.json

* page title, favicon, readme

* left nav padding

Co-authored-by: Anat Balzam <anatbaz@gmail.com>
Co-authored-by: Anat Balzam <anatbalzam@microsoft.com>
Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>
Co-authored-by: Marcus Robinson <marrobi@microsoft.com>
Co-authored-by: Martin Peck <mpeck@microsoft.com>
Co-authored-by: tanya-borisova <tborisova@microsoft.com>
Co-authored-by: Stuart Leeks <stuartle@microsoft.com>
Co-authored-by: Tamir Kamara <26870601+tamirkamara@users.noreply.github.com>
Co-authored-by: Sven Aelterman <17446043+SvenAelterman@users.noreply.github.com>
Co-authored-by: Sonali Rajput <71600666+Sonali-Rajput@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Liza Shakury <42377481+LizaShak@users.noreply.github.com>
Co-authored-by: Liza Shakury <lizashakury@Lizas-MacBook-Pro.local>
Co-authored-by: James Griffin <me@JamesGriff.in>
Co-authored-by: oliver7598 <oliver.a@live.co.uk>
Co-authored-by: Stuart Leeks <stuart@leeks.net>
Co-authored-by: ross-p-smith <rosmith@microsoft.com>
Co-authored-by: Jamie D <daltskin@hotmail.com>
Co-authored-by: Elad Iwanir <13205761+eladiw@users.noreply.github.com>
Co-authored-by: Sharon Hart <sharonh.dev@gmail.com>
Co-authored-by: sharon <sharon.hart@microsoft.com>
Co-authored-by: Guy Bertental <gubert@microsoft.com>
Co-authored-by: dusan-ilic-mhra <104429461+dusan-ilic-mhra@users.noreply.github.com>
Co-authored-by: Ciprian Maftei <104429915+ciprianmaf@users.noreply.github.com>
  • Loading branch information
1 parent 5523113 commit 4989a1c
Show file tree
Hide file tree
Showing 147 changed files with 1,584 additions and 1,007 deletions.
14 changes: 14 additions & 0 deletions .github/actions/devcontainer_run_command/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,12 @@ inputs:
TF_VAR_api_client_secret:
description: "The API Client Secret."
required: false
TF_VAR_application_admin_client_id:
description: "The Client ID of an identity that can manage the AAD Applications."
required: false
TF_VAR_application_admin_client_secret:
description: "The Client secret of an identity that can manage the AAD Applications."
required: false
ACR_NAME:
description: "The Container Registry that holds our Research images."
required: false
Expand All @@ -98,6 +104,9 @@ inputs:
BUNDLE_TYPE:
description: "The Bundle type (workspace / Workspace-service / User Resource)."
required: false
WORKSPACE_SERVICE_NAME:
description: "The workspace service name for the bundle you are registering."
required: false
IS_API_SECURED:
description: "Indicates if the API endpoint has valid TLS certificate and if we validate it during E2E."
required: false
Expand Down Expand Up @@ -156,6 +165,8 @@ runs:
TF_VAR_aad_tenant_id: "${{ inputs.AAD_TENANT_ID }}"
TF_VAR_api_client_id: "${{ inputs.TF_VAR_api_client_id }}"
TF_VAR_api_client_secret: "${{ inputs.TF_VAR_api_client_secret }}"
TF_VAR_application_admin_client_id: "${{ inputs.TF_VAR_application_admin_client_id }}"
TF_VAR_application_admin_client_secret: "${{ inputs.TF_VAR_application_admin_client_secret }}"
TF_VAR_acr_name: ${{ inputs.ACR_NAME }}
IS_API_SECURED: ${{ inputs.IS_API_SECURED }}
run: |
Expand All @@ -168,6 +179,7 @@ runs:
-e TF_IN_AUTOMATION="${{ inputs.TF_IN_AUTOMATION }}" \
-e USE_ENV_VARS_NOT_FILES="${{ inputs.USE_ENV_VARS_NOT_FILES }}" \
-e BUNDLE_TYPE="${{ inputs.BUNDLE_TYPE }}" \
-e WORKSPACE_SERVICE_NAME="${{ inputs.WORKSPACE_SERVICE_NAME }}" \
-e LOCATION="${{ inputs.LOCATION }}" \
-e TF_VAR_location="${{ inputs.LOCATION }}" \
-e RESOURCE_LOCATION="${{ inputs.LOCATION }}" \
Expand All @@ -183,6 +195,8 @@ runs:
-e TF_VAR_aad_tenant_id \
-e TF_VAR_api_client_id \
-e TF_VAR_api_client_secret \
-e TF_VAR_application_admin_client_id \
-e TF_VAR_application_admin_client_secret \
-e TF_VAR_arm_subscription_id="${{ inputs.ARM_SUBSCRIPTION_ID }}" \
-e TF_VAR_swagger_ui_client_id \
-e TF_VAR_core_address_space \
Expand Down
2 changes: 2 additions & 0 deletions .github/linters/.yaml-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@ rules:
line-length:
max: 120 # Keep this updated with the editorconfig file
level: warning
comments:
min-spaces-from-content: 1 # Used to follow prettier standard: https://github.com/prettier/prettier/pull/10926
10 changes: 5 additions & 5 deletions .github/workflows/build_validation_develop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,9 @@ jobs:
- name: Lint code base
# the slim image is 2GB smaller and we don't use the extra stuff
# Moved this after the Terraform checks above due something similar to this issue: https://github.com/github/super-linter/issues/2433
uses: github/super-linter/slim@v4.9.3
# Moved this after the Terraform checks above due something similar to this issue:
# https://github.com/github/super-linter/issues/2433
uses: github/super-linter/slim@v4.9.4
env:
# Until https://github.com/github/super-linter/commit/ec0662756da93f1e3aad4df049712df7d764d143 is released
# we need to set the correct plugin directory (which is incorrectly set to github/home/.tflint.d/plugins by default)
Expand All @@ -64,6 +65,5 @@ jobs:
JAVA_FILE_NAME: checkstyle.xml
VALIDATE_BASH: true
VALIDATE_BASH_EXEC: true
# https://github.com/microsoft/AzureTRE/issues/1723 tracks re-instating VALIDATE_GITHUB_ACTIONS
# Note: in the meantime, the `.github/scripts/run-test.sh` script includes the `actionlint` checks)
# VALIDATE_GITHUB_ACTIONS: true
VALIDATE_GITHUB_ACTIONS: true
VALIDATE_DOCKERFILE_HADOLINT: true
3 changes: 1 addition & 2 deletions .github/workflows/deploy_tre.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,7 @@ jobs:
uses: ./.github/workflows/deploy_tre_reusable.yml
with:
ciGitRef: ${{ github.ref }}
runExtendedTests: true
runSharedServicesTests: true
e2eTestsCustomSelector: "extended or shared_services"
secrets:
AAD_TENANT_ID: ${{ secrets.AAD_TENANT_ID }}
ACR_NAME: ${{ secrets.ACR_NAME }}
Expand Down
13 changes: 6 additions & 7 deletions .github/workflows/deploy_tre_branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,18 @@ name: Deploy Azure TRE (branch)
on: # yamllint disable-line rule:truthy
workflow_dispatch:
inputs:
runExtendedTests:
description: Run the extended tests as part of the deployment?
type: boolean
default: false
e2eTestsCustomSelector:
description: A pytest marker selector for the e2e tests to be run
type: string
default: ""
required: false
runSharedServicesTests:
description: Run the shared services tests as part of the deployment?
type: boolean
default: false
required: false


# This will prevent multiple runs of this entire workflow.
# We should NOT cancel in progress runs as that can destabilize the environment.
concurrency: "${{ github.workflow }}-${{ github.ref }}"
Expand Down Expand Up @@ -54,9 +55,7 @@ jobs:
uses: ./.github/workflows/deploy_tre_reusable.yml
with:
ciGitRef: ${{ github.ref }}
# testing input against string 'true' - see https://github.com/actions/runner/issues/1483
runExtendedTests: ${{ github.event.inputs.runExtendedTests == 'true' }}
runSharedSevicesTests: ${{ github.event.inputs.runSharedServicesTests == 'true' }}
e2eTestsCustomSelector: ${{ github.event.inputs.e2eTestsCustomSelector }}
secrets:
AAD_TENANT_ID: ${{ secrets.AAD_TENANT_ID }}
ACR_NAME: ${{ format('tre{0}', needs.prepare-not-main.outputs.refid) }}
Expand Down
98 changes: 53 additions & 45 deletions .github/workflows/deploy_tre_reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,19 @@ on: # yamllint disable-line rule:truthy
type: string
required: false
prHeadSha:
description: For PR builds where GITHUB_REF isn't set to the PR (e.g. comment trigger), pass the PR's head SHA commit here
description: >-
For PR builds where GITHUB_REF isn't set to the PR (e.g. comment trigger),
pass the PR's head SHA commit here
type: string
required: false
ciGitRef:
description: The git ref to use in annotations to associate a deployment with the code that triggered it
type: string
required: true
runExtendedTests:
description: Controls whether to run the extended tests as part of the deployment
type: boolean
default: false
e2eTestsCustomSelector:
description: The pytest marker selector for the e2e tests to be run
type: string
default: ""
required: false
runSharedServicesTests:
description: Controls whether to run the shared services tests as part of the deployment
Expand Down Expand Up @@ -212,10 +214,7 @@ jobs:
strategy:
fail-fast: true
matrix:
target: [
build-and-push-api,
build-and-push-resource-processor,
build-and-push-gitea]
target: [build-and-push-api, build-and-push-resource-processor]

steps:
- name: Checkout
Expand Down Expand Up @@ -277,18 +276,18 @@ jobs:
TRE_ID: "${{ secrets.TRE_ID }}"
LOCATION: ${{ secrets.LOCATION }}
ACR_NAME: ${{ secrets.ACR_NAME }}
TF_VAR_terraform_state_container_name:
${{ secrets.TF_STATE_CONTAINER }}
TF_VAR_terraform_state_container_name: ${{ secrets.TF_STATE_CONTAINER }}
TF_VAR_mgmt_resource_group_name: ${{ secrets.MGMT_RESOURCE_GROUP }}
TF_VAR_mgmt_storage_account_name:
${{ secrets.STATE_STORAGE_ACCOUNT_NAME }}
TF_VAR_mgmt_storage_account_name: ${{ secrets.STATE_STORAGE_ACCOUNT_NAME }}
TF_VAR_core_address_space: ${{ secrets.CORE_ADDRESS_SPACE }}
TF_VAR_tre_address_space: ${{ secrets.TRE_ADDRESS_SPACE }}
TF_VAR_swagger_ui_client_id: "${{ secrets.SWAGGER_UI_CLIENT_ID }}"
TF_VAR_api_client_id: "${{ secrets.API_CLIENT_ID }}"
TF_VAR_api_client_secret: "${{ secrets.API_CLIENT_SECRET }}"
TF_VAR_keyvault_purge_protection_enabled: "${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_stateful_resources_locked: "${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_keyvault_purge_protection_enabled:
"${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_stateful_resources_locked:
"${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"

deploy_tre:
name: Deploy TRE
Expand Down Expand Up @@ -326,18 +325,20 @@ jobs:
TRE_ID: "${{ secrets.TRE_ID }}"
LOCATION: ${{ secrets.LOCATION }}
ACR_NAME: ${{ secrets.ACR_NAME }}
TF_VAR_terraform_state_container_name:
${{ secrets.TF_STATE_CONTAINER }}
TF_VAR_terraform_state_container_name: ${{ secrets.TF_STATE_CONTAINER }}
TF_VAR_mgmt_resource_group_name: ${{ secrets.MGMT_RESOURCE_GROUP }}
TF_VAR_mgmt_storage_account_name:
${{ secrets.STATE_STORAGE_ACCOUNT_NAME }}
TF_VAR_mgmt_storage_account_name: ${{ secrets.STATE_STORAGE_ACCOUNT_NAME }}
TF_VAR_core_address_space: ${{ secrets.CORE_ADDRESS_SPACE }}
TF_VAR_tre_address_space: ${{ secrets.TRE_ADDRESS_SPACE }}
TF_VAR_swagger_ui_client_id: "${{ secrets.SWAGGER_UI_CLIENT_ID }}"
TF_VAR_api_client_id: "${{ secrets.API_CLIENT_ID }}"
TF_VAR_api_client_secret: "${{ secrets.API_CLIENT_SECRET }}"
TF_VAR_keyvault_purge_protection_enabled: "${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_stateful_resources_locked: "${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_application_admin_client_id: "${{ secrets.API_CLIENT_ID }}"
TF_VAR_application_admin_client_secret: "${{ secrets.API_CLIENT_SECRET }}"
TF_VAR_keyvault_purge_protection_enabled:
"${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_stateful_resources_locked:
"${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"

- name: API Healthcheck
uses: ./.github/actions/devcontainer_run_command
Expand All @@ -361,18 +362,18 @@ jobs:
TRE_ID: "${{ secrets.TRE_ID }}"
LOCATION: ${{ secrets.LOCATION }}
ACR_NAME: ${{ secrets.ACR_NAME }}
TF_VAR_terraform_state_container_name:
${{ secrets.TF_STATE_CONTAINER }}
TF_VAR_terraform_state_container_name: ${{ secrets.TF_STATE_CONTAINER }}
TF_VAR_mgmt_resource_group_name: ${{ secrets.MGMT_RESOURCE_GROUP }}
TF_VAR_mgmt_storage_account_name:
${{ secrets.STATE_STORAGE_ACCOUNT_NAME }}
TF_VAR_mgmt_storage_account_name: ${{ secrets.STATE_STORAGE_ACCOUNT_NAME }}
TF_VAR_core_address_space: ${{ secrets.CORE_ADDRESS_SPACE }}
TF_VAR_tre_address_space: ${{ secrets.TRE_ADDRESS_SPACE }}
TF_VAR_swagger_ui_client_id: "${{ secrets.SWAGGER_UI_CLIENT_ID }}"
TF_VAR_api_client_id: "${{ secrets.API_CLIENT_ID }}"
TF_VAR_api_client_secret: "${{ secrets.API_CLIENT_SECRET }}"
TF_VAR_keyvault_purge_protection_enabled: "${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_stateful_resources_locked: "${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_keyvault_purge_protection_enabled:
"${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_stateful_resources_locked:
"${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"

publish_bundles:
name: Publish Bundles
Expand Down Expand Up @@ -404,6 +405,8 @@ jobs:
BUNDLE_DIR: "./templates/shared_services/sonatype-nexus/"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/gitea/"}
- {BUNDLE_TYPE: "user_resource",
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm"}
environment: CICD
steps:
- name: Checkout
Expand Down Expand Up @@ -437,7 +440,7 @@ jobs:
strategy:
fail-fast: true
matrix:
target: [build-and-push-guacamole]
target: [build-and-push-gitea, build-and-push-guacamole]

steps:
- name: Checkout
Expand Down Expand Up @@ -493,6 +496,9 @@ jobs:
BUNDLE_DIR: "./templates/shared_services/sonatype-nexus"}
- {BUNDLE_TYPE: "shared_service",
BUNDLE_DIR: "./templates/shared_services/gitea"}
- {BUNDLE_TYPE: "user_resource",
WORKSPACE_SERVICE_NAME: "tre-service-guacamole",
BUNDLE_DIR: "./templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm"}
environment: CICD
steps:
- name: Checkout
Expand Down Expand Up @@ -523,6 +529,7 @@ jobs:
TRE_ID: "${{ secrets.TRE_ID }}"
LOCATION: "${{ secrets.LOCATION }}"
BUNDLE_TYPE: ${{ matrix.BUNDLE_TYPE }}
WORKSPACE_SERVICE_NAME: ${{ matrix.WORKSPACE_SERVICE_NAME }}

deploy_shared_services:
name: Deploy shared services
Expand All @@ -538,7 +545,7 @@ jobs:
# then the default checkout will apply
ref: ${{ inputs.prRef }}

- name: Register/deploy firewall
- name: Deploy firewall
uses: ./.github/actions/devcontainer_run_command
with:
COMMAND: "make deploy-shared-service DIR=./templates/shared_services/firewall/ BUNDLE_TYPE=shared_service"
Expand Down Expand Up @@ -581,18 +588,18 @@ jobs:
TRE_ID: "${{ secrets.TRE_ID }}"
LOCATION: ${{ secrets.LOCATION }}
ACR_NAME: ${{ secrets.ACR_NAME }}
TF_VAR_terraform_state_container_name:
${{ secrets.TF_STATE_CONTAINER }}
TF_VAR_terraform_state_container_name: ${{ secrets.TF_STATE_CONTAINER }}
TF_VAR_mgmt_resource_group_name: ${{ secrets.MGMT_RESOURCE_GROUP }}
TF_VAR_mgmt_storage_account_name:
${{ secrets.STATE_STORAGE_ACCOUNT_NAME }}
TF_VAR_mgmt_storage_account_name: ${{ secrets.STATE_STORAGE_ACCOUNT_NAME }}
TF_VAR_core_address_space: ${{ secrets.CORE_ADDRESS_SPACE }}
TF_VAR_tre_address_space: ${{ secrets.TRE_ADDRESS_SPACE }}
TF_VAR_swagger_ui_client_id: "${{ secrets.SWAGGER_UI_CLIENT_ID }}"
TF_VAR_api_client_id: "${{ secrets.API_CLIENT_ID }}"
TF_VAR_api_client_secret: "${{ secrets.API_CLIENT_SECRET }}"
TF_VAR_keyvault_purge_protection_enabled: "${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_stateful_resources_locked: "${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_keyvault_purge_protection_enabled:
"${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"
TF_VAR_stateful_resources_locked:
"${{ github.ref == 'refs/heads/main' && inputs.prRef == '' && true || false }}"

e2e_tests_smoke:
name: "Run E2E Tests (Smoke)"
Expand Down Expand Up @@ -659,13 +666,14 @@ jobs:
with:
files: "./e2e_tests/pytest_e2e_smoke.xml"

e2e_tests_extended:
name: "Run E2E Tests (Extended)"
if: ${{ inputs.runExtendedTests }}

e2e_tests_custom:
name: "Run E2E Tests"
if: ${{ inputs.e2eTestsCustomSelector != '' }}
runs-on: ubuntu-latest
environment: CICD
needs: [deploy_shared_services, build_additional_images]
timeout-minutes: 50
timeout-minutes: 120
steps:
- name: Checkout
uses: actions/checkout@v2
Expand All @@ -675,10 +683,10 @@ jobs:
# then the default checkout will apply
ref: ${{ inputs.prRef }}

- name: Run E2E Tests (Extended)
- name: Run E2E Tests
uses: ./.github/actions/devcontainer_run_command
with:
COMMAND: "make test-e2e-extended"
COMMAND: "make test-e2e-custom SELECTOR='${{ inputs.e2eTestsCustomSelector }}'"
ACTIONS_ACR_NAME: ${{ secrets.ACTIONS_ACR_NAME }}
ACTIONS_ACR_URI: ${{ secrets.ACTIONS_ACR_URI }}
ACTIONS_ACR_PASSWORD: ${{ secrets.ACTIONS_ACR_PASSWORD }}
Expand All @@ -702,14 +710,14 @@ jobs:
if: always()
uses: actions/upload-artifact@v2
with:
name: E2E Test (Extended) Results
path: "./e2e_tests/pytest_e2e_extended.xml"
name: E2E Test Results
path: "./e2e_tests/pytest_e2e_custom.xml"

- name: Publish Test Results
if: always()
uses: EnricoMi/publish-unit-test-result-action@v1
with:
files: "./e2e_tests/pytest_e2e_extended.xml"
files: "./e2e_tests/pytest_e2e_custom.xml"

e2e_tests_shared_services:
name: "Run E2E Tests (Shared Services)"
Expand Down Expand Up @@ -765,7 +773,7 @@ jobs:

summary:
name: Summary Notification
needs: [e2e_tests_smoke, e2e_tests_extended, e2e_tests_shared_services]
needs: [e2e_tests_smoke, e2e_tests_custom]
runs-on: ubuntu-latest
if: ${{ always() && (github.ref == 'refs/heads/main' && inputs.prRef == '') }}
environment: CICD
Expand Down
10 changes: 7 additions & 3 deletions .github/workflows/pr_comment_bot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -134,15 +134,19 @@ jobs:
run_test:
# Run the tests with the re-usable workflow
needs: [pr_comment]
if: ${{ needs.pr_comment.outputs.command == 'run-tests' || needs.pr_comment.outputs.command == 'run-tests-extended' || needs.pr_comment.outputs.command == 'run-tests-shared-services' }}
if: |
${{ needs.pr_comment.outputs.command == 'run-tests' ||
needs.pr_comment.outputs.command == 'run-tests-extended' ||
needs.pr_comment.outputs.command == 'run-tests-shared-services' }}
name: Deploy PR
uses: ./.github/workflows/deploy_tre_reusable.yml
with:
prRef: ${{ needs.pr_comment.outputs.prRef }}
prHeadSha: ${{ needs.pr_comment.outputs.prHeadSha }}
ciGitRef: ${{ needs.pr_comment.outputs.ciGitRef }}
runExtendedTests: ${{ needs.pr_comment.outputs.command == 'run-tests-extended' }}
runSharedServicesTests: ${{ needs.pr_comment.outputs.command == 'run-tests-shared-services' }}
e2eTestsCustomSelector: >-
${{ (needs.pr_comment.outputs.command == 'run-tests-extended' && 'extended') ||
(needs.pr_comment.outputs.command == 'run-tests-shared-services' && 'shared_sevices') }}
secrets:
AAD_TENANT_ID: ${{ secrets.AAD_TENANT_ID }}
ACR_NAME: ${{ format('tre{0}', needs.pr_comment.outputs.prRefId) }}
Expand Down
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -201,8 +201,7 @@ templates/core/terraform/scripts/validation.txt
templates/core/terraform/plan

# Test results
e2e_tests/pytest_e2e_smoke.xml
e2e_tests/pytest_e2e_extended.xml
e2e_tests/pytest_e2e_*.xml
e2e_tests/workspace_id.txt
pytest_api_unit.xml
pytest_api_unit_failed
Expand Down
Loading

0 comments on commit 4989a1c

Please sign in to comment.