-
Notifications
You must be signed in to change notification settings - Fork 297
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Automate Azure WebJobs testing #902
Comments
We need to automate the way we run tests. The steps to run MSI tests in Azure Web Jobs for the identity component are as follows: prerequisite tools
Azure resourcesThis test requires instances of these Azure resources:
Initial Setup:PLEASE NOTE: This setup is Java specific since it installs the Java & Maven environments. Other languages will have a different set up. Set environment variables to simplify copy-pasting
Create resources
az group create -n $RESOURCE_GROUP -l $LOCATION
az appservice plan create -n $PLAN_NAME -g $RESOURCE_GROUP -l $LOCATION --sku $PLAN_SKU
az webapp create -n $APP_NAME_SYSTEM_ASSIGNED -g $RESOURCE_GROUP --plan $PLAN_NAME
az webapp identity assign --name $APP_NAME_SYSTEM_ASSIGNED --resource-group $RESOURCE_GROUP
az identity create -n $MANAGED_IDENTITY_NAME -g $RESOURCE_GROUP -l $LOCATION
az webapp create -n $APP_NAME_USER_ASSIGNED -g $RESOURCE_GROUP --plan $PLAN_NAME
az keyvault create -n $KEY_VAULT_NAME -g $RESOURCE_GROUP Get the principal ID for the Web app with system assigned identity: $principalId=az webapp show -n $APP_NAME_SYSTEM_ASSIGNED -g $RESOURCE_GROUP --query "identity.principalId"
$principalId=$principalId.Replace('"', '') Allow the Web app with system assigned identity to access the key vault secrets: az keyvault set-policy -n $KEY_VAULT_NAME --object-id $principalId --secret-permissions get list Get the principal ID for the second Web app with user assigned identity: $principalId=az identity show -n $MANAGED_IDENTITY_NAME -g $RESOURCE_GROUP --query "principalId"
$principalId=$principalId.Replace('"', '') Allow the Web app with user assigned identity to access the key vault secrets: az keyvault set-policy -n $KEY_VAULT_NAME --object-id $principalId --secret-permissions get list Add a secret called "secret": az keyvault secret set --vault-name $KEY_VAULT_NAME -n secret --value "Secret value" Build the executable jarThe executable jar holds the logic to run tests. will be different per language.
mvn clean install -Dgpg.skip -f eng\code-quality-reports\pom.xml
mvn clean install -Dgpg.skip -Dcheckstyle.skip -Dspotbugs.skip -Drevapi.skip -Dmaven.javadoc.skip=true -DskipTests -f sdk/keyvault/pom.service.xml
mvn clean package -Dpackage-with-dependencies -f sdk\e2e\pom.xml sdk\e2e\target will contain an executable with jar with dependencies. Package, Deploy and run testsRun System Assigned Web Job Test
mkdir systemAssignedTest
$content="set AZURE_VAULT_URL=https://$KEY_VAULT_NAME.vault.azure.net"
$content += "`r`n"
$content +="set AZURE_WEBJOBS_TEST_MODE=system"
$content += "`r`n"
$content +="set AZURE_IDENTITY_TEST_PLATFORM=webjobs"
$content += "`r`n"
$content +="java -jar azure-e2e-1.0.0-beta.1-jar-with-dependencies.jar"
Set-Content -Value $content -Path '<Relative-path-to-systemAssignedTest-directory>\run.bat' Create system assigned test web job deployment file $netFrameworkVersion="v<YOUR-DOT-NET-FRAMEWORK-VERSION>"
Add-Type -Path "C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\$netFrameworkVersion\System.IO.Compression.FileSystem.dll"
$systemAssignedWebJobzip = "<OUTPUT_PATH>\systemwebjob.zip"
if(Test-path $systemAssignedWebJobzip) {Remove-item $systemAssignedWebJobzip}
[io.compression.zipfile]::CreateFromDirectory("<path-to-systemAssignedTest-directory>", $systemAssignedWebJobzip)
$user = az webapp deployment list-publishing-profiles -n $APP_NAME_SYSTEM_ASSIGNED -g $RESOURCE_GROUP `
--query "[?publishMethod=='MSDeploy'].userName" -o tsv $pass = az webapp deployment list-publishing-profiles -n $APP_NAME_SYSTEM_ASSIGNED -g $RESOURCE_GROUP `
--query "[?publishMethod=='MSDeploy'].userPWD" -o tsv $creds = "$($user):$($pass)"
$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($creds))
$basicAuthValue = "Basic $encodedCreds" Deploy the system assigned web job $ZipHeaders = @{
Authorization = $basicAuthValue
"Content-Disposition" = "attachment; filename=run.cmd"
}
# upload the system assigned job using the Kudu WebJobs API
Invoke-WebRequest -Uri https://$APP_NAME_SYSTEM_ASSIGNED.scm.azurewebsites.net/api/triggeredwebjobs/Systemassignedjob -Headers $ZipHeaders `
-InFile $systemAssignedWebJobzip -ContentType "application/zip" -Method Put
$Headers = @{
Authorization = $basicAuthValue
} Run the system assigned web job $resp = Invoke-WebRequest -Uri "https://$APP_NAME_SYSTEM_ASSIGNED.scm.azurewebsites.net/api/triggeredwebjobs/Systemassignedjob/run?arguments=eggs bacon" -Headers $Headers `
-Method Post -ContentType "multipart/form-data" Print the web job output log: if ($resp.RawContent -match "\nLocation\: (.+)\n")
{
$historyLocation = $matches[1]
$hist = Invoke-RestMethod -Uri $historyLocation -Headers $Headers -Method Get
# $hist has status, start_time, end_time, duration, error_url etc
# get the logs from output_url
Invoke-RestMethod -Uri $hist.output_url -Headers $Headers -Method Get
} Run User Assigned Web job Test
mkdir userAssignedTest
$clientId=az identity show -n $MANAGED_IDENTITY_NAME -g $RESOURCE_GROUP --query "clientId"
$clientId=$clientId.Replace('"', '')
$content="set AZURE_VAULT_URL=https://$KEY_VAULT_NAME.vault.azure.net"
$content += "`r`n"
$content +="set AZURE_WEBJOBS_TEST_MODE=user"
$content += "`r`n"
$content +="set AZURE_CLIENT_ID=$clientId"
$content += "`r`n"
$content +="set AZURE_IDENTITY_TEST_PLATFORM=webjobs"
$content += "`r`n"
$content +="java -jar azure-e2e-1.0.0-beta.1-jar-with-dependencies.jar"
Set-Content -Value $content -Path '<Relative-Path-To-userAssignedTest-directory>\run.bat' Create user assigned test web job deployment file $netFrameworkVersion="v<YOUR-DOT-NET-FRAMEWORK-VERSION>"
Add-Type -Path "C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\$netFrameworkVersion\System.IO.Compression.FileSystem.dll"
$userAssignedWebJobzip = "<OUTPUT_PATH>\userwebjob.zip"
if(Test-path $userAssignedWebJobzip) {Remove-item $userAssignedWebJobzip}
[io.compression.zipfile]::CreateFromDirectory("<path-to-userAssignedTest-directory>", $userAssignedWebJobzip)
$user = az webapp deployment list-publishing-profiles -n $APP_NAME_USER_ASSIGNED -g $RESOURCE_GROUP `
--query "[?publishMethod=='MSDeploy'].userName" -o tsv $pass = az webapp deployment list-publishing-profiles -n $APP_NAME_USER_ASSIGNED -g $RESOURCE_GROUP `
--query "[?publishMethod=='MSDeploy'].userPWD" -o tsv $creds = "$($user):$($pass)"
$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($creds))
$basicAuthValue = "Basic $encodedCreds" Deploy the system assigned web job $ZipHeaders = @{
Authorization = $basicAuthValue
"Content-Disposition" = "attachment; filename=run.cmd"
}
# upload the user assigned job using the Kudu WebJobs API
Invoke-WebRequest -Uri https://$APP_NAME_USER_ASSIGNED.scm.azurewebsites.net/api/triggeredwebjobs/Userassignedjob -Headers $ZipHeaders `
-InFile $userAssignedWebJobzip -ContentType "application/zip" -Method Put
$Headers = @{
Authorization = $basicAuthValue
} Run the user assigned web job $resp = Invoke-WebRequest -Uri "https://$APP_NAME_USER_ASSIGNED.scm.azurewebsites.net/api/triggeredwebjobs/Userassignedjob/run?arguments=eggs bacon" -Headers $Headers `
-Method Post -ContentType "multipart/form-data" Print the web job output log: if ($resp.RawContent -match "\nLocation\: (.+)\n")
{
$historyLocation = $matches[1]
$hist = Invoke-RestMethod -Uri $historyLocation -Headers $Headers -Method Get
# $hist has status, start_time, end_time, duration, error_url etc
# get the logs from output_url
Invoke-RestMethod -Uri $hist.output_url -Headers $Headers -Method Get
} |
Hi @g2vinay , I am following above steps to do java E2E test. My suggestion as below. Please correct me if I miss anything. 1.Since the /keyvault /pom.service.xml file has been deleted by this PR, we need change sdk /keyvault/pom.service.xml to sdk/keyvault /pom.xml in Build the executable jar step. 2.Commands with .skip fails with below error. So we need to inclose Dgpg.skip with quotation marks.
|
For JavaScript: InstructionsFollow these instructions using PowerShell. set environment variables to simplify copy-pasting
App ServiceCreate the app service plan
Create a web app
Enable System assigned identity on the web app.
Resource Groupaz group create -n $RESOURCE_GROUP --location westus2 Managed IdentityCreate the managed identity: az identity create -g $RESOURCE_GROUP -n $MANAGED_IDENTITY_NAME Save its $MANAGED_IDENTITY_PRINCIPAL_ID=az identity show -g $RESOURCE_GROUP -n $MANAGED_IDENTITY_NAME --query principalId -o tsv Key VaultCreate the Vault: az keyvault create -g $RESOURCE_GROUP -n $KEY_VAULT_NAME --sku standard Add an access policy for the managed identity: az keyvault set-policy -n $KEY_VAULT_NAME --object-id $MANAGED_IDENTITY_PRINCIPAL_ID --secret-permissions set delete Web App
Build the webapp
Install the requirements:
Build the job:
Zip the App_Data directory, and all its contents, into "AzureTestJob.zip" Upload the job
Trigger the job
Test the job completed successfully
Delete Azure resourcesAfter running tests, delete the resources provisioned earlier: az group delete -n $RESOURCE_GROUP -y --no-wait |
Hi @jonathandturner, I am following above steps to do JS E2E test. My suggestion as below. Please correct me if I miss anything. If I run Anyway, I tried to run the Could you please help to check the test steps and update them? |
@XuGuang-Yao - apologies, yes, it should be I've updated the instructions, please try again. |
@jonathandturner , -Thanks you for your quick reply. As I mentioned before, I have tried to run the |
Re-opening issue; @danieljurek will handle #1539 and once #1539 is resolved then @g2vinay can help onboard #902 with Java first, and the rest of the languages next. |
Steps to test for Golang: Prerequisites
Azure resourcesThis test requires instances of these Azure resources:
Initial SetupSet the following environment variables to simplify copy-pasting
In a powershell with Azure CLI, create the resource group. This step is optional, skip if resource group already exists.
Create the app service plan
Create a web app
Enable System assigned identity on the web app.
Create a user identity for the second web app:
Create the second web app:
Assign the user Identity from Step 5 to the second web app.
Build the webappDownload the Go SDK:
Go to the samples folder
Build the project
This should generate a Execute the following commands to set up for the headers that will be sent with the requests
Set the header variables that will be used when uploading and triggering the jobs
Upload the job
Trigger the job
You can run Check the resultsTo get information about the webjob that was triggered run the following command until you see that it changes status from
To see the response run Once you see that the request was successful, run the following command:
Copy the url under
|
Hi @g2vinay Following above steps, when run system Assigned Web job Test about java E2E, we encountered an error after executing the following command:
The detailed error information is as follows: Could you help to check and resolve the error? |
@catalinaperalta @chlowell Following above steps, encountered some issues about Golang E2E Test. Firstly, the following command, Then, when run Could you help to check and resolve the error? |
Hi @g2vinay,@jntrnr Following above steps, encountered some issues about java and JavaScript E2E Test. we encountered an error after executing the command:
or
The error messages of java and JavaScript are the same, and the detailed error messages are as follows: Could you help to check and resolve the error? |
Hi @joshfree, we deeply appreciate your input into this project. Regrettably, this issue has remained inactive for over 2 years, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support. |
@joshfree This issue has been closed. Do we need to continue testing this |
Tracking issue for step-by-step instructions for how to manually run e2e tests for Azure WebJobs hosted environments for each of the azure-sdk-for-* languages. This issue will be used by our Vendor team for manually running tests in this scenario, and will be tracked on the EngSys backlog for eventual automation.
Assigning to @g2vinay who will create the initial instructions specific to Java Azure SDKs. Other SDK languages will be appended to this issue as further comments.
The text was updated successfully, but these errors were encountered: