Fixing the Microsoft.Insights Resource Provider Registration

[tombuildsstuff]

Upon some investigation it appears the Resource Provider API is case-sensitive - this PR fixes the registration for Microsoft.Insights to be microsoft.insights so we don't re-register every time the plugin's re-initialized

Before:

2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.Cdn
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.Compute
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.ContainerRegistry
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.ContainerService
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.DocumentDB
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.EventGrid
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.EventHub
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.KeyVault
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.Network
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.Search
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.ServiceBus
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.Sql
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.Storage
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Skipping provider registration for namespace Microsoft.Resources
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] Registering provider with namespace Microsoft.Insights
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:14 [DEBUG] AzureRM Request:
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: POST /subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Insights/register?api-version=2016-09-01 HTTP/1.1
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: Host: management.azure.com
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: User-Agent: HashiCorp-Terraform-v0.10.0-dev
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: Content-Length: 0
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm: Accept-Encoding: gzip
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm:
2017/08/29 07:06:14 [DEBUG] plugin: terraform-provider-azurerm:
2017/08/29 07:06:18 [DEBUG] dag/walk: vertex "azurerm_subnet.test", waiting for: "azurerm_virtual_network.test"
2017/08/29 07:06:18 [DEBUG] dag/walk: vertex "azurerm_lb.test", waiting for: "azurerm_public_ip.test"
2017/08/29 07:06:18 [DEBUG] dag/walk: vertex "azurerm_lb_nat_pool.lbnatpool", waiting for: "azurerm_lb.test"
2017/08/29 07:06:18 [DEBUG] dag/walk: vertex "azurerm_resource_group.test", waiting for: "provider.azurerm"
2017/08/29 07:06:18 [DEBUG] dag/walk: vertex "azurerm_lb_backend_address_pool.bpepool", waiting for: "azurerm_lb.test"
2017/08/29 07:06:18 [DEBUG] dag/walk: vertex "azurerm_public_ip.test", waiting for: "azurerm_resource_group.test"
2017/08/29 07:06:18 [DEBUG] dag/walk: vertex "azurerm_virtual_machine_scale_set.test", waiting for: "azurerm_lb_nat_pool.lbnatpool"
2017/08/29 07:06:18 [DEBUG] dag/walk: vertex "azurerm_virtual_network.test", waiting for: "azurerm_resource_group.test"
2017/08/29 07:06:18 [DEBUG] dag/walk: vertex "provider.azurerm (close)", waiting for: "azurerm_virtual_machine_scale_set.test"
2017/08/29 07:06:23 [DEBUG] dag/walk: vertex "provider.azurerm (close)", waiting for: "azurerm_virtual_machine_scale_set.test"
2017/08/29 07:06:23 [DEBUG] dag/walk: vertex "azurerm_resource_group.test", waiting for: "provider.azurerm"
2017/08/29 07:06:23 [DEBUG] dag/walk: vertex "azurerm_virtual_machine_scale_set.test", waiting for: "azurerm_lb_nat_pool.lbnatpool"
2017/08/29 07:06:23 [DEBUG] dag/walk: vertex "azurerm_lb.test", waiting for: "azurerm_public_ip.test"
2017/08/29 07:06:23 [DEBUG] dag/walk: vertex "azurerm_lb_backend_address_pool.bpepool", waiting for: "azurerm_lb.test"
2017/08/29 07:06:23 [DEBUG] dag/walk: vertex "azurerm_virtual_network.test", waiting for: "azurerm_resource_group.test"
2017/08/29 07:06:23 [DEBUG] dag/walk: vertex "azurerm_subnet.test", waiting for: "azurerm_virtual_network.test"
2017/08/29 07:06:23 [DEBUG] dag/walk: vertex "azurerm_lb_nat_pool.lbnatpool", waiting for: "azurerm_lb.test"
2017/08/29 07:06:23 [DEBUG] dag/walk: vertex "azurerm_public_ip.test", waiting for: "azurerm_resource_group.test"
2017/08/29 07:06:28 [DEBUG] dag/walk: vertex "azurerm_lb.test", waiting for: "azurerm_public_ip.test"
2017/08/29 07:06:28 [DEBUG] dag/walk: vertex "azurerm_subnet.test", waiting for: "azurerm_virtual_network.test"
2017/08/29 07:06:28 [DEBUG] dag/walk: vertex "azurerm_lb_backend_address_pool.bpepool", waiting for: "azurerm_lb.test"
2017/08/29 07:06:28 [DEBUG] dag/walk: vertex "provider.azurerm (close)", waiting for: "azurerm_virtual_machine_scale_set.test"
2017/08/29 07:06:28 [DEBUG] dag/walk: vertex "azurerm_virtual_machine_scale_set.test", waiting for: "azurerm_lb_nat_pool.lbnatpool"
2017/08/29 07:06:28 [DEBUG] dag/walk: vertex "azurerm_lb_nat_pool.lbnatpool", waiting for: "azurerm_lb.test"
2017/08/29 07:06:28 [DEBUG] dag/walk: vertex "azurerm_virtual_network.test", waiting for: "azurerm_resource_group.test"
2017/08/29 07:06:28 [DEBUG] dag/walk: vertex "azurerm_resource_group.test", waiting for: "provider.azurerm"
2017/08/29 07:06:28 [DEBUG] dag/walk: vertex "azurerm_public_ip.test", waiting for: "azurerm_resource_group.test"
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:06:29 [DEBUG] AzureRM Response for https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Insights/register?api-version=2016-09-01:
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: HTTP/1.1 500 Internal Server Error
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: Connection: close
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: Content-Length: 312
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: Cache-Control: no-cache
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: Content-Type: application/json; charset=utf-8
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: Date: Tue, 29 Aug 2017 06:06:28 GMT
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: Expires: -1
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: Pragma: no-cache
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: Strict-Transport-Security: max-age=31536000; includeSubDomains
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: X-Ms-Correlation-Request-Id: 03654298-3f5d-4c78-bbf2-e3d8a8c7d23f
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: X-Ms-Failure-Cause: gateway
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: X-Ms-Ratelimit-Remaining-Subscription-Writes: 1199
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: X-Ms-Request-Id: 03654298-3f5d-4c78-bbf2-e3d8a8c7d23f
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: X-Ms-Routing-Request-Id: UKWEST:20170829T060628Z:03654298-3f5d-4c78-bbf2-e3d8a8c7d23f
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm:
2017/08/29 07:06:29 [DEBUG] plugin: terraform-provider-azurerm: {"error":{"code":"InternalServerError","message":"Encountered internal server error. Diagnostic information: timestamp '20170829T060628Z', subscription id 'c0a607b2-6372-4ef3-abdb-dbe52a7b56ba', tracking id '03654298-3f5d-4c78-bbf2-e3d8a8c7d23f', request correlation id '03654298-3f5d-4c78-bbf2-e3d8a8c7d23f'."}}

After:

2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.Cache
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.Cdn
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.Compute
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.ContainerRegistry
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.ContainerService
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.DocumentDB
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.EventGrid
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.EventHub
2017/08/29 07:15:36 [TRACE] [walkRefresh] Exiting eval tree: provider.azurerm
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace microsoft.insights
2017/08/29 07:15:36 [DEBUG] dag/walk: walking "azurerm_resource_group.test"
2017/08/29 07:15:36 [DEBUG] vertex 'root.azurerm_resource_group.test': walking
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.KeyVault
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.Network
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.Search
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.ServiceBus
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.Sql
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.Storage
2017/08/29 07:15:36 [DEBUG] plugin: terraform-provider-azurerm: 2017/08/29 07:15:36 [DEBUG] Skipping provider registration for namespace Microsoft.Resources

[radeksimko]

Do you mind explaining how did you discover this bug? If it's already used in any resource then creation of such resource would fail as part of acceptance test I'd assume?

In other words is there any way we could've prevented this bug from appearing and can we do it retroactively?

[tombuildsstuff]

@radeksimko

Do you mind explaining how did you discover this bug? If it's already used in any resource then creation of such resource would fail as part of acceptance test I'd assume?

Context: Resource Provider registration is required to be able to use the API associated with the Resource Provider.

During debugging an issue I noticed this Provider was being re-registering multiple times with the debug logs enabled (the provider's init'd multiple times - each time at startup we check to see that the resource providers are registered). This won't affect many users since this is enabled by default, but there's still the potential for this to be de-registered.

In other words is there any way we could've prevented this bug from appearing and can we do it retroactively?

We're deliberately catching any errors when registering the Resource Providers, since users may not have permissions to register all of them / they may not all be available in that region, as with Germany and Azure Stack - where they're add-ons. Whilst there is an environment variable to permit resource provider registration to be skipped which would solve the first case - implementing a whitelist is the wrong approach since additional Resource Providers can be installed in some circumstances.

After some 🤔 we could probably add a test to try registering the providers, then query again and see if any remain to be registered?

[tombuildsstuff]

@radeksimko added a test covering the Resource Provider registration:

Result of this test before this change:

2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.Cache
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.Cdn
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.Compute
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.ContainerRegistry
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.ContainerService
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.DocumentDB
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.EventGrid
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.EventHub
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.KeyVault
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.Network
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.Search
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.ServiceBus
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.Sql
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.Storage
2017/08/29 11:18:56 [DEBUG] Skipping provider registration for namespace Microsoft.Resources
--- FAIL: TestAzureRMResourceProviderRegistration (3.31s)
	provider_test.go:119: '1' Resource Providers are still Pending Registration: map[Microsoft.Insights:{}]
FAIL
FAIL	github.com/terraform-providers/terraform-provider-azurerm/azurerm	3.329s

Now:

$ envchain azurerm go test ./azurerm -timeout 120m -run TestAzureRMResourceProviderRegistration
ok  	github.com/terraform-providers/terraform-provider-azurerm/azurerm	1.630s

[grubernaut]

LGTM!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!