Enable private link for Azure Monitor in Workspaces

Builds upon #742

The AMPLS (defined in /templates/core/terraform/azure-monitor/azure-monitor.tf) should be able to support the Application Insights instances in Workspaces too.

Otherwise traffic to app insights is blocked by the firewall and creates many Deny logs.

Acceptance criteria

• Prototype the use of AMPLS for Workspace App Insights instances in Portal and verify that the logs sent by the Workspace/Workspace Service get through (note: may require new storage to be linked with App Insights, see other task below)
• If the aforementioned approach was successful, update the Workspace (Service) installation/uninstallation pipelines to add/remove App Insights and other Azure Monitor resources deployed by the Workspace (Service) to/from the AMPLS
  • The resource ID of AMPLS need to be provided as output by tre-deploy target - OR - provide it directly in Terraform to Resource Processor as an env variable to be then injected to Porter installations requiring it?
• App Insights behind Private Link requires a separately created, linked storage, see Configure Bring Your Own Storage (BYOS) for Application Insights
  • Use /templates/core/terraform/azure-monitor/azure-monitor.tf as reference

[stuartleeks]

Assumption: the desire is to have a separate App Insights instance per workspace (to preserve workspace isolation)

To enable connecting to Azure Monitor from within a workspace VNET (where traffic is restricted), we need to have an Azure Monitor Private Link Scope (AMPLS) that is connected to a Private Endpoint within the VNET.

An AMPLS can be connected to multiple Private Endpoints and multiple Azure Monitor resources, but an AMPLS can only connect to up to 10 Private Endpoints so the suggestion is to deploy an AMPLS per workspace for simplicity.

Because there are some shared endpoints (i.e. not resource-specific), a single AMPLS should be used for all VNETs that share the same DNS. Currently, we have separate VNETs for each workspace but each VNET is linked to the same, single private DNS Zone for Azure Monitor/App Insights. To enable an AMPLS per workspace, we need to update the private DNS Zones for Azure Monitor so that the existing zones are just used for the core VNET and deploy separate zones for each workspace.

Summary of proposed changes

• Update the existing Azure Monitor DNS zones to only be used for the core VNET (i.e. don’t link to workspace VNETs)
• For workspaces, add
  • DNS zones for Azure Monitor and link to the VNET
  • Azure Monitor Private Endpoints
  • An AMPLS connected to the Private Endpoint

[stuartleeks]

@marrobi - I’m in the middle of that currently (fitting it around other work)