Deep Dive: Azure Virtual Network Peering, Service Endpoints, and Private Link

The Azure Virtual Network (VNet) is the foundational building block for your private network in the cloud. While many are familiar with its basic capabilities like subnets, NSGs, and VPN gateways, true mastery of Azure’s networking requires understanding its more advanced features. This deep-dive explores the ‘under-the-hood’ mechanisms of VNet Peering, Service Endpoints, and Private Link, revealing how they enable secure, efficient, and scalable network architectures.

We’ll move beyond generic overviews to dissect their architectural implications, practical implementation challenges, and how they solve real-world connectivity problems.

1. Azure VNet Peering: Connecting Your Private Networks Seamlessly

Azure VNet Peering allows you to connect two or more Azure Virtual Networks, making them appear as one for connectivity purposes. Traffic between peered VNets uses Microsoft’s backbone infrastructure, benefiting from its low latency and high bandwidth. Crucially, this traffic remains entirely private and encrypted.

How VNet Peering Works Under the Hood

When VNets are peered, Azure injects routes into the effective route table of VMs within each VNet. These routes point to the address spaces of the peered VNets. Unlike traditional VPN gateways, VNet peering doesn’t route traffic through a gateway appliance, offering direct connectivity. This direct path is why peering offers superior performance compared to gateway-based solutions for inter-VNet communication.

Key Architectural Implication: Non-Transitive Routing

A critical concept to grasp is that VNet peering is non-transitive. If VNet A is peered with VNet B, and VNet B is peered with VNet C, VNet A cannot directly communicate with VNet C through VNet B. You’d need to explicitly peer VNet A with VNet C for direct communication. This is fundamental for designing hub-spoke topologies.

Regional vs. Global VNet Peering

Practical Implementation: VNet Peering via Azure CLI

Let’s assume we have two VNets: vnet-prod-weu (10.0.0.0/16) and vnet-dev-weu (10.1.0.0/16) in West Europe.

# Create Resource Group
az group create --name "rg-network-weu" --location "westeurope"

# Create vnet-prod-weu
az network vnet create \
    --resource-group "rg-network-weu" \
    --name "vnet-prod-weu" \
    --address-prefix "10.0.0.0/16" \
    --subnet-name "snet-app" \
    --subnet-prefix "10.0.0.0/24"

# Create vnet-dev-weu
az network vnet create \
    --resource-group "rg-network-weu" \
    --name "vnet-dev-weu" \
    --address-prefix "10.1.0.0/16" \
    --subnet-name "snet-web" \
    --subnet-prefix "10.1.0.0/24"

# Peer vnet-prod-weu to vnet-dev-weu
az network vnet peering create \
    --resource-group "rg-network-weu" \
    --name "ProdToDevPeering" \
    --vnet-name "vnet-prod-weu" \
    --remote-vnet "vnet-dev-weu" \
    --allow-vnet-access \
    --allow-forwarded-traffic \
    --allow-gateway-transit # Only if vnet-prod-weu has a gateway you want to use

# Peer vnet-dev-weu to vnet-prod-weu (bidirectional peering is required)
az network vnet peering create \
    --resource-group "rg-network-weu" \
    --name "DevToProdPeering" \
    --vnet-name "vnet-dev-weu" \
    --remote-vnet "vnet-prod-weu" \
    --allow-vnet-access \
    --allow-forwarded-traffic \
    --use-remote-gateways # Only if vnet-dev-weu wants to use vnet-prod-weu's gateway

# Verify peering status
az network vnet peering list --resource-group "rg-network-weu" --vnet-name "vnet-prod-weu" --output table
az network vnet peering list --resource-group "rg-network-weu" --vnet-name "vnet-dev-weu" --output table

Notice the --allow-gateway-transit and --use-remote-gateways flags. These are crucial for hub-spoke scenarios where spokes need to leverage a VPN/ExpressRoute gateway deployed in the hub VNet.

2. Securing PaaS Services: Service Endpoints vs. Private Link

Connecting your Azure IaaS resources (VMs) to Azure PaaS services (Azure SQL Database, Storage Accounts, Key Vault) securely is paramount. Azure offers two primary mechanisms: Service Endpoints and Private Link, each with distinct architectural implications.

2.1 Azure Service Endpoints: Extending Your VNet Perimeter

Service Endpoints allow you to extend your VNet’s private address space to Azure PaaS services. When enabled on a subnet, traffic from that subnet to supported PaaS services travels directly over the Azure backbone, bypassing the public internet. The PaaS service then only accepts traffic from specific VNet subnets and not from public IPs.

How Service Endpoints Work Under the Hood

When a Service Endpoint is enabled for a service (e.g., Microsoft.Storage) on a subnet: 1. Route Table Injection: A system route is injected into the effective route table of the subnet. This route has a more specific prefix for the PaaS service (e.g., 13.x.x.x/32 for a specific storage account) and directs traffic to the service’s private backbone endpoint instead of its public IP. 2. Service Delegation: The subnet is ‘delegated’ to the specific service. While not a full delegation, it’s an indication that the subnet can host resources that use this endpoint. 3. Firewall Rules: The PaaS service’s firewall can then be configured to only allow traffic from the source VNet’s subnet(s), effectively blocking all public internet access to the service.

Analogy: Think of Service Endpoints as giving your subnet a VIP pass to a specific PaaS service’s private entrance within the Azure data center, rather than having to use the crowded public entrance.

Limitations: * Regional Scope: Service Endpoints are regional. You cannot connect a VNet in West Europe to a Storage Account in East US using a Service Endpoint. * Public IP still present: The PaaS service itself still has a public IP address, though only traffic from allowed subnets can reach it via the Service Endpoint route. * Service-Specific: You need to enable them per-service (e.g., Storage, SQL, Key Vault).

Practical Implementation: Service Endpoints via Azure CLI

# Assume existing VNet 'vnet-prod-weu' and subnet 'snet-app'
# Create a Storage Account
az storage account create \
    --resource-group "rg-network-weu" \
    --name "mydatastorage12345" \
    --location "westeurope" \
    --sku "Standard_LRS"

# Enable Service Endpoint for Microsoft.Storage on snet-app
az network vnet subnet update \
    --resource-group "rg-network-weu" \
    --vnet-name "vnet-prod-weu" \
    --name "snet-app" \
    --service-endpoints "Microsoft.Storage"

# Get the ID of the 'snet-app' subnet
SUBNET_ID=$(az network vnet subnet show \
    --resource-group "rg-network-weu" \
    --vnet-name "vnet-prod-weu" \
    --name "snet-app" \
    --query "id" -o tsv)

# Configure Storage Account firewall to allow traffic only from 'snet-app'
az storage account network-rule add \
    --resource-group "rg-network-weu" \
    --account-name "mydatastorage12345" \
    --subnet "$SUBNET_ID"

# Set default action to deny all other traffic
az storage account update \
    --resource-group "rg-network-weu" \
    --name "mydatastorage12345" \
    --default-action Deny

# Verify the subnet service endpoints
az network vnet subnet show \
    --resource-group "rg-network-weu" \
    --vnet-name "vnet-prod-weu" \
    --name "snet-app" \
    --query "serviceEndpoints" -o json

2.2 Azure Private Link: Bringing PaaS into Your VNet

Azure Private Link takes secure PaaS connectivity a step further. Instead of extending your VNet to the PaaS service, Private Link brings the PaaS service into your VNet by provisioning a Private Endpoint. This Private Endpoint is a network interface (NIC) with a private IP address from your VNet, associated with a specific PaaS resource.

How Private Link Works Under the Hood

1. Private Endpoint Creation: You create a Private Endpoint resource within a specific subnet of your VNet. This endpoint is configured to connect to a specific PaaS resource (e.g., mydatastorage12345.blob.core.windows.net).
2. NIC Provisioning: A NIC is provisioned in the designated subnet, assigned a private IP address from that subnet’s range.
3. DNS Resolution: This is critical. By default, PaaS services resolve to their public IP. For Private Link to work, you must ensure that the DNS name of the PaaS service resolves to the private IP of your Private Endpoint from within your VNet. This is typically achieved using an Azure Private DNS Zone (privatelink.blob.core.windows.net for Azure Blob Storage, for example) linked to your VNet.
4. Traffic Flow: When a resource in your VNet tries to connect to mydatastorage12345.blob.core.windows.net, the DNS resolution provides the Private Endpoint’s private IP. Traffic then flows entirely within your VNet to the Private Endpoint, and from there over the Microsoft backbone to the PaaS service, completely bypassing the public internet and any public IP addresses.

Analogy: Private Link is like giving your PaaS service its own dedicated, private extension cord that plugs directly into an outlet within your VNet, making it appear as if the service is natively hosted within your VNet.

Advantages over Service Endpoints: * True Private IP: No public IP involved at any point for VNet-to-PaaS traffic. * Cross-Regional: Private Endpoints can connect to PaaS services in any Azure region. * Cross-Subscription/Tenant: Can securely connect services across different Azure subscriptions or even Azure AD tenants. * Consistent Experience: All traffic (data plane) is treated as VNet internal traffic.

Practical Implementation: Private Link via Azure CLI

# Assume existing VNet 'vnet-prod-weu' and subnet 'snet-app'
# Assume existing Storage Account 'mydatastorage12345'

# Create a subnet specifically for Private Endpoints (optional but recommended)
az network vnet subnet create \
    --resource-group "rg-network-weu" \
    --vnet-name "vnet-prod-weu" \
    --name "snet-private-endpoints" \
    --address-prefix "10.0.1.0/24" \
    --disable-private-endpoint-network-policies true # IMPORTANT: Disable network policies

# Get the subnet ID for Private Endpoint
PE_SUBNET_ID=$(az network vnet subnet show \
    --resource-group "rg-network-weu" \
    --vnet-name "vnet-prod-weu" \
    --name "snet-private-endpoints" \
    --query "id" -o tsv)

# Get Storage Account resource ID for linking
STORAGE_ID=$(az storage account show \
    --resource-group "rg-network-weu" \
    --name "mydatastorage12345" \
    --query "id" -o tsv)

# Create a Private Endpoint for the Storage Account (Blob service)
az network private-endpoint create \
    --resource-group "rg-network-weu" \
    --name "pe-storage-blob" \
    --location "westeurope" \
    --vnet-name "vnet-prod-weu" \
    --subnet "$PE_SUBNET_ID" \
    --private-connection-resource-id "$STORAGE_ID" \
    --group-id "blob" \
    --connection-name "storage-blob-connection"

# Get the Private IP address of the newly created Private Endpoint NIC
PRIVATE_IP=$(az network private-endpoint show \
    --resource-group "rg-network-weu" \
    --name "pe-storage-blob" \
    --query "networkInterfaces[0].ipConfigurations[0].privateIpAddress" -o tsv)

# Create a Private DNS Zone for Blob Storage
az network private-dns zone create \
    --resource-group "rg-network-weu" \
    --name "privatelink.blob.core.windows.net"

# Link the Private DNS Zone to 'vnet-prod-weu'
az network private-dns link vnet create \
    --resource-group "rg-network-weu" \
    --zone-name "privatelink.blob.core.windows.net" \
    --name "vnet-prod-link" \
    --virtual-network "vnet-prod-weu" \
    --registration-enabled false # Important: No auto-registration for PaaS

# Create a DNS A record in the Private DNS Zone pointing to the Private Endpoint IP
az network private-dns record-set a add-record \
    --resource-group "rg-network-weu" \
    --zone-name "privatelink.blob.core.windows.net" \
    --record-set-name "mydatastorage12345" \
    --ipv4-address "$PRIVATE_IP"

# Update Storage Account firewall to allow only Private Link traffic
az storage account update \
    --resource-group "rg-network-weu" \
    --name "mydatastorage12345" \
    --default-action Deny \
    --bypass AzureServices # Bypass necessary for other Azure services (e.g., backup) to access the storage account

Note on disable-private-endpoint-network-policies true: This is crucial. By default, Network Security Groups (NSGs) and User-Defined Routes (UDRs) apply to Private Endpoint traffic. Disabling network policies on the Private Endpoint subnet ensures that Private Endpoint traffic is correctly routed and not inadvertently blocked by NSGs or UDRs applied to that subnet. This does NOT mean NSGs on other subnets are ignored.

Service Endpoints vs. Private Link: A Comparison

3. Architectural Pattern: Hub-Spoke with Secure PaaS Access

Let’s visualize how these concepts come together in a common hub-spoke architecture.

graph LR
    subgraph Hub VNet (10.0.0.0/16)
        HVN["Hub VNet"]
        GW(GatewaySubnet)
        FW(Azure Firewall Subnet)
        PED(Private Endpoint Subnet)
        HVN --- GW
        HVN --- FW
        HVN --- PED
    end

    subgraph Spoke VNet A (10.1.0.0/16)
        SVA["Spoke VNet A"]
        SN1A(Application Subnet A)
        SN2A(Database Subnet A)
        SVA --- SN1A
        SVA --- SN2A
    end

    subgraph Spoke VNet B (10.2.0.0/16)
        SVB["Spoke VNet B"]
        SN1B(Application Subnet B)
        SN2B(Web Subnet B)
        SVB --- SN1B
        SVB --- SN2B
    end

    subgraph Azure PaaS Services
        SQL[("Azure SQL DB")]
        SA[["Storage Account (Blob)"]]
        KV(("Azure Key Vault"))
    end

    HVN -- VNet Peering --> SVA
    HVN -- VNet Peering --> SVB

    SVA -- Traffic via Hub Gateway (via UDR) --> FW
    SVB -- Traffic via Hub Gateway (via UDR) --> FW

    FW -- SNAT Outbound --> Internet

    SN1A -- Private Link --> SQL
    SN2A -- Service Endpoint --> SA
    SN1B -- Private Link --> KV

    PED -- Private Endpoint NIC (SQL) --> SQL
    PED -- Private Endpoint NIC (KV) --> KV

    classDef default fill:#D4E8F8,stroke:#3498DB,stroke-width:2px;
    classDef PaaS fill:#F8E8D4,stroke:#E67E22,stroke-width:2px;
    class SQL,SA,KV PaaS;

In this typical architecture: * VNet Peering connects spoke VNets to a central hub VNet. All inter-VNet communication, and access to on-premises resources via the hub’s VPN/ExpressRoute gateway, flows through the peering. * Azure Firewall in the hub VNet acts as a central point for egress filtering and potentially ingress inspection for traffic going to/from the internet or even between spokes (with forced tunneling/UDRs). * Private Link is used for critical PaaS services like Azure SQL DB and Key Vault, ensuring their private IPs are exposed directly into the hub’s Private Endpoint subnet. Spokes can access these via peering and potential firewall rules. * Service Endpoints might be used for less critical or regionally constrained PaaS services like Storage Accounts, enabling secure access directly from specific application subnets.

Conclusion

Azure VNet Peering, Service Endpoints, and Private Link are powerful networking constructs that form the backbone of secure and scalable cloud architectures. Understanding their ‘under-the-hood’ mechanisms, including the non-transitive nature of peering, the route injection of service endpoints, and the critical DNS resolution for Private Link, is essential for designing robust solutions.

By carefully combining these capabilities, architects and engineers can build sophisticated network topologies that meet stringent security, compliance, and performance requirements, enabling seamless private connectivity across their Azure footprint.