Infrastructure by Vitale Mazo

Advanced AWS Cloud WAN with Service Insertion — Network Firewall Integration (Part 2 of 2)

Part 2 of 2: Extending your Cloud WAN architecture with AWS Network Firewall service insertion for east-west and north-south traffic inspection using Terraform modules and policy-based routing.

#Terraform #AWS #Cloud WAN #Network Firewall #Service Insertion #Security #DevOps #Infrastructure as Code

Advanced AWS Cloud WAN with Service Insertion — Network Firewall Integration

This is Part 2 of 2. Start with Part 1 — Supercharging Terraform with AWS Cloud WAN Modules for the foundation.

Extending your Cloud WAN architecture with AWS Network Firewall service insertion for east-west and north-south traffic inspection using Terraform modules and policy-based routing. This builds upon the foundation established in Part 1: Supercharging Terraform with AWS Cloud WAN Modules.

Table of Contents

Overview

Building on the foundation from Part 1, this guide extends your AWS Cloud WAN architecture with service insertion capabilities using AWS Network Firewall. Service insertion allows you to steer same-segment or cross-segment traffic through network functions deployed in VPCs, enabling centralized security inspection and policy enforcement.

This implementation is based on the AWS Cloud WAN service insertion best practices and provides a production-ready approach to global security inspection.

AWS Cloud WAN Service Insertion Figure 1: AWS Cloud WAN Service Insertion Architecture - Shows how traffic is steered through network functions for inspection

Key Benefits

  • Simplified Routing — Automatically steer inter-VPC and VPC-to-Internet traffic through security appliances
  • Multi-Region Inspection — Deploy centralized security across multiple regions without complex routing configurations
  • Policy-Based Control — Use Cloud WAN policies to define traffic steering rules and security boundaries
  • Cost Optimization — Centralize security functions and reduce the need for distributed firewalls

Service Insertion Architecture

Service insertion works by creating network function groups that contain security appliances (like AWS Network Firewall) and then configuring Cloud WAN policies to redirect traffic through these groups.

High-Level Design

flowchart TB
  subgraph Global_Network
    subgraph Core_Network
      direction TB
      SEG_PROD[Segment: prod]
      SEG_DEV[Segment: dev]
      SEG_SHARED[Segment: shared]
      SEG_INSPECTION[Segment: inspection]
    end
  end

  subgraph Network_Function_Group
    direction TB
    NFW1[AWS Network Firewall<br/>us-east-1]
    NFW2[AWS Network Firewall<br/>us-west-2]
  end

  VPCp1[VPC: prod (us-east-1)] --> SEG_PROD
  VPCd1[VPC: dev (us-east-1)] --> SEG_DEV
  VPCs1[VPC: shared (us-east-1)] --> SEG_SHARED
  VPCi1[VPC: inspection (us-east-1)] --> SEG_INSPECTION

  VPCp2[VPC: prod (us-west-2)] --> SEG_PROD
  VPCd2[VPC: dev (us-west-2)] --> SEG_DEV
  VPCs2[VPC: shared (us-west-2)] --> SEG_SHARED
  VPCi2[VPC: inspection (us-west-2)] --> SEG_INSPECTION

  SEG_PROD -.->|Service Insertion| NFW1
  SEG_DEV -.->|Service Insertion| NFW1
  SEG_SHARED -.->|Service Insertion| NFW2

  NFW1 --> NFW2

Prerequisites

  • Part 1 Implementation — Complete the foundation workspace from Part 1
  • AWS Network Firewall — Familiarity with AWS Network Firewall concepts
  • Terraform Cloud — Active workspace with appropriate permissions
  • IAM Permissions — Network Manager, VPC, and Network Firewall permissions

Required AWS Services

  • AWS Cloud WAN (existing)
  • AWS Network Firewall
  • AWS VPC (for inspection VPCs)
  • AWS Route 53 (for DNS resolution)
  • AWS CloudWatch (for monitoring)

Quick Start

  1. Extend foundation workspace — Add service insertion policy to existing core network
  2. Create inspection VPCs — Deploy VPCs with AWS Network Firewall in each region
  3. Configure network function groups — Group inspection VPCs for traffic steering
  4. Deploy service insertion policy — Update Cloud WAN policy with service insertion actions
  5. Test traffic flows — Verify east-west and north-south traffic inspection

Repository Layout

cloud-wan-service-insertion/
├── foundation/
│   ├── 01-global-network.tf
│   ├── 02-core-network.tf
│   ├── 03-policy-v1.tf
│   ├── 04-policy-v2.tf
│   └── 05-service-insertion-policy.tf
├── modules/
│   ├── logical-vpc/
│   │   └── main.tf
│   └── inspection-vpc/
│       ├── main.tf
│       ├── variables.tf
│       └── outputs.tf
├── inspection/
│   ├── us-east-1/
│   │   ├── main.tf
│   │   └── variables.tf
│   └── us-west-2/
│       ├── main.tf
│       └── variables.tf
└── terraform.tfvars.example

Core Components

Network Function Groups

Network function groups are collections of attachments specifically used for network or security functions. Each group can contain one attachment per region.

# foundation/05-service-insertion-policy.tf
data "aws_networkmanager_core_network_policy_document" "service_insertion" {
  version = "2021.12"

  core_network_configuration {
    asn_ranges = ["65000-65535"]
    edge_locations {
      location = "us-east-1"
      asn      = 65000
    }
    edge_locations {
      location = "us-west-2"
      asn      = 65001
    }
  }

  segments {
    name = "prod"
  }

  segments {
    name = "dev"
  }

  segments {
    name = "shared"
  }

  segments {
    name = "inspection"
  }

  # Network function group for AWS Network Firewall
  network_function_groups {
    name = "inspection-nfw"
    attachments {
      name = "inspection-vpc-us-east-1"
      edge_location = "us-east-1"
    }
    attachments {
      name = "inspection-vpc-us-west-2"
      edge_location = "us-west-2"
    }
  }

  # Segment sharing for inspection
  segment_actions {
    action = "share"
    segment = "inspection"
    share_with = ["prod", "dev", "shared"]
  }

  # East-west traffic inspection (VPC to VPC)
  segment_actions {
    action = "send-via"
    segment = "prod"
    mode = "single-hop"
    when_sent_to = {
      segments = ["dev", "shared"]
    }
    via = {
      network_function_groups = ["inspection-nfw"]
    }
  }

  segment_actions {
    action = "send-via"
    segment = "dev"
    mode = "single-hop"
    when_sent_to = {
      segments = ["prod", "shared"]
    }
    via = {
      network_function_groups = ["inspection-nfw"]
    }
  }

  # North-south traffic inspection (VPC to Internet)
  segment_actions {
    action = "send-to"
    segment = "prod"
    via = {
      network_function_groups = ["inspection-nfw"]
    }
  }

  segment_actions {
    action = "send-to"
    segment = "dev"
    via = {
      network_function_groups = ["inspection-nfw"]
    }
  }
}

Inspection VPCs

Inspection VPCs host the AWS Network Firewall and are attached to the inspection segment.

# modules/inspection-vpc/main.tf
resource "aws_vpc" "inspection" {
  cidr_block           = var.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "${var.environment}-inspection-vpc"
    Environment = var.environment
    NetworkFunction = "inspection"
  }
}

resource "aws_subnet" "inspection" {
  count = 3

  vpc_id            = aws_vpc.inspection.id
  cidr_block        = var.subnet_cidrs[count.index]
  availability_zone = data.aws_availability_zones.available.names[count.index]

  tags = {
    Name = "${var.environment}-inspection-subnet-${count.index + 1}"
    Environment = var.environment
    NetworkFunction = "inspection"
  }
}

# AWS Network Firewall
resource "aws_networkfirewall_firewall" "main" {
  name                = "${var.environment}-nfw"
  firewall_policy_arn = aws_networkfirewall_firewall_policy.main.arn
  vpc_id              = aws_vpc.inspection.id

  subnet_mapping {
    subnet_id = aws_subnet.inspection[0].id
  }

  subnet_mapping {
    subnet_id = aws_subnet.inspection[1].id
  }

  tags = {
    Name = "${var.environment}-nfw"
    Environment = var.environment
  }
}

resource "aws_networkfirewall_firewall_policy" "main" {
  name = "${var.environment}-nfw-policy"

  firewall_policy {
    stateless_default_actions          = ["aws:forward_to_sfe"]
    stateless_fragment_default_actions = ["aws:forward_to_sfe"]
    stateless_rule_group_reference {
      priority     = 1
      resource_arn = aws_networkfirewall_rule_group.stateless.arn
    }
  }

  tags = {
    Name = "${var.environment}-nfw-policy"
    Environment = var.environment
  }
}

resource "aws_networkfirewall_rule_group" "stateless" {
  capacity    = 100
  name        = "${var.environment}-stateless-rules"
  type        = "STATELESS"
  description = "Stateless inspection rules"

  rule_group {
    rules_source {
      stateless_rules_and_custom_actions {
        stateless_rule {
          priority = 1
          rule_definition {
            actions = ["aws:forward_to_sfe"]
            match_attributes {
              protocols = [6]
              source {
                address_definition = "0.0.0.0/0"
              }
              destination {
                address_definition = "0.0.0.0/0"
              }
            }
          }
        }
      }
    }
  }

  tags = {
    Name = "${var.environment}-stateless-rules"
    Environment = var.environment
  }
}

Service Insertion Policy

Traffic Actions and Modes

AWS Cloud WAN service insertion supports two primary traffic actions:

East-West Traffic (Send Via)

Traffic flows between VPCs through the inspection VPC.

{
  "segment-actions": [
    {
      "action": "send-via",
      "segment": "prod",
      "mode": "single-hop",
      "when-sent-to": { "segments": ["dev", "shared"] },
      "via": { "network-function-groups": ["inspection-nfw"] }
    }
  ]
}

North-South Traffic (Send To)

Traffic flows to the Internet or on-premises through the inspection VPC.

{
  "segment-actions": [
    {
      "action": "send-to",
      "segment": "prod",
      "via": { "network-function-groups": ["inspection-nfw"] }
    }
  ]
}

Complete Service Insertion Policy

# foundation/05-service-insertion-policy.tf
data "aws_networkmanager_core_network_policy_document" "service_insertion" {
  version = "2021.12"

  core_network_configuration {
    asn_ranges = ["65000-65535"]
    edge_locations {
      location = "us-east-1"
      asn      = 65000
    }
    edge_locations {
      location = "us-west-2"
      asn      = 65001
    }
  }

  # Segments
  segments {
    name = "prod"
  }

  segments {
    name = "dev"
  }

  segments {
    name = "shared"
  }

  segments {
    name = "inspection"
  }

  # Network function group
  network_function_groups {
    name = "inspection-nfw"
    attachments {
      name = "inspection-vpc-us-east-1"
      edge_location = "us-east-1"
    }
    attachments {
      name = "inspection-vpc-us-west-2"
      edge_location = "us-west-2"
    }
  }

  # Segment sharing for inspection
  segment_actions {
    action = "share"
    segment = "inspection"
    share_with = ["prod", "dev", "shared"]
  }

  # East-west traffic inspection (VPC to VPC)
  segment_actions {
    action = "send-via"
    segment = "prod"
    mode = "single-hop"
    when_sent_to = {
      segments = ["dev", "shared"]
    }
    via = {
      network_function_groups = ["inspection-nfw"]
    }
  }

  segment_actions {
    action = "send-via"
    segment = "dev"
    mode = "single-hop"
    when_sent_to = {
      segments = ["prod", "shared"]
    }
    via = {
      network_function_groups = ["inspection-nfw"]
    }
  }

  # North-south traffic inspection (VPC to Internet)
  segment_actions {
    action = "send-to"
    segment = "prod"
    via = {
      network_function_groups = ["inspection-nfw"]
    }
  }

  segment_actions {
    action = "send-to"
    segment = "dev"
    via = {
      network_function_groups = ["inspection-nfw"]
    }
  }
}

resource "aws_networkmanager_core_network_policy_attachment" "service_insertion" {
  core_network_id = aws_networkmanager_core_network.main.id
  policy_document = data.aws_networkmanager_core_network_policy_document.service_insertion.json
  description     = "Cloud WAN service insertion policy with AWS Network Firewall"
}

Terraform Implementation

Inspection VPC Module

# modules/inspection-vpc/variables.tf
variable "environment" {
  description = "Environment name"
  type        = string
}

variable "vpc_cidr" {
  description = "CIDR block for the inspection VPC"
  type        = string
  default     = "10.100.0.0/16"
}

variable "subnet_cidrs" {
  description = "CIDR blocks for inspection subnets"
  type        = list(string)
  default     = ["10.100.1.0/24", "10.100.2.0/24", "10.100.3.0/24"]
}

variable "core_network_id" {
  description = "Cloud WAN Core Network ID"
  type        = string
}

variable "inspection_segment" {
  description = "Cloud WAN inspection segment name"
  type        = string
  default     = "inspection"
}

Regional Inspection Deployment

# inspection/us-east-1/main.tf
module "inspection_vpc" {
  source = "../../modules/inspection-vpc"

  environment      = "prod"
  vpc_cidr        = "10.100.0.0/16"
  subnet_cidrs    = ["10.100.1.0/24", "10.100.2.0/24", "10.100.3.0/24"]
  core_network_id = var.core_network_id
}

# Cloud WAN VPC Attachment
resource "aws_networkmanager_vpc_attachment" "inspection" {
  core_network_id = var.core_network_id
  subnet_arns     = module.inspection_vpc.subnet_arns
  vpc_arn         = module.inspection_vpc.vpc_arn

  tags = {
    Name        = "inspection-vpc-us-east-1"
    Environment = "prod"
    NetworkFunction = "inspection"
    Segment     = "inspection"
  }
}

Multi-Region Considerations

Region Priority and Fallback

# Regional priority configuration
segment_actions {
  action = "send-to"
  segment = "prod"
  via = {
    network_function_groups = ["inspection-nfw"]
  }
  region_priority = ["us-east-1", "us-west-2"]
  fallback_region = "us-east-1"
}

Cross-Region Traffic Flow

  • Single Hop — Traffic traverses one intermediate attachment
  • Dual Hop — Traffic traverses attachments in both source and destination regions
  • Deterministic Selection — Cloud WAN automatically selects the optimal region based on priority

Security and Compliance

Network Firewall Rules

# Stateless rules for basic traffic inspection
resource "aws_networkfirewall_rule_group" "stateless" {
  capacity = 100
  name     = "inspection-stateless-rules"
  type     = "STATELESS"

  rule_group {
    rules_source {
      stateless_rules_and_custom_actions {
        stateless_rule {
          priority = 1
          rule_definition {
            actions = ["aws:forward_to_sfe"]
            match_attributes {
              protocols = [6] # TCP
              source {
                address_definition = "0.0.0.0/0"
              }
              destination {
                address_definition = "0.0.0.0/0"
              }
            }
          }
        }
      }
    }
  }
}

# Stateful rules for advanced inspection
resource "aws_networkfirewall_rule_group" "stateful" {
  capacity = 1000
  name     = "inspection-stateful-rules"
  type     = "STATEFUL"

  rule_group {
    rules_source {
      stateful_rule {
        action = "PASS"
        header {
          protocol = "TCP"
          source   = "ANY"
          destination = "ANY"
          source_port = "ANY"
          destination_port = "ANY"
          direction = "FORWARD"
        }
        rule_options {
          keyword = "sid"
          settings = ["1"]
        }
      }
    }
  }
}

Compliance Considerations

  • Data Residency — Ensure inspection VPCs are in compliant regions
  • Logging — Enable CloudWatch logging for all Network Firewall rules
  • Encryption — Use VPC endpoints for secure communication
  • Access Control — Implement least-privilege IAM policies

Monitoring and Observability

CloudWatch Metrics

# CloudWatch log group for Network Firewall
resource "aws_cloudwatch_log_group" "network_firewall" {
  name              = "/aws/networkfirewall/${var.environment}"
  retention_in_days = 30
}

# CloudWatch log group for Cloud WAN
resource "aws_cloudwatch_log_group" "cloud_wan" {
  name              = "/aws/networkmanager/${var.environment}"
  retention_in_days = 30
}

Monitoring Dashboard

# CloudWatch dashboard for service insertion monitoring
resource "aws_cloudwatch_dashboard" "service_insertion" {
  dashboard_name = "cloud-wan-service-insertion"

  dashboard_body = jsonencode({
    widgets = [
      {
        type   = "metric"
        x      = 0
        y      = 0
        width  = 12
        height = 6
        properties = {
          metrics = [
            ["AWS/NetworkFirewall", "PacketsDropped", "FirewallName", "inspection-nfw"],
            ["AWS/NetworkFirewall", "PacketsForwarded", "FirewallName", "inspection-nfw"]
          ]
          period = 300
          stat   = "Sum"
          region = "us-east-1"
          title  = "Network Firewall Traffic"
        }
      }
    ]
  })
}

Troubleshooting

Common Issues

Traffic not being inspected:

  • Verify network function group attachments are properly tagged
  • Check that segment actions are correctly configured
  • Ensure inspection VPC is in the correct segment

Cross-region traffic not flowing:

  • Verify both regions have inspection VPCs attached
  • Check region priority configuration
  • Ensure fallback region is properly set

Network Firewall not receiving traffic:

  • Verify subnet mappings are correct
  • Check route table configurations
  • Ensure appliance mode is enabled

Debugging Commands

# Check Cloud WAN policy status
aws networkmanager get-core-network-policy \
  --core-network-id <core-network-id> \
  --policy-version-id <version-id>

# Verify VPC attachments
aws networkmanager get-vpc-attachment \
  --attachment-id <attachment-id>

# Check Network Firewall status
aws networkfirewall describe-firewall \
  --firewall-arn <firewall-arn>

Best Practices

Design Principles

  1. Start Simple — Begin with basic stateless rules and gradually add complexity
  2. Regional Distribution — Deploy inspection VPCs in all active regions
  3. Monitoring First — Implement comprehensive logging before production traffic
  4. Gradual Rollout — Test with non-production segments first

Operational Guidelines

  • Policy Versioning — Always test policy changes in a separate version
  • Rollback Planning — Keep previous policy versions for quick rollback
  • Capacity Planning — Monitor Network Firewall capacity and scale accordingly
  • Security Updates — Regularly update firewall rules and rule groups

Cost Optimization

  • Right-Sizing — Choose appropriate Network Firewall capacity
  • Log Retention — Set appropriate CloudWatch log retention periods
  • Regional Strategy — Use single-hop mode when possible to reduce costs

FAQ

Can I use third-party firewalls with service insertion? Yes, service insertion supports third-party network appliances deployed in VPCs, not just AWS Network Firewall.

How does traffic routing work with multiple regions? Cloud WAN uses deterministic region selection based on your priority configuration and fallback settings.

What’s the difference between send-via and send-to modes? Send-via is for east-west traffic between VPCs, while send-to is for north-south traffic to the Internet or on-premises.

Can I have multiple network function groups? Yes, you can create multiple network function groups for different security functions or compliance requirements.

How do I handle high availability? Deploy inspection VPCs in multiple AZs within each region and use Cloud WAN’s automatic failover capabilities.

Credits

This guide builds upon the foundation established in Part 1: Supercharging Terraform with AWS Cloud WAN Modules and leverages AWS Cloud WAN service insertion capabilities as documented in the AWS Network Manager documentation.

Key References:

This is Part 2 of a comprehensive guide to AWS Cloud WAN with Terraform. For the foundational setup, see Part 1: Supercharging Terraform with AWS Cloud WAN Modules.

About the Author

Vitale Mazo is a Senior Cloud Engineer with 19+ years of experience in enterprise IT, specializing in cloud native technologies and multi-cloud infrastructure design.