Cloud Armor for NLB/VM with User Defined Rules

1. Introduction

Cloud Armor security policies are used to configure user-defined rules to filter traffic at the edge of Google's network, upstream from your infrastructure. Network Edge security policies can be used to help protect and allow or block traffic targeting the following endpoint types: Network Load Balancer, Protocol Forwarding and VMs with Public IPs.

In this code lab, we will demo how to configure Cloud Armor security policies with user-defined rules to prevent DDoS attacks.

Figure 1. Cloud Armor for VM with public IP protection.

What you'll learn

• Cloud Armor security policies with user-defined rules configuration
• UDP Offset configurations and testing.

What you'll need

• Knowledge of TCP/IP
• Knowledge of Unix/Linux command line

2. Before you begin

Codelab offers configuration and validation steps based on illustrated topology. IAM permissions are not in scope of the codelab. Projects must be enrolled in Managed Protection Plus to create a Network Edge Security Policy with used-configured rules.

Inside Cloud Shell, make sure that your project id is set up

gcloud config list project
gcloud config set project [YOUR-PROJECT-NAME]
prodproject=YOUR-PROJECT-NAME
echo $prodproject

3. Create Target VPC Network

In the following section, we will set up VPC networks and related networking configurations. Cloud Armor network edge security policy is regional based, we set up all related resources in asia-southeast1 region.

VPC Network

From Cloud Shell

gcloud compute networks create ca4nlb --project=$prodproject --subnet-mode=custom

Create Subnet

From Cloud Shell

gcloud compute networks subnets create ca4nlb-asia-southeast1 --project=$prodproject --range=10.0.0.0/24 --network=ca4nlb --region=asia-southeast1

Create Firewall rules.

In this section, we will add a firewall rule to allow the expected UDP traffic to port 10000.

From Cloud Shell create a firewall rule to open UDP port 10000 for following testing.

gcloud compute firewall-rules create ca4nlb-udp10000 --allow udp:10000 --network ca4nlb --source-ranges 0.0.0.0/0 --enable-logging

From Cloud Shell create a firewall rule to allow IAP to connect to your VM instances.

gcloud compute firewall-rules create ca4nlb-iap-prod --network ca4nlb --allow tcp:22 --source-ranges=35.235.240.0/20 --enable-logging

Firewall policies have a different purpose than Cloud Armor policies. Firewall policies are effective in configuring network ACLs and enforcing network segmentation policies within your VPC.

The goal of Cloud Armor network edge security policies is to block volumetric DDoS attacks without consuming any of your resources. The network edge security policies are evaluated and enforced at the edge of Google's network, prior to the Firewall policies, and can effectively mitigate high volume unwelcome traffic before it has a chance to impact the targeted workload.

4. Create Target VM Instances

Create a target VM for testing security policies. This VM should have a public IP address and open UDP port 10000.

From Cloud Shell create instance targetvm

gcloud compute instances create targetvm \
--zone=asia-southeast1-b \
--image-family=debian-11 \
--image-project=debian-cloud \
--network-interface=network-tier=PREMIUM,nic-type=GVNIC,stack-type=IPV4_ONLY,subnet=ca4nlb-asia-southeast1 \
--shielded-secure-boot \
--shielded-vtpm \
--shielded-integrity-monitoring

5. Configure advanced network DDoS protection

To be able to create a Network Edge Security Policy with used-configured rules, you must first activate advanced network DDoS protection for the desired region. Cloud Armor advanced network DDoS protection provides customers with always-on attack detection and mitigation, for workloads behind External Network Load Balancer, Protocol Forwarding, or VMs with Public IP addresses. Once activated, all regional endpoints in the activated region receive always-on advanced network DDoS protection.

From Cloud Shell

 gcloud compute security-policies create ca_advanced_ddos \
     --type CLOUD_ARMOR_NETWORK \
     --region asia-southeast1

 gcloud compute security-policies update ca_advanced_ddos \
     --network-ddos-protection ADVANCED \
     --region asia-southeast1

 gcloud compute network-edge-security-services create caedgepolicy \
     --security-policy ca_advanced_ddos \
     --region asia-southeast1

6. Create Network Edge Security Policy with default rules

You can't mix advanced network DDoS protection security policy and user defined rules security policy.

Create Network Edge Security Policy

From Cloud Shell

gcloud alpha compute security-policies create customnetworkedge --type=CLOUD_ARMOR_NETWORK --region=asia-southeast1

Modify the default rule in the policy from an allow rule to a deny rule. The default rule governs traffic that does not match any of the preceding rules. It is the last rule of policy. Changing the rule from allow to deny blocks all traffic that does not originate in the range on the allowlist.

Modify default rule

From Cloud Shell

gcloud alpha compute security-policies rules update 2147483647 --security-policy=customnetworkedge --action=deny --region=asia-southeast1

7. Create Network Edge Security Policy with used-configured rules

User predefined UDP offset and configured in cloud armor policy. Packet with this "offset values" will pass the policy check and send it to the backend VM. In the following example, we will define two "offset" with different values.

First value is just after UDP header, exactly match 2 bytes 0x1700

Second value is offset 8 bytes of UDP header, exactly match 4 bytes 0x12345678

Above predefined value will translate into a UDP packet bit view.

Key word offset defines the match value start point, including the UDP 8 bytes header. Any user predefined values will start from offset 8. Cloud Armor support max matched location is 8. Each location max matched continuous sequence length of packet is 4 bytes.

From Cloud Shell

gcloud alpha compute security-policies add-user-defined-field customnetworkedge \
--user-defined-field-name=SIG1_AT_0 \
--base=udp --offset=8 --size=2 --mask=0xFF00 \
--region=asia-southeast1

gcloud alpha compute security-policies add-user-defined-field customnetworkedge \
--user-defined-field-name=SIG2_AT_8 \
--base=udp --offset=16 --size=4 --mask=0xFFFFFFFF \
--region=asia-southeast1

gcloud alpha compute security-policies rules create 1000 \
--security-policy=customnetworkedge \
--network-user-defined-fields="SIG1_AT_0;0x1700,SIG2_AT_8;0x12345678" \
--action=allow --region=asia-southeast1

8. Attach Security Policy to Target VM

Unlike Cloud Armor Advance network DDoS protection covers all resources protection in enabled regions, Cloud Armor Network Edge Security Policy needs to attach security policy to protected resources.

From Cloud Shell attach security policy to protected VM.

gcloud alpha compute instances network-interfaces update targetvm \
--security-policy=customnetworkedge \
--security-policy-region=asia-southeast1 \
--network-interface=nic0 \
--zone=asia-southeast1-b

From Cloud Shell describe target VM, will see securityPolicy is attached. Record the public IP for following testing.

gcloud alpha compute instances describe targetvm --zone=asia-southeast1-b

networkInterfaces:
- accessConfigs:
  - kind: compute#accessConfig
    name: External NAT
    natIP: 35.240.148.100
    networkTier: PREMIUM
    securityPolicy: https://www.googleapis.com/compute/alpha/projects/<project>/regions/asia-southeast1/securityPolicies/customnetworkedge

After security policy applies to the target VM, any other unmatched traffic from the Internet will be dropped. For example, you can't run sudo apt update because we didn't allow TCP protocol.

From Cloud Shell detach security policy from protected VM.

gcloud alpha compute instances network-interfaces update targetvm \
--network-interface=nic0 \
--zone=asia-southeast1-b \
--security-policy= 

9. Prepare Test Resources.

Create Test VPC Network

From Cloud Shell

gcloud compute networks create test --project=$prodproject --subnet-mode=custom

Create Test Subnet

From Cloud Shell

gcloud compute networks subnets create test-asia-southeast1 --project=$prodproject --range=10.0.1.0/24 --network=test --region=asia-southeast1

Create Firewall

From Cloud Shell create a firewall rule to allow IAP to connect to your VM instances.

gcloud compute firewall-rules create test-iap-prod --network test --allow tcp:22 --source-ranges=35.235.240.0/20 --enable-logging

Create Test VM

From Cloud Shell

gcloud compute instances create test01 \
    --zone=asia-southeast1-b \
    --image-family=debian-11 \
    --image-project=debian-cloud \
    --network-interface=network-tier=PREMIUM,nic-type=GVNIC,stack-type=IPV4_ONLY,subnet=test-asia-southeast1 \
    --shielded-secure-boot \
    --shielded-vtpm \
    --shielded-integrity-monitoring

10. Verification

Login Test VM Console and install packet generator packit.

sudo apt install packit

Following UDP offset design, use packit to generate UDP packets. We simulate a (-t udp) packet from interface (-s ens4) source IP address (-s 10.0.1.2) with source ports (-S 10000) to targetVM destination IP address (-d 35.240.148.100) with destination ports (-D 10000). Packet content matches values (-p ‘0x 17 00 00 00 00 00 00 00 12 34 56 78'). We will send (-c 4) packets.

sudo packit -m inject -t UDP -i ens4 -s 10.0.1.2 -d 35.240.148.100 -S 10000 -D 10000 -p '0x 17 00 00 00 00 00 00 00 12 34 56 78' -c 4

In the Target VM, run tcpdump to capture the UDP packet.

sudo tcpdump port 10000 -v -n 

tcpdump: listening on ens4, link-type EN10MB (Ethernet), snapshot length 262144 bytes
06:36:18.434106 IP (tos 0x0, ttl 128, id 17173, offset 0, flags [none], proto UDP (17), length 40)
    35.197.157.140.10000 > 10.0.0.2.10000: UDP, length 12
06:36:19.433656 IP (tos 0x0, ttl 128, id 55641, offset 0, flags [none], proto UDP (17), length 40)
    35.197.157.140.10000 > 10.0.0.2.10000: UDP, length 12
06:36:20.433935 IP (tos 0x0, ttl 128, id 27161, offset 0, flags [none], proto UDP (17), length 40)
    35.197.157.140.10000 > 10.0.0.2.10000: UDP, length 12
06:36:21.434150 IP (tos 0x0, ttl 128, id 46782, offset 0, flags [none], proto UDP (17), length 40)
    35.197.157.140.10000 > 10.0.0.2.10000: UDP, length 12

If we change the traffic patterns in the test VM, we can't capture any packets in the target VM.

sudo packit -m inject -t UDP -i ens4 -s 10.148.0.6 -d 34.87.79.31 -S 10000 -D 10000 -p '0x 33 33 00 00 00 00 00 00 12 34 56 78' -c 4

Cloud Armor Security Policy is deployed on the Internet edge. If you set up a VM in ca4nlb VPC and test with target VM internal IP, this policy will not take effect.

11. Telemetry

To be able to view the telemetry, the Network Security API (networksecurity.googleapis.com) must be enabled.

Open Cloud Metric, use below MQL to query NetworkSercurityPolicy telemetry data.

fetch networksecurity.googleapis.com/RegionalNetworkSecurityPolicy
| metric 'networksecurity.googleapis.com/l3/external/packet_count'
| filter (resource.policy_name == 'customnetworkedge')
| align rate(1m)
| every 1m
| group_by [metric.blocked], [value_packet_count_mean: mean(value.packet_count)]
| group_by 1m, [value_packet_count_mean_mean: mean(value_packet_count_mean)]
| every 1m

Generate high volume traffic with match offset command.

sudo packit -m inject -t UDP -i ens4 -s 10.148.0.6 -d 34.87.79.31 -S 10000 -D 10000 -p '0x 17 00 00 00 00 00 00 00 12 34 56 78' -c 1000000 -w 0.001

[result]
Injected: 1000000  Packets/Sec: 10309.27  Bytes/Sec: 412371.13  Errors: 0

Generate high volume traffic with an unmatch offset command.

sudo packit -m inject -t UDP -i ens4 -s 10.148.0.6 -d 34.87.79.31 -S 10000 -D 10000 -p '0x 11 00 00 00 00 00 00 00 12 34 56 78' -c 1000000 -w 0.001

[result]
Injected: 1000000  Packets/Sec: 10309.27  Bytes/Sec: 412371.13  Errors: 0

Telemetry is filtered by policy_name and grouped by blocked. Blue line indicated traffic allowed by policy rules. Green line indicated traffic blocked by policy rules.

12. Cleanup steps

From a single cloud shell in the terminal delete lab components

gcloud compute instances delete targetvm --zone=asia-southeast1-b

gcloud compute firewall-rules delete ca4nlb-udp10000

gcloud compute firewall-rules delete ca4nlb-iap-prod

gcloud compute networks subnets delete ca4nlb-asia-southeast1 --region=asia-southeast1

gcloud compute networks delete ca4nlb

gcloud alpha compute security-policies delete customnetworkedge --region=asia-southeast1

gcloud alpha compute network-edge-security-services delete caedgepolicy --region=asia-southeast1

gcloud alpha compute security-policies delete ca_advanced_ddos --region=asia-southeast1

gcloud compute instances delete test01 --zone=asia-southeast1-b

gcloud compute firewall-rules delete test-iap-prod

gcloud compute networks subnets delete test-asia-southeast1 --region=asia-southeast1

gcloud compute networks delete test

13. Congratulations!

Congratulations for completing the codelab.

What we've covered

• Cloud Armor security policies with customer defined rules