TCP Proxy Codelab - Rate limiting and IP Deny list with TCP Proxy Load balancer

1. Introduction

Google Cloud load balancing is deployed at the edge of Google's network in Google points of presence (POP) around the world. User traffic directed to an TCP Proxy load balancer enters the POP closest to the user and is then load balanced over Google's global network to the closest backend that has sufficient capacity available.

Cloud Armor is Google's distributed denial of service and web application firewall (WAF) detection system. Cloud Armor is tightly coupled with the Google Cloud TCP Proxy Load Balancer and allows you to interrogate incoming traffic for unwanted requests. The rate limiting feature of this service allows you to curtail traffic to backend resources based on request volume and prevents unwelcome traffic from consuming resources on your Virtual Private Cloud (VPC) network.

Google Cloud TCP/SSL proxy load balancers allow you to proxy TCP/ SSL type traffic among your backend services.

In this lab, you will create a TCP/SSL load balancer with a backend service and limit access to the load balancer to only a specific set of user clients.

be33dadf836374bb.png

What you'll learn

  • How to create a TCP/SSL proxy load balancer
  • How to create Cloud Armor security policy
  • How to create IP deny list rule for TCP/SSL proxy load balancer in Cloud Armor
  • How to create a rate limiting rule for TCP proxy load balancer in Cloud Armor
  • How to add the security policy to a TCP/SSL load balancing backend service

What you'll need

  • Basic knowledge of Google Compute Engine ( codelab)
  • Basic networking and TCP/IP knowledge
  • Basic Unix/Linux command line knowledge
  • It is helpful to have completed a tour of networking in GCP with Networking in the Google Cloud

2. Requirements

Self-paced environment setup

  1. Sign in to Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.

Note: You can easily access Cloud Console by memorizing its URL, which is console.cloud.google.com.

96a9c957bc475304.png

b9a10ebdf5b5a448.png

a1e3c01a38fa61c2.png

Remember the project ID, a unique name across all Google Cloud projects (the name above has already been taken and will not work for you, sorry!). It will be referred to later in this codelab as PROJECT_ID.

Note: If you're using a Gmail account, you can leave the default location set to No organization. If you're using a Google Workspace account, then choose a location that makes sense for your organization.

  1. Next, you'll need to enable billing in Cloud Console in order to use Google Cloud resources.

Running through this codelab shouldn't cost much, if anything at all. Be sure to to follow any instructions in the "Cleaning up" section which advises you how to shut down resources so you don't incur billing beyond this tutorial. New users of Google Cloud are eligible for the $300 USD Free Trial program.

Start Cloud Shell

While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.

From the GCP Console click the Cloud Shell icon on the top right toolbar:

bce75f34b2c53987.png

It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:

f6ef2b5f13479f3a.png

This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this lab can be done with simply a browser.

Before you begin

Inside Cloud Shell, make sure that your project id is set up

gcloud config list project
gcloud config set project [YOUR-PROJECT-NAME]
PROJECT_ID=[YOUR-PROJECT-NAME]
echo $PROJECT_ID

Enable APIs

Enable all necessary services

gcloud services enable compute.googleapis.com
gcloud services enable logging.googleapis.com        
gcloud services enable monitoring.googleapis.com

3. Create backend services

Create 2 instances as follows - Create instance1-b1 in zone us-central1-b

gcloud compute instances create vm-1-b1 \
    --image-family debian-9 \
    --image-project debian-cloud \
    --tags tcp-lb \
    --zone us-central1-b \
    --metadata startup-script="#! /bin/bash
      sudo apt-get update
      sudo apt-get install apache2 -y
      sudo sed -i '/Listen 80/c\Listen 110' /etc/apache2/ports.conf
      sudo service apache2 restart
      echo '<!doctype html><html><body><h1>This is VM1-b1 in central1-b</h1></body></html>' | tee /var/www/html/index.html
      EOF"

Create instance 1-b2 in zone us-central1-b

gcloud compute instances create vm-1-b2 \
    --image-family debian-9 \
    --image-project debian-cloud \
    --tags tcp-lb \
    --zone us-central1-b \
    --metadata startup-script="#! /bin/bash
      sudo apt-get update
      sudo apt-get install apache2 -y
      sudo sed -i '/Listen 80/c\Listen 110' /etc/apache2/ports.conf
      sudo service apache2 restart
      echo '<!doctype html><html><body><h1>This is VM1-b2 in central1-b</h1></body></html>' | tee /var/www/html/index.html
      EOF"

Create an instance group vm-ig1

gcloud compute instance-groups unmanaged create vm-ig1  --zone us-central1-b

Create a named port for the instance group. For this lab, we will use port 110

    gcloud compute instance-groups set-named-ports vm-ig1 \
--named-ports tcp 110:110 --zone us-central1-b

Add the instances to the instance group

gcloud compute instance-groups unmanaged add-instances vm-ig1 \
   --instances vm-1-b1,vm-1-b2 --zone us-central1-b

4. Configuring the load balancer

Next, we will create a health check.

gcloud compute health-checks create tcp my-tcp-health-check --port 110

Create a backend service

gcloud compute backend-services create my-tcp-lb  --global-health-checks --global \
--protocol TCP --health-checks my-tcp-health-check --timeout 5m --port-name tcp110

Add the instance group to the backend service

gcloud compute backend-services add-backend my-tcp-lb --global --instance-group \ vm-ig1 --instance-group-zone us-central1-b --balancing-mode UTILIZATION \ --max-utilization 0.8

Configure a target TCP proxy

gcloud compute target-tcp-proxies create my-tcp-lb-target-proxy --backend-service \ my-tcp-lb --proxy-header NONE

Reserve global static IPv4 addresses

You will use this IP address to reach your load balanced service.

gcloud compute addresses create tcp-lb-static-ipv4  --ip-version=IPV4   --global

Configure global forwarding rules for the LB IP address.

gcloud compute forwarding-rules create my-tcp-lb-ipv4-forwarding-rule \
    --global --target-tcp-proxy my-tcp-lb-target-proxy --address LB_STATIC_IPV4 \ --ports 110

5. Creating a firewall rule for the TCP proxy load balancer

gcloud compute firewall-rules create allow-tcplb-and-health \
   --source-ranges 130.211.0.0/22,35.191.0.0/16 \
   --target-tags tcp-lb \
   --allow tcp:110

Once your load balancer is created test it with the following command

Curl LB_IP:110

Next create VM's for Validation of access denial to the LB

You should create 2 instances, each with a public IP address and named test-server1 and test-server2

6. Create a Security Policy in Cloud Armor

In this section, you will create a backend security policy and 2 rules in the policy in Cloud Armor.

The first rule will deny a limited set of IPs from accessing the TCP Load Balancer by setting a security policy to deny certain IPs and the second rule will perform rate limiting.

  1. In Cloud Shell(refer to "Start Cloud Shell" under "Setup and Requirements" for instructions on how to use Cloud Shell), create a backend service security policy called rate-limit-and-deny-tcp as follows
gcloud compute security-policies create rate-limit-and-deny-tcp \
    --description "policy for tcp proxy rate limiting and IP deny"

Add Rules to Security Policy

Next add a deny list rule to the Cloud Armor policy "rate-limit-and-deny-tcp".

gcloud compute security-policies rules create 1000 --action deny --security-policy \ rate-limit-and-deny-tcp --description "deny test-server1" --src-ip-ranges \ "enter-test-server-1ip-here"

Add a rate limiting rule to Cloud Armor security policy "rate-limit-and-deny-tcp"

gcloud compute security-policies rules create 3000   \ --security-policy=rate-limit-and-deny-tcp  \       
--expression="true"  --action=rate-based-ban  --rate-limit-threshold-count=5  \          
--rate-limit-threshold-interval-sec=60  --ban-duration-sec=300      \         
--conform-action=allow  --exceed-action=deny-404  --enforce-on-key=IP

Attach policy to TCP Proxy backend service:

Run the following command to ensure that the security policy is attached to the TCP Proxy backend service.

gcloud compute backend-services update my-tcp-lb --security-policy \ rate-limit-and-deny-tcp

Enable logging on TCP Proxy Load Balancer

gcloud beta compute backend-services update my-tcp-lb \ 
--enable-logging --logging-sample-rate=1

7. Validate deny list rule

Validate the deny list rule by logging into the test-server whose IP was specified in the deny list rule and run the following command

Curl LB_IP:110

Immediate requests may give a response from the LB but wait till the curl request is denied or dropped and then take a look at the logs in Cloud Logging to verify log entry for the ip deny rule being triggered.

Go to Cloud Logging and under resources, select the resource type as "tcp_ssl_proxy_rule" and set backend target as "my-tcp-lb".

With the resources defined for filtering., we can validate the ip deny rule is in effect from the PRIORITY value of 1000 in the log entry and the configured action "DENY" action is in effect as both were instructed from the deny rule and the IP being denied as shown below

db9b835e0360dcaf.png

8. Validate rate limiting rule

Validate the rate limit rule is in effect by sending many requests in a short time frame that exceeds the defined threshold (5 requests per minute).

Once this is done, click the view logs in the cloud armor service and this takes you to cloud logging where you can filter the logs by the load balancer to see the cloud armor logs as they come in.

A rate limiting entry should be as per screenshot below. We can validate the rate limit rule is in effect from the PRIORITY value of 3000 in the log entry and from the configured action, the "RATE BASED BAN" action is in effect as instructed from the rate limiting rule.

37c76e5d7532623.png

9. Environment Clean UP

Be sure to clean up the infrastructure created to avoid running costs of unused infrastructure.

Quickest way is to delete the entire Project in GCP to ensure there are no dangling resources left unattended to.However delete the individual resources with the following commands

The TCP Proxy load balancer

gcloud compute target-tcp-proxies delete my-tcp-lb

The instance group

gcloud compute instance-groups unmanaged delete vm-ig1

The 2 test VM instances created

gcloud compute instances delete Instance_name --zone=instance_zone

The backend service

gcloud compute backend-services delete BACKEND_SERVICE_NAME

The Cloud Armor rules within the policy

gcloud compute security-policies rules delete 1000  \ --security-policy=rate-limit-and-deny-tcp && 
gcloud compute security-policies rules delete 3000  \ --security-policy=rate-limit-and-deny-tcp

The cloud armor security policy

gcloud compute security-policies delete rate-limit-and-deny-tcp