Using IPv6 Static Routes next hop instance (untagged and tagged) and next hop gateway

1. Introduction

Static custom routes influence the default routing behavior in a VPC. IPv6 custom routes now support new next-hop attributes: next-hop-gateway and next-hop-instance. This codelab describes how to use IPv6 custom routes with these new next-hop options using two VPCs connected by a multi-NIC VM instance. You will also demonstrate mixing ULA and GUA addressing and providing reachability to the ULA VPC to the public internet using the new custom route capability.

What you'll learn

  • How to create an IPv6 custom route with a next-hop-instance next-hop.
  • How to create an IPv6 custom route with a next-hop-gateway next-hop.

What you'll need

  • Google Cloud Project

2. Before you begin

Update the project to support the codelab

This Codelab makes use of $variables to aid gcloud configuration implementation in Cloud Shell.

Inside Cloud Shell, perform the following

gcloud config list project
gcloud config set project [YOUR-PROJECT-NAME]
export projectname=$(gcloud config list --format="value(core.project)")

Overall Lab Architecture

66f72fc758d762f7.png

To demonstrate both types of custom route next-hops, you will create 3 VPCs: A client VPC that uses GUA addressing, a server VPC that uses ULA addressing and a second server VPC that uses GUA addressing.

For the client VPC to access the ULA server, you will utilize a custom route with next-hop-instance pointing at a multi-NIC gateway instance. To provide access to the GUA server (after deleting the default ::/0 route), you will utilize a custom route with next-hop-gateway pointing at the Default Internet Gateway to provide routing over the internet.

3. Client VPC Setup

Create the Client VPC

Inside Cloud Shell, perform the following:

gcloud compute networks create client-vpc \
    --project=$projectname \
    --subnet-mode=custom \
    --mtu=1500 --bgp-routing-mode=regional

Create the Client subnet

Inside Cloud Shell, perform the following:

gcloud compute networks subnets create client-subnet  \
    --network=client-vpc \
    --project=$projectname \
    --range=192.168.1.0/24 \
    --stack-type=IPV4_IPV6 \
    --ipv6-access-type=external \
    --region=us-central1

Record the assigned GUA subnet in an environment variable using this command

export client_subnet=$(gcloud compute networks subnets \
    describe client-subnet \
    --project $projectname \
    --format="value(externalIpv6Prefix)" \
    --region us-central1)

Launch client instance

Inside Cloud Shell, perform the following:

gcloud compute instances create client-instance \
    --subnet client-subnet \
    --stack-type IPV4_IPV6 \
    --zone us-central1-a \
    --project=$projectname

Add firewall rule to for client VPC traffic

Inside Cloud Shell, perform the following:

gcloud compute firewall-rules create allow-gateway-client \
    --direction=INGRESS --priority=1000 \
    --network=client-vpc --action=ALLOW \
    --rules=tcp --source-ranges=$client_subnet \
    --project=$projectname 

Add firewall rule to allow IAP for the client instance

Inside Cloud Shell, perform the following:

gcloud compute firewall-rules create allow-iap-client \
    --direction=INGRESS --priority=1000 \
    --network=client-vpc --action=ALLOW \
    --rules=tcp:22 --source-ranges=35.235.240.0/20 \
    --project=$projectname 

Confirm SSH access into the client instance

Inside Cloud Shell, log into the client-instance:

gcloud compute ssh client-instance \
    --project=$projectname \
    --zone=us-central1-a \
    --tunnel-through-iap

If successful, you'll see a terminal window from the client instance. Exit from the SSH session to continue on with the codelab.

4. ULA Server VPC Setup

Create the ULA server VPC

Inside Cloud Shell, perform the following:

gcloud compute networks create server-vpc1 \
    --project=$projectname \
    --subnet-mode=custom --mtu=1500 \
    --bgp-routing-mode=regional \
    --enable-ula-internal-ipv6

Create the ULA server subnets

Inside Cloud Shell, perform the following:

gcloud compute networks subnets create server-subnet1 \
    --network=server-vpc1 \
    --project=$projectname \
    --range=192.168.0.0/24 \
    --stack-type=IPV4_IPV6 \
    --ipv6-access-type=internal \
    --region=us-central1

Record the assigned ULA subnet in an environment variable using this command

export server_subnet1=$(gcloud compute networks subnets \
    describe server-subnet1 \
    --project $projectname \
    --format="value(internalIpv6Prefix)" \
    --region us-central1)

Launch server VM with a ULA internal IPV6 address

Inside Cloud Shell, perform the following:

gcloud compute instances create server-instance1 \
    --subnet server-subnet1 \
    --stack-type IPV4_IPV6 \
    --zone us-central1-a \
    --project=$projectname

Add firewall rule to allow access to the server from client

Inside Cloud Shell, perform the following:

gcloud compute firewall-rules create allow-client-server1 \
    --direction=INGRESS --priority=1000 \
    --network=server-vpc1 --action=ALLOW \
    --rules=tcp --source-ranges=$client_subnet \
    --project=$projectname 

Add firewall rule to allow IAP

Inside Cloud Shell, perform the following:

gcloud compute firewall-rules create allow-iap-server1 \
    --direction=INGRESS --priority=1000 \
    --network=server-vpc1 --action=ALLOW \
    --rules=tcp:22 \
    --source-ranges=35.235.240.0/20 \
    --project=$projectname 

Install Apache in ULA server instance

Inside Cloud Shell, log into the client-instance:

gcloud compute ssh server-instance1 \
    --project=$projectname \
    --zone=us-central1-a \
    --tunnel-through-iap

Inside the Server VM shell, run the following command

sudo apt update && sudo apt -y install apache2

Verify that Apache is running

sudo systemctl status apache2

Overwrite the default web page

echo '<!doctype html><html><body><h1>Hello World! From Server1!</h1></body></html>' | sudo tee /var/www/html/index.html

Exit from the SSH session to continue on with the codelab.

5. GUA Server VPC Setup

Create the GUA server VPC

Inside Cloud Shell, perform the following:

gcloud compute networks create server-vpc2 \
    --project=$projectname \
    --subnet-mode=custom --mtu=1500 \
    --bgp-routing-mode=regional

Create the GUA server subnets

Inside Cloud Shell, perform the following:

gcloud compute networks subnets create server-subnet2 \
    --network=server-vpc2 \
    --project=$projectname \
    --range=192.168.0.0/24 \
    --stack-type=IPV4_IPV6 \
    --ipv6-access-type=external \
    --region=us-central1

Record the assigned GUA subnet in an environment variable using this command

export server_subnet2=$(gcloud compute networks subnets \
    describe server-subnet2 \
    --project $projectname \
    --format="value(externalIpv6Prefix)" \
    --region us-central1)

Launch server VM with a GUA IPV6 address

Inside Cloud Shell, perform the following:

gcloud compute instances create server-instance2 \
    --subnet server-subnet2 \
    --stack-type IPV4_IPV6 \
    --zone us-central1-a \
    --project=$projectname

Add firewall rule to allow access within the subnet

Inside Cloud Shell, perform the following:

gcloud compute firewall-rules create allow-client-server2 \
    --direction=INGRESS \
    --priority=1000 \
    --network=server-vpc2 \
    --action=ALLOW \
    --rules=tcp --source-ranges=$client_subnet \
    --project=$projectname 

Add firewall rule to allow IAP

Inside Cloud Shell, perform the following:

gcloud compute firewall-rules create allow-iap-server2 \
    --direction=INGRESS \
    --priority=1000 \
    --network=server-vpc2 \
    --action=ALLOW \
    --rules=tcp:22 \
    --source-ranges=35.235.240.0/20 \
    --project=$projectname 

Confirm SSH access into the GUA server instance and install Apache

Inside Cloud Shell, log into the client-instance:

gcloud compute ssh server-instance2 \
    --project=$projectname \
    --zone=us-central1-a \
    --tunnel-through-iap

Inside the Server VM shell, run the following command

sudo apt update && sudo apt -y install apache2

Verify that Apache is running

sudo systemctl status apache2

Overwrite the default web page

echo '<!doctype html><html><body><h1>Hello World! From Server2!</h1></body></html>' | sudo tee /var/www/html/index.html

Exit from the SSH session to continue on with the codelab.

6. Create Gateway Instance

Delete the Client VPC's default route

In preparation for redirecting ULA v6 traffic to the multi-NIC instance and to disable internet egress routing. Delete the default ::/0 route pointing at the default internet gateway.

Inside Cloud Shell, perform the following:

export client_defroutename=$(gcloud compute routes list \
--project $projectname \
--format='value(name)' \
--filter="network:client-vpc AND destRange~'::/0'")

gcloud compute routes delete $client_defroutename \
--project $projectname \
--quiet

Launch gateway multi-NIC VM

Inside Cloud Shell, perform the following:Inside Cloud Shell, perform the following:

gcloud compute instances create gateway-instance \
    --project=$projectname \
    --zone=us-central1-a \
--network-interface=stack-type=IPV4_IPV6,subnet=client-subnet,no-address \
--network-interface=stack-type=IPV4_IPV6,subnet=server-subnet1,no-address \
    --can-ip-forward

Configure gateway instance

Inside Cloud Shell, log into the gateway instance (it might take a few minutes to SSH successfully while the instance is booting up):

gcloud compute ssh gateway-instance \
    --project=$projectname \
    --zone=us-central1-a \
    --tunnel-through-iap

Inside the gateway VM shell, run the following command to enable IPv6 forwarding and keep accepting RAs with forwarding enabled (accept_ra = 2)

sudo sysctl -w net.ipv6.conf.ens4.accept_ra=2
sudo sysctl -w net.ipv6.conf.ens5.accept_ra=2
sudo sysctl -w net.ipv6.conf.ens4.accept_ra_defrtr=1
sudo sysctl -w net.ipv6.conf.all.forwarding=1

Verify the IPv6 routing table on the instance

ip -6 route show

Sample output showing both ULA and GUA subnet routes, with the default route pointing at the GUA interface.

::1 dev lo proto kernel metric 256 pref medium
2600:1900:4000:7a7f:0:1:: dev ens4 proto kernel metric 256 expires 83903sec pref medium
2600:1900:4000:7a7f::/65 via fe80::4001:c0ff:fea8:101 dev ens4 proto ra metric 1024 expires 88sec pref medium
fd20:3df:8d5c::1:0:0 dev ens5 proto kernel metric 256 expires 83904sec pref medium
fd20:3df:8d5c::/64 via fe80::4001:c0ff:fea8:1 dev ens5 proto ra metric 1024 expires 84sec pref medium
fe80::/64 dev ens5 proto kernel metric 256 pref medium
fe80::/64 dev ens4 proto kernel metric 256 pref medium
default via fe80::4001:c0ff:fea8:101 dev ens4 proto ra metric 1024 expires 88sec pref medium

Exit from the SSH session to continue on with the codelab.

7. Create and test routes to gateway instance

Make note of server addresses

Inside Cloud Shell, perform the following:

gcloud compute instances list \
   --project $projectname \
   --filter="name~server-instance" \
--format='value[separator=","](name,networkInterfaces[0].ipv6Address,networkInterfaces[0].ipv6AccessConfigs[0].externalIpv6)'

This should output both server instance names and their IPv6 prefixes. Sample output

server-instance1,fd20:3df:8d5c:0:0:0:0:0,
server-instance2,,2600:1900:4000:71fd:0:0:0:0

Make note of both addresses as you will use them later in curl commands from the client instance. Unfortunately environment variables cannot be easily used to store these as they don't transfer over SSH sessions.

Run curl command from client to ULA server instance

To see the behavior before adding any new routes. Run a curl command from the client instance towards the server-instance1.

Inside Cloud Shell, log into the client-instance:

gcloud compute ssh client-instance \
    --project=$projectname \
    --zone=us-central1-a \
    --tunnel-through-iap

Inside the client instance, perform a curl using the ULA IPV6 address of the server1 instance (the command sets a short timeout of 5s to avoid curl waiting for too long)

curl -m 5.0 -g -6 'http://[ULA-ipv6-address-of-server1]:80/'

This curl command should timeout because the Client VPC doesn't have a route towards the Server VPC yet.

Let's try to fix that! Exit from the SSH session for now.

Add custom route in client VPC

Since the client VPC Is missing a route towards the ULA prefix. Let's add it now.

Inside Cloud Shell, perform the following:

gcloud compute routes create client-to-server1-route \
   --project=$projectname \
   --destination-range=$server_subnet1 \
   --network=client-vpc \
   --next-hop-instance=gateway-instance \
   --next-hop-instance-zone=us-central1-a

SSH back to the client instance:

gcloud compute ssh client-instance \
    --project=$projectname \
    --zone=us-central1-a \
    --tunnel-through-iap

Inside the client instance, attempt the curl to the server instance again. (the command sets a short timeout of 5s to avoid curl waiting for too long)

curl -m 5.0 -g -6 'http://[ULA-ipv6-address-of-server1]:80/'

This curl command still times out because the server1 VPC, doesn't have a route back towards the client VPC through the gateway instance yet.

Exit from the SSH session to continue on with the codelab.

Add custom route in ULA Server VPC

Inside Cloud Shell, perform the following:

gcloud compute routes create server1-to-client-route \
   --project=$projectname \
   --destination-range=$client_subnet \
   --network=server-vpc1 \
   --next-hop-instance=gateway-instance \
   --next-hop-instance-zone=us-central1-a

SSH back to the client instance:

gcloud compute ssh client-instance \
    --project=$projectname \
    --zone=us-central1-a \
    --tunnel-through-iap

Inside the client instance, attempt the curl to the server instance one more time.

curl -m 5.0 -g -6 'http://[ULA-ipv6-address-of-server1]:80/'

This curl command now succeeds showing you have end-to-end reachability from the client instance towards the ULA server instance. This connectivity is only possible now through the use of IPv6 custom routes with next-hop-instance as next-hops.

Sample Output

<user id>@client-instance:~$ curl -m 5.0 -g -6 'http://[fd20:3df:8d5c:0:0:0:0:0]:80/'
<!doctype html><html><body><h1>Hello World! From Server1!</h1></body></html>

Exit from the SSH session to continue on with the codelab.

8. Create and test route to internet gateway

While you have this lab setup, let's also test the functionality of the new next-hop property: next-hop-gateway.

Run curl command from client to GUA server instance

To see the behavior before adding any new routes. Run a curl command from the client instance towards server2's IP address.

Inside Cloud Shell, log into the client instance:

gcloud compute ssh client-instance \
    --project=$projectname \
    --zone=us-central1-a \
    --tunnel-through-iap

Inside the client instance, perform a curl towards the IPv6 endpoint

curl -m 5.0 -g -6 'http://[GUA-ipv6-address-of-server2]:80/'

This curl command should timeout because the client VPC only has its own subnet route and a route to server1's VPC. To be able to reach server2 VPC's GUA range, you need to use the default internet gateway through a custom route.

Exit from the SSH session to continue on with the codelab.

Add custom gateway route in client VPC

Inside Cloud Shell, perform the following:

gcloud compute routes create client-to-server2-route \
   --project=$projectname \
   --destination-range=$server_subnet2 \
   --network=client-vpc \
   --next-hop-gateway=default-internet-gateway

SSH back to the client instance:

gcloud compute ssh client-instance \
    --project=$projectname \
    --zone=us-central1-a \
    --tunnel-through-iap

Inside the client instance, repeat the same curl

curl -m 5.0 -g -6 'http://[GUA-ipv6-address-of-server2]:80/'

This curl command should now succeed in returning the custom hello message, indicating that you could successfully reach the other server's IPv6 address through the default internet gateway successfully.

Sample output:

<user id>@client-instance:~$ curl -m 5.0 -g -6 'http://[2600:1900:4000:71fd:0:0:0:0]:80/'
<!doctype html><html><body><h1>Hello World! From Server2!</h1></body></html>

Exit from the SSH session to go through the clean up section of the lab.

9. Clean up

Clean up instances

Inside Cloud Shell, perform the following:

gcloud compute instances delete client-instance --zone us-central1-a --quiet --project=$projectname

gcloud compute instances delete server-instance1 --zone us-central1-a --quiet --project=$projectname

gcloud compute instances delete server-instance2 --zone us-central1-a --quiet --project=$projectname

gcloud compute instances delete gateway-instance --zone us-central1-a --quiet --project=$projectname

Clean up subnets

Inside Cloud Shell, perform the following:

gcloud compute networks subnets delete client-subnet --region=us-central1 --quiet --project=$projectname

gcloud compute networks subnets delete server-subnet1 --region=us-central1 --quiet --project=$projectname

gcloud compute networks subnets delete server-subnet2 --region=us-central1 --quiet --project=$projectname

Clean up firewall rules

Inside Cloud Shell, perform the following:

gcloud compute firewall-rules delete allow-iap-client  --quiet --project=$projectname
gcloud compute firewall-rules delete allow-iap-server1  --quiet --project=$projectname
gcloud compute firewall-rules delete allow-iap-server2  --quiet --project=$projectname
gcloud compute firewall-rules delete allow-gateway-client  --quiet --project=$projectname
gcloud compute firewall-rules delete allow-client-server1  --quiet --project=$projectname
gcloud compute firewall-rules delete allow-client-server2  --quiet --project=$projectname

Clean up custom routes

Inside Cloud Shell, perform the following:

gcloud compute routes delete client-to-server1-route  --quiet --project=$projectname
gcloud compute routes delete client-to-server2-route  --quiet --project=$projectname
gcloud compute routes delete server1-to-client-route  --quiet --project=$projectname

Clean up VPCs

Inside Cloud Shell, perform the following:

gcloud compute networks delete client-vpc --quiet --project=$projectname
gcloud compute networks delete server-vpc1 --quiet --project=$projectname
gcloud compute networks delete server-vpc2 --quiet --project=$projectname

10. Congratulations

You have successfully used static custom IPv6 routes with next-hops set to next-hop-gateway and next-hop-instance and validated end-to-end IPv6 communication using those routes.

What's next?

Check out some of these codelabs...

Further reading & Videos

Reference docs