Trigger Kubernetes services with Eventarc events

1. Introduction

cb762f29e9183a3f.png

Eventarc makes it easy to connect various services (Cloud Run, Kubernetes, Workflows) with events from a variety of sources. It allows you to build event-driven architectures in which microservices are loosely coupled and distributed. It also takes care of event ingestion, delivery, security, authorization, and error-handling for you which improves developer agility and application resilience. Check out Trigger Cloud Run with events from Eventarc codelab for an introduction to Eventarc.

In this codelab, you will use Eventarc to read events from Pub/Sub, Cloud Storage and Cloud Audit Logs and pass them to a Kubernetes service running on Google Kubernetes Engine (GKE).

What you'll learn

  • Create a GKE cluster.
  • Create a GKE service as an event sink.
  • Create a Pub/Sub trigger.
  • Create a Cloud Storage trigger
  • Create a Cloud Audit Logs trigger.

2. Setup and requirements

Self-paced environment setup

  1. Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.

b35bf95b8bf3d5d8.png

a99b7ace416376c4.png

bd84a6d3004737c5.png

  • The Project name is the display name for this project's participants. It is a character string not used by Google APIs. You can update it at any time.
  • The Project ID is unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference the Project ID (it is typically identified as PROJECT_ID). If you don't like the generated ID, you may generate another random one. Alternatively, you can try your own and see if it's available. It cannot be changed after this step and will remain for the duration of the project.
  • For your information, there is a third value, a Project Number which some APIs use. Learn more about all three of these values in the documentation.
  1. Next, you'll need to enable billing in the Cloud Console to use Cloud resources/APIs. Running through this codelab shouldn't cost much, if anything at all. To shut down resources so you don't incur billing beyond this tutorial, you can delete the resources you created or delete the whole project. New users of Google Cloud are eligible for the $300 USD Free Trial program.

Start Cloud Shell

While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.

From the Google Cloud Console, click the Cloud Shell icon on the top right toolbar:

55efc1aaa7a4d3ad.png

It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:

7ffe5cbb04455448.png

This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this codelab can be done within a browser. You do not need to install anything.

Before you begin

Inside Cloud Shell, make sure that your project ID is setup:

PROJECT_ID=your-project-id
gcloud config set project $PROJECT_ID

3. Create a GKE cluster

First, enable required services for GKE:

gcloud services enable container.googleapis.com

Create an Autopilot GKE cluster:

CLUSTER_NAME=eventarc-cluster
REGION=us-central1
gcloud container clusters create-auto $CLUSTER_NAME --region $REGION

Make sure the cluster creation is completed before moving onto the next step.

4. Deploy a GKE service

Before deploying a service, get authentication credentials to interact with the cluster with kubectl:

gcloud container clusters get-credentials $CLUSTER_NAME \
    --region $REGION

Next, deploy Cloud Run's hello container as a Kubernetes deployment on GKE. This service logs received HTTP requests and CloudEvents:

kubectl create deployment $SERVICE_NAME \
    --image=gcr.io/cloudrun/hello

Expose the deployment as an internal Kubernetes service. This creates a service with a stable IP accessible within the cluster:

kubectl expose deployment $SERVICE_NAME \
  --type ClusterIP --port 80 --target-port 8080

Before moving onto the next step, make sure the pod is running:

kubectl get pods

NAME                        READY   STATUS
hello-gke-df6469d4b-5vv22   1/1     Running

You can also see the service:

kubectl get svc

NAME         TYPE           CLUSTER-IP    EXTERNAL-IP
hello-gke    LoadBalancer   10.51.1.26    <none>

5. Setup Eventarc

In this step, you will run steps to setup Eventarc and initialize Eventarc GKE destinations.

First, enable required services for Eventarc and Eventarc GKE destinations:

gcloud services enable eventarc.googleapis.com \
  cloudresourcemanager.googleapis.com

Next, enable Eventarc to manage GKE clusters:

gcloud eventarc gke-destinations init

Eventarc creates a separate Event Forwarder pod for each trigger targeting a GKE service, and requires explicit permissions to make changes to the cluster. This is done by granting permissions to a special service account to manage resources in the cluster. This needs to be done once per Google Cloud project.

6. Event discovery

Before creating triggers, you can discover what the event sources are, the types of events they can emit, and how to configure triggers in order to consume them.

You can check the documentation page on events supported by Eventarc.You can also explore the events using gcloud.

To see the list of different types of events:

gcloud beta eventarc attributes types list

NAME                                           DESCRIPTION
google.cloud.audit.log.v1.written              Cloud Audit Log written
google.cloud.pubsub.topic.v1.messagePublished  Cloud Pub/Sub message published
google.cloud.storage.object.v1.archived         Cloud Storage: Sent when a live version of an (object versioned) object is archived or deleted.
google.cloud.storage.object.v1.deleted          Cloud Storage: Sent when an object has been permanently deleted.
google.cloud.storage.object.v1.finalized        Cloud Storage: Sent when a new object (or a new generation of an existing object).
google.cloud.storage.object.v1.metadataUpdated  Cloud Storage: Sent when the metadata of an existing object changes.

To get more information about each event type:

gcloud beta eventarc attributes types describe google.cloud.audit.log.v1.written

attributes: type,serviceName,methodName,resourceName
description: 'Cloud Audit Log: Sent when a log is written.'
name: google.cloud.audit.log.v1.written

To see the list of services that emit a certain event type:

gcloud beta eventarc attributes service-names list --type=google.cloud.audit.log.v1.written

SERVICE_NAME                                DISPLAY_NAME
accessapproval.googleapis.com               Access Approval
accesscontextmanager.googleapis.com         Access Context Manager
admin.googleapis.com                        Google Workspace Admin
aiplatform.googleapis.com                   AI Platform (under Vertex AI)
apigee.googleapis.com                       Apigee
apigeeconnect.googleapis.com                Apigee Connect
...
workflows.googleapis.com                    Workflows

To see the list of method names (sub-events) that each service can emit:

gcloud beta eventarc attributes method-names list --type=google.cloud.audit.log.v1.written --service-name=workflows.googleapis.com

METHOD_NAME
google.cloud.workflows.v1.Workflows.CreateWorkflow
google.cloud.workflows.v1.Workflows.DeleteWorkflow
google.cloud.workflows.v1.Workflows.GetWorkflow
google.cloud.workflows.v1.Workflows.ListWorkflows
google.cloud.workflows.v1.Workflows.UpdateWorkflow
google.cloud.workflows.v1beta.Workflows.CreateWorkflow
google.cloud.workflows.v1beta.Workflows.DeleteWorkflow
google.cloud.workflows.v1beta.Workflows.GetWorkflow
google.cloud.workflows.v1beta.Workflows.ListWorkflows
google.cloud.workflows.v1beta.Workflows.UpdateWorkflow

7. Create a Pub/Sub trigger

One way of receiving events is through Pub/Sub. Any application can publish messages to Pub/Sub and these messages can be delivered to services via Eventarc.

Setup

Before creating any triggers, you need a service account to be used by triggers.

Create a service account:

SERVICE_ACCOUNT=eventarc-gke-trigger-sa
gcloud iam service-accounts create $SERVICE_ACCOUNT

The service account must be granted the following roles for triggers with GKE destinations:

  • roles/pubsub.subscriber
  • roles/monitoring.metricWriter
  • roles/eventarc.eventReceiver (Only for AuditLog triggers. This will be added in a later step)

Grant the roles:

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member serviceAccount:$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com \
  --role roles/pubsub.subscriber

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member serviceAccount:$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com \
  --role roles/monitoring.metricWriter

Create

Create a trigger to route Pub/Sub messages to your service:

TRIGGER_NAME=trigger-pubsub-gke
gcloud eventarc triggers create $TRIGGER_NAME \
  --destination-gke-cluster=$CLUSTER_NAME \
  --destination-gke-location=$REGION \
  --destination-gke-namespace=default \
  --destination-gke-service=$SERVICE_NAME \
  --destination-gke-path=/ \
  --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
  --location=$REGION \
  --service-account=$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com

Test

You can check that the trigger is created by listing all triggers:

gcloud eventarc triggers list

Pub/Sub trigger creates a Pub/Sub topic under the covers. Let's find it out and assign to a variable:

TOPIC_ID=$(gcloud eventarc triggers describe $TRIGGER_NAME --location $REGION --format='value(transport.pubsub.topic)')

Use gcloud to publish a message to the topic:

gcloud pubsub topics publish $TOPIC_ID --message="Hello World"

To check if the event is received, first, find the pod name:

kubectl get pods

NAME                        READY   STATUS
hello-gke-df6469d4b-5vv22   1/1     Running

And store it to a variable:

POD_NAME=hello-gke-df6469d4b-5vv22 

Check the logs of the pod to verify the received event:

kubectl logs $POD_NAME

{
  "severity": "INFO",
  "eventType": "google.cloud.pubsub.topic.v1.messagePublished",
  "message": "Received event of type google.cloud.pubsub.topic.v1.messagePublished. Event data: Hello World",
  "event": {
    "data": {
      "subscription": "projects/atamel-eventarc-gke/subscriptions/eventarc-us-central1-trigger-pubsub-gke-sub-270",
      "message": {
        "data": "SGVsbG8gV29ybGQ=",
        "messageId": "6031025573654834",
        "publishTime": "2022-10-19T14:13:07.990Z"
      }
    },
    "datacontenttype": "application/json",
    "id": "6031025573654834",
    "source": "//pubsub.googleapis.com/projects/atamel-eventarc-gke/topics/eventarc-us-central1-trigger-pubsub-gke-729",
    "specversion": "1.0",
    "time": "2022-10-19T14:13:07.99Z",
    "type": "google.cloud.pubsub.topic.v1.messagePublished"
  }
}

8. Create a Cloud Storage trigger

Another way of receiving events is through Cloud Storage. For example, an application can upload a file to a bucket and that event can be delivered to services via Eventarc.

Setup

Before creating a Cloud Storage trigger, create a bucket to receive events from:

BUCKET_NAME=eventarc-gcs-$PROJECT_ID
gsutil mb -l $REGION gs://$BUCKET_NAME

You also need to add the pubsub.publisher role to the Cloud Storage service account for Cloud Storage triggers:

SERVICE_ACCOUNT_STORAGE=$(gsutil kms serviceaccount -p $PROJECT_ID)
gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member serviceAccount:$SERVICE_ACCOUNT_STORAGE \
    --role roles/pubsub.publisher

Create

Create a trigger to route new file creation events from the bucket to your service:

TRIGGER_NAME=trigger-storage-gke
gcloud eventarc triggers create $TRIGGER_NAME \
  --destination-gke-cluster=$CLUSTER_NAME \
  --destination-gke-location=$REGION \
  --destination-gke-namespace=default \
  --destination-gke-service=$SERVICE_NAME \
  --destination-gke-path=/ \
  --event-filters="type=google.cloud.storage.object.v1.finalized" \
  --event-filters="bucket=$BUCKET_NAME" \
  --location=$REGION \
  --service-account=$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com

Test

You can check that the trigger is created by listing all triggers:

gcloud eventarc triggers list

Create a file and use gsutil to upload the file to the bucket:

echo "Hello World" > random.txt
gsutil cp random.txt gs://$BUCKET_NAME/random.txt

Check the logs of the pod to verify the received event:

kubectl logs $POD_NAME

{
  "severity": "INFO",
  "eventType": "google.cloud.storage.object.v1.finalized",
  "message": "Received event of type google.cloud.storage.object.v1.finalized. Event data: {\n  \"kind\": \"storage#object\",\n  \"id\": \"eventarc-gcs-atamel-eventarc-gke/random.txt/1666190425669022\",\n  \"selfLink\": \"https://www.googleapis.com/storage/v1/b/eventarc-gcs-atamel-eventarc-gke/o/random.txt\",\n  \"name\": \"random.txt\",\n  \"bucket\": \"eventarc-gcs-atamel-eventarc-gke\",\n  \"generation\": \"1666190425669022\",\n  \"metageneration\": \"1\",\n  \"contentType\": \"text/plain\",\n  \"timeCreated\": \"2022-10-19T14:40:25.678Z\",\n  \"updated\": \"2022-10-19T14:40:25.678Z\",\n  \"storageClass\": \"STANDARD\",\n  \"timeStorageClassUpdated\": \"2022-10-19T14:40:25.678Z\",\n  \"size\": \"12\",\n  \"md5Hash\": \"5Z/5eUEET4XfUpfhwwLSYA==\",\n  \"mediaLink\": \"https://storage.googleapis.com/download/storage/v1/b/eventarc-gcs-atamel-eventarc-gke/o/random.txt?generation=1666190425669022\u0026alt=media\",\n  \"contentLanguage\": \"en\",\n  \"crc32c\": \"R1jUOQ==\",\n  \"etag\": \"CJ77zIPD7PoCEAE=\"\n}\n",
  "event": {
    "bucket": "eventarc-gcs-atamel-eventarc-gke",
    "data": {
      "kind": "storage#object",
      "id": "eventarc-gcs-atamel-eventarc-gke/random.txt/1666190425669022",
      "selfLink": "https://www.googleapis.com/storage/v1/b/eventarc-gcs-atamel-eventarc-gke/o/random.txt",
      "name": "random.txt",
      "bucket": "eventarc-gcs-atamel-eventarc-gke",
      "generation": "1666190425669022",
      "metageneration": "1",
      "contentType": "text/plain",
      "timeCreated": "2022-10-19T14:40:25.678Z",
      "updated": "2022-10-19T14:40:25.678Z",
      "storageClass": "STANDARD",
      "timeStorageClassUpdated": "2022-10-19T14:40:25.678Z",
      "size": "12",
      "md5Hash": "5Z/5eUEET4XfUpfhwwLSYA==",
      "mediaLink": "https://storage.googleapis.com/download/storage/v1/b/eventarc-gcs-atamel-eventarc-gke/o/random.txt?generation=1666190425669022\u0026alt=media",
      "contentLanguage": "en",
      "crc32c": "R1jUOQ==",
      "etag": "CJ77zIPD7PoCEAE="
    },
    "datacontenttype": "application/json",
    "id": "6031255652220627",
    "source": "//storage.googleapis.com/projects/_/buckets/eventarc-gcs-atamel-eventarc-gke",
    "specversion": "1.0",
    "subject": "objects/random.txt",
    "time": "2022-10-19T14:40:25.678152Z",
    "type": "google.cloud.storage.object.v1.finalized"
  }
}

9. Create a Cloud Audit Logs trigger

Although Cloud Storage trigger is the better way to listen for Cloud Storage events, in this step, you create a Cloud Audit Log trigger to do the same.

Setup

To receive events from a service, you need to enable Audit Logs. From the Google Cloud Console, select IAM & Admin and Audit Logs from the upper left-hand menu. In the list of services, check Google Cloud Storage:

91d1bcef8f953fe3.png

On the right hand side, make sure Admin, Read and Write are selected and click Save:

ccb31db1e55fd2e3.png

You also need to add eventarc.eventReceiver role to the trigger service account for Cloud Audit Logs triggers:

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member serviceAccount:$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com \
  --role roles/eventarc.eventReceiver

Create

Create a trigger to route new file creation events from the bucket to your service:

TRIGGER_NAME=trigger-auditlog-storage-gke
gcloud eventarc triggers create $TRIGGER_NAME \
  --destination-gke-cluster=$CLUSTER_NAME \
  --destination-gke-location=$REGION \
  --destination-gke-namespace=default \
  --destination-gke-service=$SERVICE_NAME \
  --destination-gke-path=/ \
  --event-filters="type=google.cloud.audit.log.v1.written" \
  --event-filters="serviceName=storage.googleapis.com" \
  --event-filters="methodName=storage.objects.create" \
  --event-filters-path-pattern="resourceName=/projects/_/buckets/$BUCKET_NAME/objects/*" \
  --location=$REGION \
  --service-account=$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com

Test

Audit Logs triggers take a little bit of time to initialize. You can check that the trigger is created by listing all triggers:

gcloud eventarc triggers list

You should see that the Active field is Yes:

NAME                          TYPE                                           DESTINATION     ACTIVE
trigger-auditlog-storage-gke  google.cloud.audit.log.v1.written              GKE: hello-gke  Yes

Create a file and use gsutil to upload the file to the bucket:

echo "Hello World" > random.txt
gsutil cp random.txt gs://$BUCKET_NAME/random.txt

Check the logs of the pod to verify the received event:

kubectl logs $POD_NAME

{
  "severity": "INFO",
  "eventType": "google.cloud.audit.log.v1.written",
  "message": "Received event of type google.cloud.audit.log.v1.written. Event data: {\"protoPayload\":{\"status\":{},\"authenticationInfo\":{\"principalEmail\":\"atameldev@gmail.com\"},\"requestMetadata\":{\"callerIp\":\"149.71.143.227\",\"callerSuppliedUserAgent\":\"apitools Python/3.10.4 gsutil/5.14 (darwin) analytics/disabled interactive/True command/cp google-cloud-sdk/405.0.1,gzip(gfe)\",\"requestAttributes\":{\"time\":\"2022-10-19T15:05:54.144615670Z\",\"auth\":{}},\"destinationAttributes\":{}},\"serviceName\":\"storage.googleapis.com\",\"methodName\":\"storage.objects.create\",\"authorizationInfo\":[{\"resource\":\"projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt\",\"permission\":\"storage.objects.delete\",\"granted\":true,\"resourceAttributes\":{}},{\"resource\":\"projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt\",\"permission\":\"storage.objects.create\",\"granted\":true,\"resourceAttributes\":{}}],\"resourceName\":\"projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt\",\"serviceData\":{\"@type\":\"type.googleapis.com/google.iam.v1.logging.AuditData\",\"policyDelta\":{\"bindingDeltas\":[{\"action\":\"ADD\",\"role\":\"roles/storage.legacyObjectOwner\",\"member\":\"projectOwner:atamel-eventarc-gke\"},{\"action\":\"ADD\",\"role\":\"roles/storage.legacyObjectOwner\",\"member\":\"projectEditor:atamel-eventarc-gke\"},{\"action\":\"ADD\",\"role\":\"roles/storage.legacyObjectOwner\",\"member\":\"user:atameldev@gmail.com\"},{\"action\":\"ADD\",\"role\":\"roles/storage.legacyObjectReader\",\"member\":\"projectViewer:atamel-eventarc-gke\"}]}},\"resourceLocation\":{\"currentLocations\":[\"us-central1\"]}},\"insertId\":\"-8vmrbve7pol2\",\"resource\":{\"type\":\"gcs_bucket\",\"labels\":{\"project_id\":\"atamel-eventarc-gke\",\"bucket_name\":\"eventarc-gcs-atamel-eventarc-gke\",\"location\":\"us-central1\"}},\"timestamp\":\"2022-10-19T15:05:54.138732321Z\",\"severity\":\"INFO\",\"logName\":\"projects/atamel-eventarc-gke/logs/cloudaudit.googleapis.com%2Fdata_access\",\"receiveTimestamp\":\"2022-10-19T15:05:54.839604461Z\"}",
  "event": {
    "data": {
      "protoPayload": {
        "status": {
        },
        "authenticationInfo": {
          "principalEmail": "atameldev@gmail.com"
        },
        "requestMetadata": {
          "callerIp": "149.71.143.227",
          "callerSuppliedUserAgent": "apitools Python/3.10.4 gsutil/5.14 (darwin) analytics/disabled interactive/True command/cp google-cloud-sdk/405.0.1,gzip(gfe)",
          "requestAttributes": {
            "time": "2022-10-19T15:05:54.144615670Z",
            "auth": {
            }
          },
          "destinationAttributes": {
          }
        },
        "serviceName": "storage.googleapis.com",
        "methodName": "storage.objects.create",
        "authorizationInfo": [
          {
            "resource": "projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt",
            "permission": "storage.objects.delete",
            "granted": true,
            "resourceAttributes": {
            }
          },
          {
            "resource": "projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt",
            "permission": "storage.objects.create",
            "granted": true,
            "resourceAttributes": {
            }
          }
        ],
        "resourceName": "projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt",
        "serviceData": {
          "@type": "type.googleapis.com/google.iam.v1.logging.AuditData",
          "policyDelta": {
            "bindingDeltas": [
              {
                "action": "ADD",
                "role": "roles/storage.legacyObjectOwner",
                "member": "projectOwner:atamel-eventarc-gke"
              },
              {
                "action": "ADD",
                "role": "roles/storage.legacyObjectOwner",
                "member": "projectEditor:atamel-eventarc-gke"
              },
              {
                "action": "ADD",
                "role": "roles/storage.legacyObjectOwner",
                "member": "user:atameldev@gmail.com"
              },
              {
                "action": "ADD",
                "role": "roles/storage.legacyObjectReader",
                "member": "projectViewer:atamel-eventarc-gke"
              }
            ]
          }
        },
        "resourceLocation": {
          "currentLocations": [
            "us-central1"
          ]
        }
      },
      "insertId": "-8vmrbve7pol2",
      "resource": {
        "type": "gcs_bucket",
        "labels": {
          "project_id": "atamel-eventarc-gke",
          "bucket_name": "eventarc-gcs-atamel-eventarc-gke",
          "location": "us-central1"
        }
      },
      "timestamp": "2022-10-19T15:05:54.138732321Z",
      "severity": "INFO",
      "logName": "projects/atamel-eventarc-gke/logs/cloudaudit.googleapis.com%2Fdata_access",
      "receiveTimestamp": "2022-10-19T15:05:54.839604461Z"
    },
    "datacontenttype": "application/json; charset=utf-8",
    "dataschema": "https://googleapis.github.io/google-cloudevents/jsonschema/google/events/cloud/audit/v1/LogEntryData.json",
    "id": "projects/atamel-eventarc-gke/logs/cloudaudit.googleapis.com%2Fdata_access-8vmrbve7pol21666191954138732",
    "methodname": "storage.objects.create",
    "recordedtime": "2022-10-19T15:05:54.138732321Z",
    "resourcename": "projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt",
    "servicename": "storage.googleapis.com",
    "source": "//cloudaudit.googleapis.com/projects/atamel-eventarc-gke/logs/data_access",
    "specversion": "1.0",
    "subject": "storage.googleapis.com/projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt",
    "time": "2022-10-19T15:05:54.839604461Z",
    "type": "google.cloud.audit.log.v1.written"
  }
}

10. Congratulations!

Congratulations for completing the codelab.

What we've covered

  • Create a GKE cluster.
  • Create a GKE service as an event sink.
  • Create a Pub/Sub trigger.
  • Create a Cloud Storage trigger.
  • Create a Cloud Audit Logs trigger.