Access files in Cloud Storage with the Spring Resource abstraction

1. Overview

Spring Framework provides a ResourceLoader abstraction to easily read and write files from various sources, such as the file system, classpath, or web. You simply need to specify the URI to the resource using the well-known protocol prefix. For example, to access a file on the local file system, you would specify a URI like file:/data/config.yaml.

You'll write a Spring Boot app that will access files stored in Cloud Storage by using the Spring Resource abstraction and the gs: protocol prefix.

You'll do that by using Cloud Shell and the Cloud SDK gcloud command-line tool.

What you'll learn

• How to use the Cloud Storage Spring Boot starter
• How to access files in Cloud Storage with Spring
• How to use Spring's Resource and WritableResource abstractions

What you'll need

• A Google Cloud project
• A browser, such as Google Chrome
• Familiarity with standard Linux text editors, such as Vim, Emacs, and GNU Nano

How will you use the codelab?

Read only Read and complete the exercises

How would you rate your experience with building HTML and CSS web apps?

Novice Intermediate Proficient

How would you rate your experience with using Google Cloud services?

Novice Intermediate Proficient

2. Setup and requirements

Self-paced environment setup

1. Sign in to Cloud Console and create a new project or reuse an existing one. (If you don't already have a Gmail or G Suite account, you must create one.)

Note: You can easily access Cloud Console by memorizing its URL, which is console.cloud.google.com.

Remember the project ID, a unique name across all Google Cloud projects (the name above has already been taken and will not work for you, sorry!). It will be referred to later in this codelab as PROJECT_ID.

Note: If you're using a Gmail account, you can leave the default location set to No organization. If you're using a G Suite account, then choose a location that makes sense for your organization.

2. Next, you'll need to enable billing in Cloud Console in order to use Google Cloud resources.

Running through this codelab shouldn't cost much, if anything at all. Be sure to to follow any instructions in the "Cleaning up" section which advises you how to shut down resources so you don't incur billing beyond this tutorial. New users of Google Cloud are eligible for the $300USD Free Trial program.

Cloud Shell

You'll use Cloud Shell, a command-line environment running in Google Cloud.

Activate Cloud Shell

1. From the Cloud Console, click Activate Cloud Shell .

If you've never started Cloud Shell before, you'll be presented with an intermediate screen (below the fold) describing what it is. If that's the case, click Continue (and you won't ever see it again). Here's what that one-time screen looks like:

It should only take a few moments to provision and connect to Cloud Shell.

This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory and runs in Google Cloud, greatly enhancing network performance and authentication. Much, if not all, of your work in this codelab can be done with simply a browser or your Chromebook.

Once connected to Cloud Shell, you should see that you are already authenticated and that the project is already set to your project ID.

2. Run the following command in Cloud Shell to confirm that you are authenticated:

gcloud auth list

Command output

 Credentialed Accounts
ACTIVE  ACCOUNT
*       <my_account>@<my_domain.com>

To set the active account, run:
    $ gcloud config set account `ACCOUNT`

Note: The gcloud command-line tool is the powerful and unified command-line tool in Google Cloud. It comes preinstalled in Cloud Shell. You will notice its support for tab completion. For more information, see gcloud command-line tool overview.

gcloud config list project

Command output

[core]
project = <PROJECT_ID>

If it is not, you can set it with this command:

gcloud config set project <PROJECT_ID>

Command output

Updated property [core/project].

3. Create a file in Cloud Storage

After Cloud Shell launches, you can start creating files and transferring them to Cloud Storage.

Create a file named my-file.txt:

$ echo "Hello World from GCS" > my-file.txt

Then create a new unique bucket in Cloud Storage and transfer the file there using gsutil.

$ BUCKET=spring-bucket-$USER
$ gsutil makebucket gs://$BUCKET
$ gsutil copy my-file.txt gs://$BUCKET

Navigate to the storage browser in Cloud Storage, and verify that the bucket and the file are there.

4. Initialize a Spring Boot app

Start writing the app by using the command line to generate a new Spring Boot app with Spring Initializr:

$ curl https://start.spring.io/starter.tgz \
  -d type=maven-project \
  -d dependencies=web,cloud-gcp-storage -d baseDir=spring-gcs | tar -xzvf -

Note that the Initializr will automatically add the spring-boot-starter-web and spring-cloud-gcp-starter-storage to your dependencies in the pom.xml of the template app.

Change to the directory of the template app:

$ cd spring-gcs

Make sure JAVA_HOME is set to the correct JDK version:

$ export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64/

Build and run the app using Maven.

$ ./mvnw spring-boot:run

The app will start listening on port 8080. Open a new Cloud Shell tab and run curl to access the app.

$ curl localhost:8080

You should get a 404 response because the app doesn't do anything useful yet.

Return to the previous Cloud Shell tab where the app is running and kill it with Control+C (Command+C on Macintosh).

5. Read the file in Cloud Storage

Modify your Spring Boot app to access my-file.txt, the file that you previously stored in Cloud Storage. Your goal is to simply return the contents of the file via HTTP.

In the following instructions, you'll use Vim to edit the files, but you can also use Emacs, GNU Nano, or the built-in code editor in Cloud Shell:

$ cd ~/spring-gcs

Add a REST controller GcsController to the app.

$ vi src/main/java/com/example/demo/GcsController.java

Paste the following code, and don't forget to fix the resource URI with the bucket that you created previously. You can check the bucket by running echo $BUCKET command.

src/main/java/com/example/demo/GcsController.java

package com.example.demo;

import java.io.IOException;
import java.nio.charset.Charset;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.util.StreamUtils;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class GcsController {

  @Value("gs://REPLACE_WITH_YOUR_BUCKET/my-file.txt")
  private Resource gcsFile;

  @GetMapping("/")
  public String readGcsFile() throws IOException {
    return StreamUtils.copyToString(
        gcsFile.getInputStream(),
        Charset.defaultCharset());
  }
}

Build and run the app with Maven:

$ ./mvnw spring-boot:run

The app starts listening on port 8080. Open a new Cloud Shell tab and run curl to access the app.

$ curl localhost:8080

You should now see that the contents of the file returned from the app. Go to the previous Cloud Shell tab where the app is running and kill it with Control+C (Command+C on Macintosh).

6. Write to the file in Cloud Storage

You read the contents of the file in Cloud Storage and exposed it through a Spring REST controller. Now, change the contents of the file by posting the new file content to the same HTTP endpoint.

You need to add another method to GcsController that will respond to HTTP POST and write the data to your file in Cloud Storage. This time, cast the Spring Resource to WritableResource.

Update the GcsController with additional imports that you need.

src/main/java/com/example/demo/GcsController.java

import java.io.OutputStream;
import org.springframework.core.io.WritableResource;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.PostMapping;

Add the new endpoint method to the controller.

src/main/java/com/example/demo/GcsController.java

@RestController
public class GcsController {

  @PostMapping("/")
  String writeGcs(@RequestBody String data) throws IOException {
    try (OutputStream os = ((WritableResource) gcsFile).getOutputStream()) {
      os.write(data.getBytes());
    }
    return "file was updated\n";
  }
  ...
}

Build and run the app with Maven:

$ ./mvnw spring-boot:run

The app starts listening on port 8080. Open up a new Cloud Shell tab and run curl to post a message to the app.

$ curl -d 'new message' -H 'Content-Type: text/plain' localhost:8080

You should see a confirmation that the contents of the file were updated. However, verify that by doing a GET.

$ curl localhost:8080

You should see the updated contents of the file returned from the app. Return to the previous Cloud Shell tab where the app is running and kill it with Control+C (Command+C on Macintosh).

7. Congratulations!

You learned to use the Spring Resource abstraction to easily access files in Cloud Storage. You wrote a Spring Boot web app that can read and write to a file in Cloud Storage. You also learned about the Spring Boot starter for Cloud Storage that enables that functionality.

Learn More

• Cloud Storage
• Spring Framework on Google Cloud
• Spring on Google Cloud GitHub repository
• Java on Google Cloud

License

This work is licensed under a Creative Commons Attribution 2.0 Generic License.