This codelab will teach you how to embed 360 degree VR media into websites using VR View. This technology is designed for developers of traditional apps to enhance them with immersive content. For example, VR view makes it easy for a travel app to provide viewers with an underwater scuba diving tour as they plan vacation or for a home builder to take prospective buyers on virtual walkthrough before the home is built.

VR View is available not only on Android, but also for iOS and web browsers.. There is a companion codelab to this one which introduces VR view in HTML.

What you'll learn

What you'll need

Tell us a little about you

How will you use use this tutorial?

Read it through only Read it and complete the exercises

How would you rate your experience with building Android apps?

Never tried Novice Intermediate Proficient

How would you rate your experience with Google Cardboard VR?

Never tried Novice Intermediate Proficient

How would you rate your experience with VR image and video capture?

Never tried Novice Intermediate Proficient

For this codelab, we'll start with an application that has 2 fragments. One is about Google I/O 2016 codelabs. The other fragment is like an encyclopedia page. We'll replace some of the included images with VR View elements that will give the 360 degree view, and an immersive experience via Cardboard.

Get the sample project

For this codelab, we've made an Android application as a starting point. During this codelab, we'll walk through how easy it is to add VR View.

Download sample code package

...or clone the GitHub repository from the command line.

$ git clone https://github.com/googlecodelabs/vr_view_app_101.git

First, let's look at the starter app in Android Studio.

Start Android Studio and open the vr_view_app_101 project, by selecting "Import project" from the Welcome screen, or from the menu by File > New > Import project.... Select the build.gradle file and press OK.

Next, we need to build the project manually the first time. After this first build changes will be automatically detected by Android Studio. To Build, select Build > Make Project.

To run the application, press the Run button on the toolbar. This will present a dialog to select the deployment target. If no devices are connected, connect a device or start an emulator to run. When the application launches you will see:

The sample application is a simple tab layout with 2 fragments. Next, let's take a panoramic picture and add it to the project.

While the web page is nice, let's enrich the experience by adding a 360 degree view of I/O. Grab the phone and start the Cardboard Camera app. This app will take a pretty good panoramic picture with a stereoscopic effect.

Click on the camera button and follow the prompts to take a picture. It will take a few seconds to process the image and save it.

Copy the image to the computer

Once it is saved, we'll copy it from the phone to the computer. The easiest way to do this is reconnect your phone to the computer as a media device. Then copy the Cardboard Camera image which is found in DCIM/CardboardCamera. It will have a name something like IMG_20160512_105228.vr.jpg.

Supported image and video formats

Whew! Now we have our image, but unfortunately it is not quite in the format we need. VR View has specific image configuration requirements:

Format the image

Whew! Now we have our image, but unfortunately it is not quite in the format we need.

If we look at the properties of our image (using Finder/Get Info) it is 10515x1765. To change it, we'll use a converter page to reformat it.

Open https://storage.googleapis.com/cardboard-camera-converter/index.html

Select the image copied or drop it on the page. Then download the converted image and copy it to app/src/main/assets/converted.jpg.

Open app/build.gradle in Android Studio and scroll to the bottom of the file to the dependencies section and add the google vr components. The dependency section should then look like:

dependencies {
    compile 'com.android.support:appcompat-v7:25.1.0'
    compile 'com.android.support:design:25.1.0'

    compile 'com.google.vr:sdk-audio:1.10.0'
    compile 'com.google.vr:sdk-base:1.10.0'
    compile 'com.google.vr:sdk-common:1.10.0'
    compile 'com.google.vr:sdk-commonwidget:1.10.0'
    compile 'com.google.vr:sdk-panowidget:1.10.0'
    compile 'com.google.vr:sdk-videowidget:1.10.0'
}

Save the file and click the "Sync Now" alert to re-sync the Gradle project. Now we can reference the library components in our project.

Now that we have a panoramic (and stereo!) image, let's add to our application.

Add VrPanoramaView to the layout

Now that we have the dependencies added, let's edit the layout to replace the static image with a panoramic image.

Open app/res/layout/welcome_fragment.xml

Find the ImageView element and replace it with the VrPanoramaView

    <com.google.vr.sdk.widgets.pano.VrPanoramaView
        android:id="@+id/pano_view"
        android:layout_weight="5"
        android:layout_height="0dp"
        android:layout_margin="5dip"
        android:layout_width="match_parent"
        android:scrollbars="none"
        android:contentDescription="@string/codelab_img_description"/>

Add code to control the VrPanoramaView

Open the file app/java/com.google.devrel.vrviewapp/WelcomeFragment.java in Android Studio.

At the top of the class, add a member variable for the view:

private VrPanoramaView panoWidgetView;

Make sure the class is imported by adding the import statement above the class:

import com.google.vr.sdk.widgets.pano.VrPanoramaView;

Then, in the onCreateView() method, initialize the panoWidgetView variable from the inflated view. Replace the contents of onCreateView() with:

View v =  inflater.inflate(R.layout.welcome_fragment, container,false);
panoWidgetView = (VrPanoramaView) v.findViewById(R.id.pano_view);
return v;

Also add onPause(),onResume(), and onDestroy() methods to pass them to the VrPanoramaView. You should add them just below the onCreateView() method.

@Override
public void onPause() {
   panoWidgetView.pauseRendering();
   super.onPause();
}

@Override
public void onResume() {
   panoWidgetView.resumeRendering();
   super.onResume();
}

@Override
public void onDestroy() {
   // Destroy the widget and free memory.
   panoWidgetView.shutdown();
   super.onDestroy();
}

Now we have a place to display the image, let's add the code to load it.

Since the image is large we don't want to load it in the main UI thread when the application starts. We want to load it asynchronously. For more details about image loading best practices see: http://developer.android.com/training/displaying-bitmaps/index.html.

Create a new class

Select the app/java folder in the project outline, and then right mouse click and select New > Java Class. Name the class com.google.devrel.vrviewapp.ImageLoaderTask.

It needs to extend the AsyncTask class so it can perform the image loading in a background thread. The parameters to the AsyncTask are:

Change the class declaration to be:

public class ImageLoaderTask extends AsyncTask<AssetManager, Void, Bitmap>  {
}

Make sure the classes are imported:

import android.content.res.AssetManager;
import android.graphics.Bitmap;
import android.os.AsyncTask;

There still is a problem. AsyncTask is an abstract class, so we need to implement the abstract methods.

Add a skeleton implementation of doInBackground at the bottom of the class to resolve the error.

@Override
protected Bitmap doInBackground(AssetManager... params) {
    return null;
}

Add member variables and constructor

When we load the image, we need to pass which image to load and where to display it. We'll pass this information via the constructor. Also add a TAG member so we can log errors. Just below the opening brace of the class add:

private static final String TAG = "ImageLoaderTask";
private final String assetName;
private final WeakReference<VrPanoramaView> viewReference;
private final VrPanoramaView.Options viewOptions;

We use a WeakReference for the VrPanoramaView since the view could be destroyed while loading the image. A common cause of this is rotating the phone to another orientation. By using a weak reference, the object can be garbage collected immediately instead of waiting for this async task to be destroyed.

Make sure to import the class for VrPanoramaView. If you have not already added them, add the import statements now:

import com.google.vr.sdk.widgets.pano.VrPanoramaView;
import java.lang.ref.WeakReference;

To avoid re-loading the image when the device is rotated, we'll cache the last image loaded. We do this by keeping the name of the last asset loaded, and the resulting bitmap. Add these member variables:

private static WeakReference<Bitmap> lastBitmap = new WeakReference<>(null);
private static String lastName;

Since these member variables are final, they need to be initialized in the constructor. Add the required constructor below the member variables:

public ImageLoaderTask(VrPanoramaView view, VrPanoramaView.Options viewOptions, String assetName) {
    viewReference = new WeakReference<>(view);
    this.viewOptions = viewOptions;
    this.assetName = assetName;
}

Load the image in the background

For this example, we've added the image to the assets directory of the project, so we'll use the AssetManager to get an InputStream to the image. Then pass that input stream to the BitmapFactory to load the image and return it back to the main thread. If there is a problem, we'll log it and return a null image. We check the last image loaded before opening the stream in order to conserve memory usage.

Replace the contents of doInBackground(AssetManager... params) with:

AssetManager assetManager = params[0];

if (assetName.equals(lastName) && lastBitmap.get() != null) {
    return lastBitmap.get();
}

try(InputStream istr = assetManager.open(assetName)) {
    Bitmap b = BitmapFactory.decodeStream(istr);
    lastBitmap = new WeakReference<>(b);
    lastName = assetName;
    return b;
} catch (IOException e) {
    Log.e(TAG, "Could not decode default bitmap: " + e);
    return null;
}

Then add the import statements at the top:

import android.graphics.BitmapFactory;
import android.util.Log;

import java.io.IOException;
import java.io.InputStream;

Display the image

Back in the main thread, we can render the bitmap in the VrPanoramaView. Once the background work is done, the onPostExecute(Bitmap bitmap) method is called by AsyncTask on the main thread. In here, we'll pass the bitmap to the VrPanoramaView.

Just below doInBackground(), add:

@Override
protected void onPostExecute(Bitmap bitmap) {
    final VrPanoramaView vw = viewReference.get();
    if (vw != null && bitmap != null) {
        vw.loadImageFromBitmap(bitmap, viewOptions);
    }
}

Start the background loading

OK - so now that we have an AsyncTask to load and display the image, all that is left is calling it.

Go back to WelcomeFragment.java. We need a new member variable, so at the top of the class add

private ImageLoaderTask backgroundImageLoaderTask;

Then at the bottom of the class add a new method loadPanoImage(). This will create a new loader task and start it.

private synchronized void loadPanoImage() {
    ImageLoaderTask task = backgroundImageLoaderTask;
    if (task != null && !task.isCancelled()) {
        // Cancel any task from a previous loading.
        task.cancel(true);
    }

    // pass in the name of the image to load from assets.
    VrPanoramaView.Options viewOptions = new VrPanoramaView.Options();
    viewOptions.inputType = VrPanoramaView.Options.TYPE_STEREO_OVER_UNDER;

    // use the name of the image in the assets/ directory.
    String panoImageName = "converted.jpg";

    // create the task passing the widget view and call execute to start.
    task = new ImageLoaderTask(panoWidgetView, viewOptions, panoImageName);
    task.execute(getActivity().getAssets());
    backgroundImageLoaderTask = task;
}

Finally, we need to tie it to a lifecycle event to start the actual loading. In a fragment we'll start it in onActivityCreated(). Add a new function at the end of the WelcomeFragment class:

@Override
public void onActivityCreated(@Nullable Bundle savedInstanceState) {
    super.onActivityCreated(savedInstanceState);
    loadPanoImage();
}

Save the file and try out the VR view image!

Make sure your device is attached, and click Run. Hold the phone upright and as you turn around the view changes as if you are looking around in the image!.

If you click on the Full Screen icon in the bottom right of the image, the image will be displayed using the entire screen.

If you click the cardboard icon, it will start the view in Cardboard mode for stereo VR mode.

Next step, Let's try some video!

Capturing panoramic and stereo video is more than we can do on our simple Android phone. For this example, we'll model an encyclopedia page about gorillas. We'll replace a normal image with a panoramic video.

We can add video using the same general approach, just use the video view widget instead of the image one. We'll also add a seekbar control so we can move through the video.

Add the Video to the layout

Open res/layout/gorilla_fragment.xml (using the Text view) and replace the ImageView element with the following elements:

<com.google.vr.sdk.widgets.video.VrVideoView
    android:id="@+id/video_view"
    android:layout_width="match_parent"
    android:scrollbars="none"
    android:layout_height="250dip"/>

<!-- Seeking UI & progress indicator.-->
<SeekBar
    android:id="@+id/seek_bar"
    style="?android:attr/progressBarStyleHorizontal"
    android:layout_height="32dp"
    android:layout_width="fill_parent"/>
<TextView
    android:id="@+id/status_text"
    android:text="Loading Video..."
    android:layout_height="wrap_content"
    android:layout_width="fill_parent"
    android:textSize="12sp"
    android:paddingStart="32dp"
    android:paddingEnd="32dp"/>

Add member variables

Open app/java/com.google.devrel.vrviewapp/GorillaFragment.java, and at the top of the class add these member variables:

   /**
     * Preserve the video's state and duration when rotating the phone. This improves 
     * performance when rotating or reloading the video.
     */
    private static final String STATE_IS_PAUSED = "isPaused";
    private static final String STATE_VIDEO_DURATION = "videoDuration";
    private static final String STATE_PROGRESS_TIME = "progressTime";


    /**
     * The video view and its custom UI elements.
     */
    private VrVideoView videoWidgetView;

    /**
     * Seeking UI & progress indicator. The seekBar's progress value represents milliseconds in the
     * video.
     */
    private SeekBar seekBar;
    private TextView statusText;

    /**
     * By default, the video will start playing as soon as it is loaded. 
     */
    private boolean isPaused = false;


Remember to add the import statements if they are not already there:

import com.google.vr.sdk.widgets.video.VrVideoView;
import android.widget.SeekBar;
import android.widget.TextView;

Initialize the member variables

Now that we have declared the member variables, we need to initialize them from the inflated view fragment. Find the onCreateView() method and replace it with this version which initializes the variables:

@Nullable
@Override
public View onCreateView(LayoutInflater inflater, @Nullable ViewGroup container, @Nullable Bundle savedInstanceState) {
    View view = inflater.inflate(R.layout.gorilla_fragment, container,false);
    seekBar = (SeekBar) view.findViewById(R.id.seek_bar);
    statusText = (TextView) view.findViewById(R.id.status_text);
    videoWidgetView = (VrVideoView) view.findViewById(R.id.video_view);

    // Add the restore state code here.
        
    // Add the seekbar listener here.

    // Add the VrVideoView listener here

    return view;
}

Initialize the SeekBar Listener

The SeekBar needs to have a listener set to respond to the user changing the seekbar position by moving the video playback to that point.

Add the listener to onCreateView(), just before the return statement.

 // initialize the seekbar listener 
 seekBar.setOnSeekBarChangeListener(new SeekBar.OnSeekBarChangeListener() {
    @Override
    public void onProgressChanged(SeekBar seekBar, int progress, boolean fromUser) {

        // if the user changed the position, seek to the new position.
        if (fromUser) {
            videoWidgetView.seekTo(progress);
            updateStatusText();
        }
    }

    @Override
    public void onStartTrackingTouch(SeekBar seekBar) {
        // ignore for now.
    }

    @Override
    public void onStopTrackingTouch(SeekBar seekBar) {
        // ignore for now.
    }
});

Initialize the VrVideoView Listener

The VrVideoView needs to have a listener set to respond to the changing state. Specifically:

Add the listener to onCreateView(), just before the return statement.

// initialize the video listener
videoWidgetView.setEventListener(new VrVideoEventListener() {
    /**
     * Called by video widget on the UI thread when it's done loading the video.
     */
    @Override
    public void onLoadSuccess() {
        Log.i(TAG, "Successfully loaded video " + videoWidgetView.getDuration());
        seekBar.setMax((int) videoWidgetView.getDuration());
        seekBar.setEnabled(true);
        updateStatusText();
   }

   /**
    * Called by video widget on the UI thread on any asynchronous error.
    */
   @Override
   public void onLoadError(String errorMessage) {
       Toast.makeText(
          getActivity(), "Error loading video: " + errorMessage, Toast.LENGTH_LONG)
                      .show();
       Log.e(TAG, "Error loading video: " + errorMessage);
    }

    @Override
    public void onClick() {
        if (isPaused) {
              videoWidgetView.playVideo();
        } else {
            videoWidgetView.pauseVideo();
        }

        isPaused = !isPaused;
        updateStatusText();
    }

    /**
    * Update the UI every frame.
    */
    @Override
    public void onNewFrame() {
        updateStatusText();
        seekBar.setProgress((int) videoWidgetView.getCurrentPosition());
    }

    /**
     * Make the video play in a loop. This method could also be used to move to the next video in
     * a playlist.
     */
    @Override
    public void onCompletion() {
        videoWidgetView.seekTo(0);
    }
});

This code references some classes not referenced before, so add the import statements at the top:

import com.google.vr.sdk.widgets.video.VrVideoEventListener;
import android.util.Log;
import android.widget.Toast;

This code is almost done, there is a missing member function called in a couple places, updateStatusText(). Let's add this at the end of the class.

private void updateStatusText() {
    String status = (isPaused ? "Paused: " : "Playing: ") +
            String.format(Locale.getDefault(), "%.2f", videoWidgetView.getCurrentPosition() / 1000f) +
            " / " +
            videoWidgetView.getDuration() / 1000f +
            " seconds.";
    statusText.setText(status);
}

And add another import statement:

import java.util.Locale;

Handle video state saving

When the phone is rotated, the view for the activity is recreated. Since the initial loading of the video is needed to set the seekbar values we want to save this value to use in the event of re-creating the activity. We also want to save the running state of the video so if the phone is rotated, it does not start playing the video if it was paused.

To do this, overload the method: onSaveInstanceState(). Add this at the end of the class:

@Override
public void onSaveInstanceState(Bundle savedInstanceState) {
    savedInstanceState.putLong(STATE_PROGRESS_TIME, videoWidgetView.getCurrentPosition());
    savedInstanceState.putLong(STATE_VIDEO_DURATION, videoWidgetView.getDuration());
    savedInstanceState.putBoolean(STATE_IS_PAUSED, isPaused);
    super.onSaveInstanceState(savedInstanceState);
}

Then add reading the saved state in the onCreateView() method. Add this code right after initializing the member variables, where there is a comment "// Add the restore state code here."

// initialize based on the saved state
if (savedInstanceState != null) {
    long progressTime = savedInstanceState.getLong(STATE_PROGRESS_TIME);
    videoWidgetView.seekTo(progressTime);
    seekBar.setMax((int)savedInstanceState.getLong(STATE_VIDEO_DURATION));
    seekBar.setProgress((int) progressTime);

    isPaused = savedInstanceState.getBoolean(STATE_IS_PAUSED);
    if (isPaused) {
        videoWidgetView.pauseVideo();
    }
} else {
    seekBar.setEnabled(false);
}

Handle lifecycle events

We need to pass the onPause, onResult and onDestroy events into the VrVideoView. Add those methods at the end of the class:

@Override
public void onPause() {
    super.onPause();
    // Prevent the view from rendering continuously when in the background.
    videoWidgetView.pauseRendering();
    // If the video was playing when onPause() is called, the default behavior will be to pause
    // the video and keep it paused when onResume() is called.
    isPaused = true;
}

@Override
public void onResume() {
    super.onResume();
    // Resume the 3D rendering.
    videoWidgetView.resumeRendering();
    // Update the text to account for the paused video in onPause().
    updateStatusText();
}

@Override
public void onDestroy() {
    // Destroy the widget and free memory.
    videoWidgetView.shutdown();
    super.onDestroy();
}

Start the video when visible

Since we are using fragments, there is only one activity for both the WelcomeFragment and the GorillaFragment. We don't want to start loading the video until we switch to the GorillaFragment. To do this, overload setUserVisibleHint() in the GorillaFragment. This method will be called when the visibility of the fragment changes.

When the fragment is visible, check to see if the video is loaded, and if not, start loading. Otherwise, just pause the playback since the video is no longer visible.

Add this method to end of the class:

@Override
public void setUserVisibleHint(boolean isVisibleToUser) {
    super.setUserVisibleHint(isVisibleToUser);

    if (isVisibleToUser) {
        try {
            if (videoWidgetView.getDuration() <= 0) {
                videoWidgetView.loadVideoFromAsset("congo_2048.mp4",
                new VrVideoView.Options());
            }
        } catch (Exception e) {
            Toast.makeText(getActivity(), "Error opening video: " + e.getMessage(), Toast.LENGTH_LONG)
                        .show();
        }
    } else {
        isPaused = true;
        if (videoWidgetView != null) {
            videoWidgetView.pauseVideo();
        }
    }
}

Save all the files and press Run. The application should start on the welcome tab, and you can look around in the VrPanoramaView that was added about codelabs. Swipe left (or tap About Gorillas) to see the video.

Congratulations! You can now add VR view to all of your applications to provide a more rich and immersive experience.

Now that you've finished the code lab, there are a few more things you can do before you are ready to publish your awesome game. This section will describe what is left and point you to some resources where you can continue learning.

.