ARCore is a platform for building augmented reality apps on Android. ARCore uses three key technologies to integrate virtual content with the real world as seen through your phone's camera:

This codelab guides you through building a simple demo game to introduce these capabilities so you can use them in your own applications.

What you'll learn

Prerequisites

Make sure you have these before starting the codelab:

Unity Game Engine

ARCore SDK for Unity

Other items

More information about getting started can be found at: developers.google.com/ar/develop/unity/getting-started

Now that you have everything you need, let's start!

Create a new Unity 3D project and change the target platform to Android (under File > Build Settings).

Select Android and click Switch Platform.

Then click Player Settings... to configure the Android specific player settings.

Change player settings for ARCore

  • In Other Settings, disable Multithreaded rendering
  • Set the Package name to a unique name e.g. com.<yourname>.arcodelab
  • Set the Minimum API level to 7.0 (Nougat) API level 24 or higher

  • In XR Settings section at the bottom of the list, enable ARCore Supported

Add the codelab assets

Import arcore-intro.unitypackage into your project. (If you haven't already done so, check the Overview step for a list of prerequisites you need to download). This contains prefabs and scripts that will expedite the parts of the codelab so you can focus on how to use ARCore.

Add the ARCore SDK

Use Assets > Import package > Custom package to import arcore-unity-sdk-v1.2.0.unitypackage which you downloaded in the Prerequisites section into your project.

Add the required scene elements

  • Add the Assets/GoogleARCore/Prefabs/ARCore Device prefab to the root of your scene. Make sure its position is set to (0,0,0).
  • Delete the default Main Camera game object. We'll use the First Person Camera from the ARCore Device prefab instead.
  • Delete the default Directional light game object.
  • Add the Assets/GoogleARCore/Prefabs/Environmental Light prefab to the root of your scene.
  • Add the built-in EventSystem (from the menu: GameObject>UI>EventSystem).

Now you have a scene setup for using ARCore. Next, let's add some code!

The scene controller is used to coordinate between ARCore and Unity. Create an empty game object and change the name to SceneController. Add a C# script component to the object also named SceneController.

Add ARCore operational checks

The default configuration for ARCore applications is AR Required. This means to run this application, the user's device must support ARCore and ARCore services must be installed. The "ARCore Device" prefab handles the AR Required checks and will start the process of downloading ARCore services automatically. If you want to make an AR Optional application, you can see more information on the Google Developer website.

Open the script. We need to check for a variety of error conditions. These conditions are also checked by the HelloARExample controller sample script in the SDK.

First add the using statement to resolve the class name from the ARCore SDK. This will make auto-complete recognize the ARCore classes and methods used.

using GoogleARCore;

Create a new method void QuitOnConnectionErrors() {} and add it to the the SceneController script. This method checks the state of the ARCore Session to make sure ARCore is working in our app:

  1. Is the permission to use the camera granted? ARCore uses the camera to sense the real world. The user is prompted to grant this permission the first time the application is run. This check is done by ARCore, so you don't have to write any code to check the permission yourself.
  2. Can the ARCore library connect to the ARCore Services? ARCore relies on AR Services which runs on the device in a separate process.
void QuitOnConnectionErrors()
{
    if (Session.Status ==  SessionStatus.ErrorPermissionNotGranted)
    {
        StartCoroutine(CodelabUtils.ToastAndExit(
            "Camera permission is needed to run this application.", 5));
    }
    else if (Session.Status.IsError())
    {
        // This covers a variety of errors.  See reference for details
        // https://developers.google.com/ar/reference/unity/namespace/GoogleARCore
        StartCoroutine(CodelabUtils.ToastAndExit(
            "ARCore encountered a problem connecting. Please restart the app.", 5));
    }
}

Now add a call to QuitOnConnectionErrors() in the Start() method.

  void Start ()
  {
    QuitOnConnectionErrors ();
  }

Check the ARCore tracking state

ARCore needs to capture and process enough information to start tracking the user's movements in the real world. Once ARCore is tracking, the Frame object is used to interact with ARCore. Add this check to the Update() method. At the same time, adjust the screen timeout so it stays on if we are tracking.

void Update() 
{
    // The session status must be Tracking in order to access the Frame.
    if (Session.Status != SessionStatus.Tracking)
    {
        int lostTrackingSleepTimeout = 15;
        Screen.sleepTimeout = lostTrackingSleepTimeout;
        return;
    }
    Screen.sleepTimeout = SleepTimeout.NeverSleep;
}

Great! Now we have the minimum amount of code to start using ARCore and make sure it works. Next, let's try it out!

Save the scene with the name "ARScene" and add it to the list of scenes when building.

Build and run the sample app. If everything is working, you should be prompted for permission to take pictures and record video, after which you'll start seeing a preview of the camera image. Once you see the preview image, you're ready to use ARCore!

If there is an error, you'll want to resolve it before continuing with the codelab.

Unity's scaling system is designed so that when working with the Physics engine, 1 unit of distance can be thought of as 1 meter in the real world. ARCore is designed with this assumption in mind. We use this scaling system to scale virtual objects so they look reasonable in the real world.

For example, an object placed on a desktop should be small enough to be on the desktop. A reasonable starting point would be ½ foot (15.24 cm), so the scale should (0.1524, 0.1524, 0.1524).

This might not look the best in your application, but it tends to be a good starting point and then you can fine tune the scale further for your specific scene.

As a convenience, the prefabs used in this codelab contain a component named GlobalScalable which supports using stretch and pinch to size the objects. To enable this, the touch input needs to be captured.

Add GlobalScalable support to the Scene Controller

Select the SceneController object in the hierarchy and add the Script component GlobalScalable.

In the properties for the component, enable "Handle Scale Input" and disable "Adjust Scale." The scale of the scene controller should not be adjusted since it is the parent of the ARCore detected planes.

Now when running the application, the user can pinch or stretch the objects to fit the scene more appropriately.

Next, let's detect and display the planes that are detected by ARCore.

ARCore uses a class named DetactedPlane to represent detected planes. This class is not a game object, so we need to make a prefab that will render the detected planes. Good news since ARCore 1.2 there's already such a prefab in the ARCore Unity SDK, it is Assets/GoogleARCore/Examples/Common/Prefabs/DetectedPlaneVisualizer.

ARCore detects horizontal and vertical planes. We'll use these planes in the game. For each newly detected plane, we'll create a game object that renders the plane using the DetectedPlaneVisualizer prefab. You may have guessed it, since ARCore 1.2 there's a convenient script in ARCore Unity SDK, Assets/GoogleArCore/Examples/Common/Scripts/DetectedPlaneGenerator.cs that just does this.

Adding plane generator and plane visualizer

Let's add the component DetectedPlaneGenerator onto SceneController object. Select SceneController object, in property Inspector, click Add Component button, and type-in DetectedPlaneGenerator. Then set the value of Detected Plane Prefab to the prefab Assets/GoogleARCore/Examples/Common/Prefabs/DetectedPlaneVisualizer.

Save and Run

Now save the scene & project and run the app.

As you look at the room, ARCore will detect planes and they should appear as different color grids. As ARCore detects more about the scene, the plane will change shape and merge with, or subsume other planes.

Depending on the physical characteristics of the environment, it might take a couple seconds for the first plane to be detected. Once a plane is detected, it will be rendered using a random color. Plane detection can be improved with good lighting and some sort of pattern or texture (like wood grain, or a rug design).

In the SceneController script, add a member variable for the first person camera.

We'll be using the first person camera in this method, so add a public variable and set it to the first person camera.

public Camera firstPersonCamera;

Save the script, switch to the scene editor, and set this property to ARCore Device/First Person Camera from the scene hierarchy.

To process the touches, we get a single touch and raycast it using the ARCore session to check if the user tapped on a plane. If so, we'll use that one to display the rest of the objects.

In SceneController script, create a new method named ProcessTouches(). This method will perform the ray casting hit test and select the plane that is tapped.

void ProcessTouches ()
{
    Touch touch;
    if (Input.touchCount != 1 ||
        (touch = Input.GetTouch (0)).phase != TouchPhase.Began)
    {
        return;
    } 

    TrackableHit hit;
    TrackableHitFlags raycastFilter =
        TrackableHitFlags.PlaneWithinBounds |
        TrackableHitFlags.PlaneWithinPolygon;

    if (Frame.Raycast (touch.position.x, touch.position.y, raycastFilter, out hit))
    {
        SetSelectedPlane (hit.Trackable as DetectedPlane);
    }
}

Still in SceneController script, create the new method SetSelectedPlane(). This is used to notify all the other controllers that a new plane has been selected. Right now it just logs we selected a plane.

void SetSelectedPlane (DetectedPlane selectedPlane)
{
    Debug.Log ("Selected plane centered at " + selectedPlane.CenterPose.position);
}

The last step is to call ProcessTouches() from Update. Add this code to the end of the Update() method:

// Add to the end of Update()
ProcessTouches();

In ARCore, objects that maintain a constant position as you move around are positioned by using an Anchor. Let's create an Anchor to hold a floating scoreboard.

Create the Game Object

To the Scoreboard object:

Write the Scoreboard controller script

In order to position the scoreboard, we'll need to know where the user is looking. So we'll add a public variable for the first person camera.

The scoreboard will also be "anchored" to the ARScene. An anchor is an object that holds it position and rotation as ARCore processes the sensor and camera data to build the model of the world.

To keep the anchor consistent with the plane, we'll keep track of the plane and make sure the distance in the Y axis is constant.

Also add a member to keep track of the score.

public Camera firstPersonCamera;
private Anchor anchor;
private DetectedPlane detectedPlane;
private float yOffset;
private int score;

Just as in the previous step, save the script, switch to the scene editor, and set "First Person Camera" property to ARCore Device/First Person Camera from the scene hierarchy.

We'll place the scoreboard above the selected plane. This way it will be visible and indicate which plane we're focused on.

Hide until Anchored

We want the scoreboard hidden until it is anchored in position. We'll do this by disabling all the mesh renderers, then once anchored, enable them.

In ScoreboardController script, in the Start() method, add the code to disable the mesh renderers.

void Start ()
{
    foreach (Renderer r in GetComponentsInChildren<Renderer>())
    {
        r.enabled = false;
    }
}

Create the function SetSelectedPlane()

This is called from the scene controller when the user taps a plane. When this happens, we'll create the anchor for the scoreboard

// in ScoreboardController.cs
public void SetSelectedPlane(DetectedPlane detectedPlane)
{
    this.detectedPlane = detectedPlane;
    CreateAnchor();
}

Create the function CreateAnchor()

The CreateAnchor method does 5 things:

  1. Raycast a screen point through the first person camera to find a position to place the scoreboard.
  2. Create an ARCore Anchor at that position. This anchor will move as ARCore builds a model of the real world in order to keep it in the same location relative to the ARCore device.
  3. Attach the scoreboard prefab to the anchor as a child object so it is displayed correctly.
  4. Record the yOffset from the plane. This will be used to keep the score the same height relative to the plane as the plane position is refined.
  5. Enable the renderers so the scoreboard is drawn.
// in ScoreboardController.cs
void CreateAnchor()
{
    // Create the position of the anchor by raycasting a point towards
    // the top of the screen.
    Vector2 pos = new Vector2 (Screen.width * .5f, Screen.height * .90f);
    Ray ray = firstPersonCamera.ScreenPointToRay (pos);
    Vector3 anchorPosition = ray.GetPoint (5f);

    // Create the anchor at that point.
    if (anchor != null) {
      DestroyObject (anchor);
    }
    anchor = detectedPlane.CreateAnchor (
        new Pose (anchorPosition, Quaternion.identity));

    // Attach the scoreboard to the anchor.
    transform.position = anchorPosition;
    transform.SetParent (anchor.transform);

    // Record the y offset from the plane.
    yOffset = transform.position.y - detectedPlane.CenterPose.position.y;

    // Finally, enable the renderers.
    foreach (Renderer r in GetComponentsInChildren<Renderer>())
    {
        r.enabled = true;
    }
}

Add code to ScoreboardController.Update()

First check for tracking to be active.

// The tracking state must be FrameTrackingState.Tracking
// in order to access the Frame.
if (Session.Status != SessionStatus.Tracking)
{
    return;
}

Check that there is a selected plane, and update it if it was subsumed by another plane.

// If there is no plane, then return
if (detectedPlane == null)
{
    return;
}

// Check for the plane being subsumed.
// If the plane has been subsumed switch attachment to the subsuming plane.
while (detectedPlane.SubsumedBy != null)
{
    detectedPlane = detectedPlane.SubsumedBy;
}

The last thing to add is to rotate the scoreboard towards the user as they move around in the real world and adjust the offset relative to the plane.

// Make the scoreboard face the viewer.
transform.LookAt (firstPersonCamera.transform); 

// Move the position to stay consistent with the plane.
transform.position = new Vector3(transform.position.x,
            detectedPlane.CenterPose.position.y + yOffset, transform.position.z);

Call SetSelectedPlane() from the scene controller

Switch back to the SceneController script and add a member variable for the ScoreboardController.

public ScoreboardController scoreboard; 

Save the script, switch back to the scene editor and set this property to the scoreboard object.

Back in SceneController, find the SetSelectedPlane() method we added earlier, and pass the selected plane to the ScoreboardController.

// Add to the end of SetSelectedPlane.
scoreboard.SetSelectedPlane(selectedPlane);

Save the scripts and the scene. Build and run the app! Now it should display planes as they are detected, and if you tap one, you'll see the scoreboard!

Now that we have a plane, let's put a snake on it and move it around on the plane.

Create a new Empty Game object named Snake.

Add the existing C# script (Assets/Codelab/Scripts/Slithering.cs). This controls the movement of the snake as it grows. In the interest of time, we'll just add it, but feel free to review the code later on.

Add a new C# script to the Snake named SnakeController.

In SnakeController.cs, we need to track the plane that the snake is traveling on. We'll also add member variables for the prefab for the head, and the instance:

private DetectedPlane detectedPlane;
  
public GameObject snakeHeadPrefab;
private GameObject snakeInstance;

Set the prefabs

Back in the editor, set the prefab values. For the snakeHeadPrefab, use Assets/Codelab/Prefabs/SnakeHeadPrefab. For the snakeBody in the Slithering component, use Assets/Codelab/Prefabs/SnakeBodyPrefab.

Create the SetPlane() method

In SnakeController script, add a method to set the plane. When the plane is set, spawn a new snake.

public void SetPlane (DetectedPlane plane)
{
    detectedPlane = plane;
    // Spawn a new snake.
    SpawnSnake();
}

Then spawn the snake.

void SpawnSnake ()
{
    if (snakeInstance != null)
    {
        DestroyImmediate (snakeInstance);
    }

    Vector3 pos = detectedPlane.CenterPose.position;

    // Not anchored, it is rigidbody that is influenced by the physics engine.
    snakeInstance = Instantiate (snakeHeadPrefab, pos,
            Quaternion.identity, transform);

    // Pass the head to the slithering component to make movement work.
    GetComponent<Slithering> ().Head = snakeInstance.transform;
}

Now add a member variable to the SceneController.cs to reference the Snake.

public SnakeController snakeController;

Save the script, switch to the scene editor and assign the Snake object to snakeController in the scene inspector.

In SceneController.SetSelectedPlane(), pass the selected plane to the snake controller

// Add to SetSelectedPlane()
snakeController.SetPlane(selectedPlane);

To move the snake, we'll use where we are looking as a point that the snake should move towards. To do this, we'll raycast the center of the screen through the ARCore session to a point on a plane.

First let's add a game object that we'll use to visualize where the user is looking at.

Edit the SnakeController and add member variables for the pointer and the first person camera. Also add a speed member variable.

public GameObject pointer;
public Camera firstPersonCamera;
// Speed to move.
public float speed = 20f;

Set the game object properties

Save the script and switch to the scene editor.

Add an instance of the Assets/CodelabPrefabs/gazePointer to the scene.

Then, select Snake object, in Inspector view, set the pointer property to the instance of the gazePointer, and the firstPersonCamera to the ARCore device's first person camera.

Update the pointer state

In SnakeController.Update() add a check for the snake being active. If it is not, just return; there is nothing to do.

if (snakeInstance == null || snakeInstance.activeSelf == false) 
{
    pointer.SetActive(false);
    return;
}
else
{
    pointer.SetActive(true);
}

Raycast the center of the screen

Use the ARCore Session to raycast from the center of the screen and if there is a hit, use that point, but average the point on the plane's y position with the snake head's so the pointer is between the plane and the head. Add below code to the end of SnakeController.Update().

  
    TrackableHit hit;
    TrackableHitFlags raycastFilter = TrackableHitFlags.PlaneWithinBounds;    

    if (Frame.Raycast (Screen.width/2, Screen.height/2, raycastFilter, out hit))
    {
      Vector3 pt = hit.Pose.position;
      //Set the Y to the Y of the snakeInstance
      pt.y = snakeInstance.transform.position.y;
      // Set the y position relative to the plane and attach the pointer to the plane
      Vector3 pos = pointer.transform.position;
      pos.y = pt.y;
      pointer.transform.position = pos; 

      // Now lerp to the position                                         
      pointer.transform.position = Vector3.Lerp (pointer.transform.position, pt,
        Time.smoothDeltaTime * speed);
    }

Move towards the pointer

Once the snake is heading in the right direction, move towards it. We want to stop before the snake is at the same spot to avoid a weird nose spin. Add below code to the end of SnakeController.Update().

    // Move towards the pointer, slow down if very close.                                                                                     
    float dist = Vector3.Distance (pointer.transform.position,
        snakeInstance.transform.position) - 0.05f;
    if (dist < 0)
    {
      dist = 0;
    }

    Rigidbody rb = snakeInstance.GetComponent<Rigidbody> ();
    rb.transform.LookAt (pointer.transform.position);
    rb.velocity = snakeInstance.transform.localScale.x *
        snakeInstance.transform.forward * dist / .01f;

Save the scripts and the scene. Build and run the app! Tap a plane and the snake will appear and follow the pointer around on the plane as you look around. (Remember, you can stretch & pinch to scale the snake if needed).

Now we want to put a tasty bit of food on the plane. This involves creating an object, placing it on a plane, then removing it after some time.

Select the SceneController object, and add a new C# script named FoodController and begin editing the script.

First, add member variables to reference the plane selected.

  1. The DetectedPlane instance to place the food on.
  2. The instance of the food object.
  3. The age of the object in seconds.
  4. The max age of the food (All food has an expiration date).
  5. An array of prefabs to use to create food instances.
private DetectedPlane detectedPlane;
private GameObject foodInstance;
private float foodAge;
private readonly float maxAge = 10f;

public GameObject[] foodModels;

Initialize the Food Prefabs

First, make sure to initialize the foodModels array in the editor, by adding prefabs in Assets/Codelab/Prefab/Foods. You need to at least add 1, but a varied diet is much more enjoyable!

Add the food tag

Add a tag in the editor by dropping down the tag selector in the object inspector, and select "Add Tag". Add a tag named "food". We'll use this tag to identify food objects during collision detection.

Add the SetSelectedPlane method

In FoodController script, create a public method named SetSelectedPlane() that will be called by the SceneController when a plane is selected.

public void SetSelectedPlane(DetectedPlane selectedPlane)
{
    detectedPlane = selectedPlane;
}

Manage the plane state

The plane will change state, size, and position as ARCore interprets the input from the sensors and camera. Since we're holding on to the plane, we need to handle these changes.

In the FoodController.Update() method:

if (detectedPlane == null)
{
    return;
}

if (detectedPlane.TrackingState != TrackingState.Tracking)
{
    return;
}
// Check for the plane being subsumed
// If the plane has been subsumed switch attachment to the subsuming plane.
while (detectedPlane.SubsumedBy != null)
{
    detectedPlane = detectedPlane.SubsumedBy;
}
if (foodInstance == null || foodInstance.activeSelf == false)
{
    SpawnFoodInstance();
    return;
}
foodAge += Time.deltaTime;
if (foodAge >= maxAge)
{
    DestroyObject(foodInstance);
    foodInstance = null;
}

Implement SpawnFoodInstance()

Spawning a new food item has several steps:

void SpawnFoodInstance ()
{
    GameObject foodItem = foodModels [Random.Range (0, foodModels.Length)];
    
    // Pick a location.  This is done by selecting a vertex at random and then
    // a random point between it and the center of the plane.
    List<Vector3> vertices = new List<Vector3> ();
    detectedPlane.GetBoundaryPolygon (vertices);
    Vector3 pt = vertices [Random.Range (0, vertices.Count)];
    float dist = Random.Range (0.05f, 1f);
    Vector3 position = Vector3.Lerp (pt, detectedPlane.CenterPose.position, dist);
    // Move the object above the plane.
    position.y += .05f;


    Anchor anchor = detectedPlane.CreateAnchor (new Pose (position, Quaternion.identity));

    foodInstance = Instantiate (foodItem, position, Quaternion.identity,
             anchor.transform);

    // Set the tag.
    foodInstance.tag = "food";
    
    foodInstance.transform.localScale = new Vector3 (.025f, .025f, .025f);
    foodInstance.transform.SetParent (anchor.transform);
    foodAge = 0;

    foodInstance.AddComponent<FoodMotion> ();
}

Set the plane from the SceneController

Add the call to SetSelectedPlane()in the SceneController.SetSelectedPlane():

// Add to the bottom of SetSelectedPlane() 
GetComponent<FoodController>().SetSelectedPlane(selectedPlane);

Now we are moving, add the test for colliding with food, eat it and grow the snake.

Create a new C# script named FoodConsumer. To do so, right click on Assets in Project window, select Create->C# Script, and rename it to FoodConsumer.

We don't want to pollute our awesome snake head prefab with the FoodConsumer, so let's add

It to the instance when we spawn.

In SnakeController.SpawnSnake(), add the component to the new instance.

 // After instantiating a new snake instance, add the FoodConsumer component.
 snakeInstance.AddComponent<FoodConsumer>();

In FoodConsumer(), add the OnCollisionEnter method (you can delete the boilerplate Start() and Update() methods).

    void OnCollisionEnter(Collision collision)
    {
        if (collision.gameObject.tag == "food")
        {
            collision.gameObject.SetActive(false);
            Slithering s = GetComponentInParent<Slithering>();

            if (s != null)
            {
                s.AddBodyPart();
            } 
        }
    }

Remember that Scoreboard from the beginning of the codelab? Well, now it is time to actually use it!

In SceneController.Update(), set the score to the length of the snake:

 scoreboard.SetScore(snakeController.GetLength());

Now we need to implement SetScore in ScoreboardController:

public void SetScore(int score)
{
    if (this.score != score)
    {
        GetComponentInChildren<TextMesh>().text = "Score: " + score;
        this.score = score;
    }
}

Add GetLength() in the SnakeController

public int GetLength()
{
    return GetComponent<Slithering>().GetLength();
}

Well done working through this codelab! A quick re-cap of what was covered:

Other Resources

As you continue your ARCore exploration. Check out these other resources: