Unity MR Part 7: Raycasts

👀 Stumbled here on accident? Start with the introduction!

📚 This article will guide you through the concept of Raycasts in MR, explaining their significance and functionality. Additionally, it will provide a comprehensive tutorial on implementing Raycasts within the Unity platform.

ℹ️ If you find yourself facing any difficulties, remember that you can always refer to or download the code from our accompanying GitHub repository

The aim of this article is to conduct hit tests on planes we created previously, using our controller. We'll then, in the next article, place an Anchor at the identified position. This process is fundamental for accurately positioning virtual objects in relation to the real world.

ℹ️ Raycasts in Unity and MR are used to determine if and where a virtual ray intersects with objects in the game world or MR environment. They're crucial for interaction, allowing the detection of objects, surfaces, or points in 3D space. In MR, Raycasts can be used for selecting or interacting with virtual objects overlaid on the real world.

Creating a custom Reticle

ℹ️ In AR, a reticle is a visual indicator, often a small pointer or dot, used to help users accurately target and interact with virtual elements overlaid onto the real world. It provides a focal point for interaction, indicating where actions like tapping, selecting, or placing virtual objects will occur. Reticles are especially useful in AR experiences to improve precision and user experience in environments where direct touch interactions are not possible.

To create a custom Reticle, we'll utilize the asset provided in the link. Download the file directly into your Assets folder. For the purposes of this article, we'll organize it in a subfolder named 3DModels. Alternatively, you can download the asset to a different folder and then drag and drop it into the 3DModels folder within Unity.

⬇️ arrow.fbx

Let's begin by creating a visual indicator, a Reticle, which will appear when we hover over any of the planes managed by the AR Plane Manager.

1. First, create a new empty GameObject via Create → Empty in your hierarchy and name it 'Reticle'.
2. Next, drag and drop this GameObject into your Prefabs folder, then remove the Reticle GameObject from the hierarchy.
3. Double-click the prefab Reticle you just created to start editing it.
4. Make sure the Reticle GameObject's position is set to X: 0, Y: 0, Z: 0.
5. Then, drag and drop the .fbx file from your 3DModels folder onto the Reticle GameObject in your hierarchy view.
6. Make sure the arrow GameObject's position is set to X: 0, Y: 0, Z: 0.
7. Finally, adjust the position, rotation, and scale of the Cone and Cylinder to suit your needs.

With the specified values, you will achieve the Prefab configuration as shown in the upcoming screenshot. If you are designing a custom Reticle, remember to modify only the position, rotation, and scale of the Mesh components. It's important to avoid altering these values for the Reticle GameObject itself to maintain its intended functionality and alignment.

Updating our AR Default Plane

We aim for the Reticle to be visible only when hovering over one of our AR Default Planes. To achieve this, modify the AR Default Plane Prefab (by double clicking the AR Default Plane in the Assets/Prefabs folder), add the XR Simple Interactable Script and select the newly created Reticle Prefab as the Custom Reticle.

This Prefab will now be instantiated at the position where our ray intersects with any of the planes registered in our AR Plane Manager. When you run your app with these settings, you'll observe that the custom Reticle is now accurately indicating the plane being hovered over.

Improve the appearance of the XR Interactor Line Visual

In this step, we'll adjust the color gradients for Valid, Invalid, and Blocked states in the XR Interactor Line Visual. This modification will change the line's color based on interaction: it will display one color when hovering over a valid target (in our scenario, detected planes) and another color when no valid target is hovered over (anything other than planes). This visual cue enhances the user's understanding of interaction possibilities within the AR environment.

You can use the following color gradients as seen in the upcoming screenshots or use your own custom gradients.

The Valid color gradients with a starting color of #00A0FF.

The Invalid color gradients with a starting color of #FFFFFF.

The Blocked color gradients with a starting color of #FFEB04.

Ensure that you also apply the same values to the Right Controller.

That concludes the adjustments to the color gradients of the XR Interactor Line Visual. We'll examine the outcome of these changes at the end of this article, providing a comprehensive view of how these modifications enhance the visual feedback in our application.

Improve appearance of the Reticle

For the final visual enhancement, let's create a new Material for the custom Reticle. Go ahead and make a new Material in your Assets/Materials folder and name it ‘Reticle’. You can either use the following suggested values or select your own based on your specific needs. This step will ensure that the custom Reticle has a distinctive and appropriate appearance in your application.

Edit the Reticle Prefab by double-clicking it in Assets/Prefabs. Now, drag and drop the new Material onto both meshes within the Reticle Prefab (Cone and Cylinder). Once you've done this, your Reticle Prefab should resemble the appearance shown in the upcoming screenshot.

Reacting to a OnButtonPressed Event and check if the Ray hits a Plane

For the final step in this article we dive a bit into code. Create a Script named MRArticleSeriesController.cs in your Assets/Scripts folder and use the following code.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.InputSystem;
using UnityEngine.XR.Interaction.Toolkit;

namespace Taikonauten.Unity.ArticleSeries
{
    public class MRArticleSeriesController : MonoBehaviour
    {
        public ActionBasedController controller;
        public InputActionReference buttonAction;
        public XRRayInteractor rayInteractor;

        void OnEnable()
        {
            Debug.Log("MRArticleSeriesController -> OnEnable()");
            buttonAction.action.performed += OnButtonPressed;
        }

        void OnDisable()
        {
            Debug.Log("MRArticleSeriesController -> OnDisable()");
            buttonAction.action.performed -= OnButtonPressed;
        }

        private void OnButtonPressed(InputAction.CallbackContext context)
        {
            Debug.Log("MRArticleSeriesController -> OnButtonPressed()");

            if (rayInteractor.TryGetCurrent3DRaycastHit(out RaycastHit hit))
            {
                Debug.Log(hit);
            }
        }
    }
}

Drag and drop the script onto your XR Origin (XR Rig). Then, as shown in the upcoming screenshot, select an ActionBasedController, InputActionReference, and XRRayInteractor. This step integrates the script with the XR Rig, ensuring that the controller, input actions, and ray interactions are properly configured.

Let's proceed to build and deploy the app onto our headset to see the results. Navigate to File → Build Settings and, as depicted in the upcoming screenshot, enable the Development Build option under the Android section.

To ensure proper functionality, it's necessary to disable Vulkan in the Graphics APIs in the Player section. Navigate to Edit -> Project Settings, and as illustrated in the following screenshot, remove Vulkan from the list of enabled Graphics APIs. This adjustment is crucial for compatibility.

As of Unity version 2023.2.3f and Unity OpenXR: Meta 1.0.1, creating a development build with Vulkan enabled leads to crashes. This known issue highlights the importance of avoiding Vulkan in your development builds with these specific versions to ensure stability and prevent unexpected crashes.

Select Build and Run to test the application. Since we have configured only the Right Controller, which suffices for our purposes, you should observe the following console output when the Trigger button is pressed.

To view the output from the build on your headset, you need to choose the connected Oculus Quest in your Console window.

As demonstrated in the upcoming screenshot, the Debug.Log statement generates the expected output. This visual confirmation in the screenshot effectively showcases that the application is functioning as intended, aligning with our anticipated results.

1. AndroidPlayer "Oculus_Quest" Player -> OnButtonPressed()
2. AndroidPlayer "Oculus_Quest" UnityEngine.RaycastHit

Next article

In our next article, we'll be focusing on an essential aspect of AR development: Planes and the AR Plane Manager in Unity. Planes are fundamental to understanding and interacting with the real world in AR applications. We will explore how the AR Plane Manager detects and manages these planes, allowing for the placement of virtual objects on flat surfaces like floors, tables, or walls.