Creating a Gaze Based UI for Gear VR in Unity

In the last tutorial, we set up Unity for Gear VR development. In this we’ll learn how to create a gaze based UI for Gear VR. We’ll see the following in detail:

• Set up a Camera to work with our UI.
• Create our World Space UI (More on that later).
• Create an Event System to work with Gaze Based Input.
• Capture Events from the UI using Scripts.

Let’s first talk about how the UI system in Unity first. If you are familiar with it then you can skip this part.

Unity’s UI System

Unity’s UI system consists of these main components:

• EventSystem
• Input Modules
• RayCasters
• Components

Unity Event System

The Unity Event System is a way of sending events to objects in the application based on input, be it keyboard, mouse, touch, or custom input. The Event System consists of a few components that work together to send events. When you add an Event System component to a GameObject you will notice that it does not have much functionality exposed. This is because the Event System itself is designed as a manager and facilitator of communication between Event System modules.

The primary roles of the Unity Event System are as follows:

1. Manage which GameObject is selected
2. To manage which Input Module is in use
3. Manage Raycasting (if required)
4. Updating all Input Modules as required

Input Modules

Input module contains the main logic of how the Event System should behave. They are used for

1. Handling Input
2. Managing event state
3. Sending events to scene objects.

Only one Input Module can be active in the Event System at a time, and they must be components on the same GameObject as the Event System component.

If you wish to write a custom input module, it is recommended that you send events supported by existing UI components in Unity, but you are also able to extend and write your own events as detailed in the Messaging System documentation.

Raycasters

Raycasters are used for figuring out what the pointer is over. It is common for Input Modules to use the Raycasters configured in the scene to calculate what the pointing device is over.

There are 3 provided Raycasters that exist by default:

1. Graphic Raycaster – Used for UI elements
2. Physics 2D Raycaster – Used for 2D physics elements
3. Physics Raycaster – Used for 3D physics elements

This default setup won’t really work in VR because there’s no actual surface for the mouse pointer to move across. So we create a UI that exists in 3D space and use a Gaze Pointer that is always in the centre of the user viewport which is controlled by the Head Movement (using Head Tracking). This is done by casting a ray from between the eyes of the viewport into the world which is then intercepted by the UI or physical objects in our world space as well.

Oculus includes these input modules which modify the UI system to work with rays rather than actual screen coordinates.

Now that we have a basic understanding on how the Unity’s UI system works, Let’s get started on getting it all to work.

Creating a Gaze Based UI

1. Setting up our GearVR camera

First of all let’s create a new scene by Going to File > New Scene. In the new scene we should already have a Main camera set up for us by Unity but we don’t need that so we can delete it.

We already imported the Oculus Utilities, if not you can do so by following the previous tutorial.

Go to the Project view > OVR > Prefabs > OVRCameraRig prefab and drag it into the Scene.

If you expand the OVRCameraRig GameObject you will find a few cameras used for VR Rendering. The “CenterEyeAnchor” GameObject is the MainCamera in this case.

1. Creating a World space UI

Now let’s create a Canvas for UI. In the Hierarchy, Create > UI > Canvas. An Event System is automatically added to the Hierarchy which is responsible for capturing all the events from the Input Devices; We will get to it later.

Since this UI is generally used for traditional applications is in 2D space, the same approach will not work for VR Applications. Hence, we need the Canvas to be in a 3D space. We change that by changing the Render Mode of the Canvas component of our canvas to World Space.

Set the CentreEyeAnchor as the Event Camera for the Canvas.

That’s all we need for now with the Canvas.

1. Creating an Event System with Gaze Base Input

The Event system default is a standalone Input Module which obviously won’t work in VR.

We will need a few Classes from the Oculus Sample Framework which can be downloaded from here.

Once that is downloaded then open the Unity project and copy the OVRInspector folder into our project directory or just export that specific folder with their dependencies into a unitypackage which you can import into any of your Unity projects

Once we have imported it into our Project. We can delete the Standalone input module from the Event system and replace it with the OVRInputModule by doing

AddComponent > Search > OVRInputModule.

Change its Ray Transform to our CentreEyeAnchor by dragging and dropping on it.

Now remove the Graphics Raycaster from the Canvas and replace it with the  OVR Raycaster. The last step is to drag and drop the GazePointerRing.prefab from OVRInspector/Resources/Prefabs into our scene.

That’s it! We have our project set up for using gaze based UI in Gear VR.