The Android XR SDK is now available in Developer Preview. We want your feedback! Visit our support page to reach out.

XR_ANDROID_eye_tracking OpenXR extension

Name String

XR_ANDROID_eye_tracking

Extension Type

Instance extension

Registered Extension Number

457

Revision

Extension and Version Dependencies

OpenXR 1.0

Last Modified Date

2025-01-17

IP Status

No known IP claims.

Contributors

Spencer Quin, Google

Jared Finder, Google

Levana Chen, Google

Kenny Vercaemer, Google

Prasanthi Gurumurthy, Google

Nihav Jain, Google

Overview

This extension enables applications to obtain position and orientation of the user's eyes, as well as eye tracking status.

Eye tracking data is provided in two modes: coarse and fine. Coarse tracking provides a coarse estimate of the user's eyes, while fine tracking provides a more accurate estimate. Coarse tracking is meant for applications that want to provide a basic avatar-like representation, while fine tracking is meant for more precise applications.

For interaction, XR_EXT_eye_gaze_interaction should be used.

Inspect system capability

The XrSystemEyeTrackingPropertiesANDROID structure is defined as:

typedef struct XrSystemEyeTrackingPropertiesANDROID {
    XrStructureType    type;
    void*              next;
    XrBool32           supportsEyeTracking;
} XrSystemEyeTrackingPropertiesANDROID;

Member Descriptions

type is the XrStructureType of this structure.
next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
supportsEyeTracking is an XrBool32, indicating if the current system supports eye tracking.

An application can inspect whether the system is capable of eye tracking by chaining an XrSystemEyeTrackingPropertiesANDROID structure to the XrSystemProperties when calling xrGetSystemProperties. If supportsEyeTracking returns XR_FALSE, then an application will receive XR_ERROR_FEATURE_UNSUPPORTED from xrCreateEyeTrackerANDROID.

Valid Usage (Implicit)

The XR_ANDROID_eye_tracking extension must be enabled prior to using XrSystemEyeTrackingPropertiesANDROID
type must be XR_TYPE_SYSTEM_EYE_TRACKING_PROPERTIES_ANDROID
next must be NULL or a valid pointer to the next structure in a structure chain

Create an eye tracker handle

XR_DEFINE_HANDLE(XrEyeTrackerANDROID)

The XrEyeTrackerANDROID handle represents an eye tracker for tracking eyes and accurately maps what the user is looking at.

Eye tracking data can be sensitive personal information and is closely linked to personal privacy and integrity. It is strongly recommended that applications that store or transfer eye tracking data always ask the user for active and specific acceptance to do so.

This handle can be used to access eye tracking data using other functions in this extension.

Eye tracking provides eye pose and status representation in the scene.

The xrCreateEyeTrackerANDROID function is defined as:

XrResult xrCreateEyeTrackerANDROID(
    XrSession                                   session,
    const XrEyeTrackerCreateInfoANDROID*        createInfo,
    XrEyeTrackerANDROID*                        eyeTracker);

Parameter Descriptions

session is an XrSession handle in which the eye tracking will be active.
createInfo is the XrEyeTrackerCreateInfoANDROID used to specify the eye tracking.
eyeTracker is the returned XrEyeTrackerANDROID handle.

An application can create an XrEyeTrackerANDROID handle using xrCreateEyeTrackerANDROID function.

If the system does not support eye tracking, then XR_ERROR_FEATURE_UNSUPPORTED will be returned from xrCreateEyeTrackerANDROID.

Valid Usage (Implicit)

The XR_ANDROID_eye_tracking extension must be enabled prior to calling xrCreateEyeTrackerANDROID
session must be a valid XrSession handle
createInfo must be a pointer to a valid XrEyeTrackerCreateInfoANDROID structure
eyeTracker must be a pointer to an XrEyeTrackerANDROID handle

Return Codes

Success

XR_SUCCESS
XR_SESSION_LOSS_PENDING

Failure

XR_ERROR_FUNCTION_UNSUPPORTED
XR_ERROR_VALIDATION_FAILURE
XR_ERROR_RUNTIME_FAILURE
XR_ERROR_HANDLE_INVALID
XR_ERROR_INSTANCE_LOST
XR_ERROR_SESSION_LOST
XR_ERROR_OUT_OF_MEMORY
XR_ERROR_LIMIT_REACHED
XR_ERROR_FEATURE_UNSUPPORTED

The XrEyeTrackerCreateInfoANDROID structure is defined as:

typedef struct XrEyeTrackerCreateInfoANDROID {
    XrStructureType    type;
    void*              next;
} XrEyeTrackerCreateInfoANDROID;

Member Descriptions

type is the XrStructureType of this structure.
next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.

The XrEyeTrackerCreateInfoANDROID structure describes the information to create an XrEyeTrackerANDROID handle.

Valid Usage (Implicit)

The XR_ANDROID_eye_tracking extension must be enabled prior to using XrEyeTrackerCreateInfoANDROID
type must be XR_TYPE_EYE_TRACKER_CREATE_INFO_ANDROID
next must be NULL or a valid pointer to the next structure in a structure chain

The xrDestroyEyeTrackerANDROID function is defined as:

XrResult xrDestroyEyeTrackerANDROID(
    XrEyeTrackerANDROID                         eyeTracker);

Parameter Descriptions

eyeTracker is an XrEyeTrackerANDROID previously created by xrCreateEyeTrackerANDROID.

xrDestroyEyeTrackerANDROID function releases the eyeTracker and the underlying resources when finished with eye tracking experiences.

Valid Usage (Implicit)

The XR_ANDROID_eye_tracking extension must be enabled prior to calling xrDestroyEyeTrackerANDROID
eyeTracker must be a valid XrEyeTrackerANDROID handle.

Thread Safety

Access to eyeTracker, and any child handles, must be externally synchronized

Return Codes

Success

XR_SUCCESS

Failure

XR_ERROR_FUNCTION_UNSUPPORTED
XR_ERROR_HANDLE_INVALID

Getting eyes information

The xrGetCoarseTrackingEyesInfoANDROID function is defined as:

XrResult xrGetCoarseTrackingEyesInfoANDROID(
    XrEyeTrackerANDROID                         eyeTracker,
    const XrEyesGetInfoANDROID*                 getInfo,
    XrEyesANDROID*                              eyesOutput);

Parameter Descriptions

eyeTracker is an XrEyeTrackerANDROID previously created by xrCreateEyeTrackerANDROID.
getInfo is a pointer to XrEyesGetInfoANDROID used to specify what output is required.
infoOutput is a pointer to XrEyesANDROID that contains the returned eyes information including poses and states.

xrGetCoarseTrackingEyesInfoANDROID function gets the information for eye states and poses in a way that preserves user privacy.

The runtime must return XR_ERROR_PERMISSION_INSUFFICIENT if the application does not have the android.permission.EYE_TRACKING_COARSE permission.

The eyes information is resolved and relative to the base space at the time of the call to xrGetCoarseTrackingEyesInfoANDROID using XrEyesGetInfoANDROID::time, XrEyesGetInfoANDROID::baseSpace.

At any point of time both the position and direction of the eye pose is tracked or untracked. This means that applications can expect that both XR_SPACE_LOCATION_POSITION_TRACKED_BIT and XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT will either be set or cleared on the supplied XrEyesANDROID::eyes, and that XrEyesANDROID::mode will indicate the tracking states.

Valid Usage (Implicit)

The XR_ANDROID_eye_tracking extension must be enabled prior to calling xrGetCoarseTrackingEyesInfoANDROID
eyeTracker must be a valid XrEyeTrackerANDROID handle
getInfo must be a pointer to a valid XrEyesGetInfoANDROID structure
eyesOutput must be a pointer to an XrEyesANDROID structure

Return Codes

Success

XR_SUCCESS
XR_SESSION_LOSS_PENDING

Failure

XR_ERROR_FUNCTION_UNSUPPORTED
XR_ERROR_VALIDATION_FAILURE
XR_ERROR_RUNTIME_FAILURE
XR_ERROR_HANDLE_INVALID
XR_ERROR_INSTANCE_LOST
XR_ERROR_SESSION_LOST
XR_ERROR_OUT_OF_MEMORY
XR_ERROR_LIMIT_REACHED
XR_ERROR_TIME_INVALID
XR_ERROR_PERMISSION_INSUFFICIENT

The xrGetFineTrackingEyesInfoANDROID function is defined as: {:#xrGetFineTrackingEyesInfoANDROID} C++ XrResult xrGetFineTrackingEyesInfoANDROID( XrEyeTrackerANDROID eyeTracker, const XrEyesGetInfoANDROID* getInfo, XrEyesANDROID* eyesOutput);

Parameter Descriptions

eyeTracker is an XrEyeTrackerANDROID previously created by xrCreateEyeTrackerANDROID.
getInfo is a pointer to XrEyesGetInfoANDROID used to specify what output is required.
infoOutput is a pointer to XrEyesANDROID that contains the returned eyes information including poses and states. xrGetFineTrackingEyesInfoANDROID function gets the information for eye states and poses with higher precision than xrGetCoarseTrackingEyesInfoANDROID.

The runtime must return XR_ERROR_PERMISSION_INSUFFICIENT if the application does not have the android.permission.EYE_TRACKING_FINE permission.

The eyes information is resolved and relative to the base space at the time of the call to xrGetFineTrackingEyesInfoANDROID using XrEyesGetInfoANDROID::time, XrEyesGetInfoANDROID::baseSpace.

Valid Usage (Implicit)

The XR_ANDROID_eye_tracking extension must be enabled prior to calling xrGetFineTrackingEyesInfoANDROID
eyeTracker must be a valid XrEyeTrackerANDROID handle
getInfo must be a pointer to a valid XrEyesGetInfoANDROID structure
eyesOutput must be a pointer to an XrEyesANDROID structure

Return Codes

Success

XR_SUCCESS
XR_SESSION_LOSS_PENDING

Failure

XR_ERROR_FUNCTION_UNSUPPORTED
XR_ERROR_VALIDATION_FAILURE
XR_ERROR_RUNTIME_FAILURE
XR_ERROR_HANDLE_INVALID
XR_ERROR_INSTANCE_LOST
XR_ERROR_SESSION_LOST
XR_ERROR_OUT_OF_MEMORY
XR_ERROR_LIMIT_REACHED
XR_ERROR_TIME_INVALID
XR_ERROR_PERMISSION_INSUFFICIENT

XrEyesGetInfoANDROID structure contains the information required to retrieve eye poses and states.

typedef struct XrEyesGetInfoANDROID {
    XrStructureType    type;
    void*              next;
    XrTime             time;
    XrSpace            baseSpace;
} XrEyesGetInfoANDROID;

Member Descriptions

type is the XrStructureType of this structure.
next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
time is the XrTime at which to evaluate the coordinates relative to the baseSpace.
baseSpace the eye pose will be relative to this XrSpace at time.

Valid Usage (Implicit)

The XR_ANDROID_eye_tracking extension must be enabled prior to using XrEyesGetInfoANDROID
type must be XR_TYPE_EYES_GET_INFO_ANDROID
next must be NULL or a valid pointer to the next structure in a structure chain
baseSpace must be a valid XrSpace handle

XrEyesANDROID structure contains information on the tracked eyes.

typedef struct XrEyesANDROID {
    XrStructureType             type;
    void*                       next;
    XrEyeANDROID                eyes[XR_EYE_MAX_ANDROID];
    XrEyeTrackingModeANDROID    mode;
} XrEyesANDROID;

Member Descriptions

type is the XrStructureType of this structure.
next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
eyes is an array of XrEyeANDROID for the left and right eyes as indexed by XrEyeIndexANDROID.
mode is the XrEyeTrackingModeANDROID to indicate if the eyes are tracking and which ones.

Valid Usage (Implicit)

The XR_ANDROID_eye_tracking extension must be enabled prior to using XrEyesANDROID
type must be XR_TYPE_EYES_ANDROID
next must be NULL or a valid pointer to the next structure in a structure chain
Any given element of eyes must be a valid XrEyeANDROID structure
mode must be a valid XrEyeTrackingModeANDROID value

XrEyeANDROID structure describes the state, position and orientation of an eye.

typedef struct XrEyeANDROID {
    XrEyeStateANDROID    eyeState;
    XrPosef              eyePose;
} XrEyeANDROID;

Member Descriptions

eyeState is the XrEyeStateANDROID of an eye.
pose is an XrPosef defining the position and orientation of the origin of an eye within the reference frame of the corresponding XrEyesGetInfoANDROID::baseSpace. An identity orientation here represents a coordinate axes with +Z into the user's eyes, +X to the right and +Y up.

Valid Usage (Implicit)

The XR_ANDROID_eye_tracking extension must be enabled prior to using XrEyeANDROID
eyeState must be a valid XrEyeStateANDROID value

The XrEyeStateANDROID enumeration identifies the different states of tracked eyes.

typedef enum XrEyeStateANDROID {
    XR_EYE_STATE_INVALID_ANDROID = 0,
    XR_EYE_STATE_GAZING_ANDROID = 1,
    XR_EYE_STATE_SHUT_ANDROID = 2
} XrEyeStateANDROID;

The enums have the following meanings:

Enum	Description
`XR_EYE_STATE_INVALID_ANDROID`	Indicates that the eye is in an error state or not present.
`XR_EYE_STATE_GAZING_ANDROID`	Indicates that the eye is gazing.
`XR_EYE_STATE_SHUT_ANDROID`	Indicates that the eye is shut due to a wink or a blink.

The XrEyeIndexANDROID enumeration identifies the index of the left or right eye.

typedef enum XrEyeIndexANDROID {
    XR_EYE_INDEX_LEFT_ANDROID = 0,
    XR_EYE_INDEX_RIGHT_ANDROID = 1
} XrEyeIndexANDROID;

The enums have the following meanings:

Enum	Description
`XR_EYE_INDEX_LEFT_ANDROID`	Left eye.
`XR_EYE_INDEX_RIGHT_ANDROID`	Right eye.

The XrEyeTrackingModeANDROID enumeration identifies the different modes of tracked eyes.

typedef enum XrEyeTrackingModeANDROID {
    XR_EYE_TRACKING_MODE_NOT_TRACKING_ANDROID = 0,
    XR_EYE_TRACKING_MODE_RIGHT_ANDROID = 1,
    XR_EYE_TRACKING_MODE_LEFT_ANDROID = 2,
    XR_EYE_TRACKING_MODE_BOTH_ANDROID = 3
} XrEyeTrackingModeANDROID;

The enums have the following meanings:

Enum	Description
`XR_EYE_TRACKING_MODE_NOT_TRACKING_ANDROID`	Indicates that eye tracking is not active.
`XR_EYE_TRACKING_MODE_RIGHT_ANDROID`	Indicates that only the right eye is tracking.
`XR_EYE_TRACKING_MODE_LEFT_ANDROID`	Indicates that only the left eye is tracking.
`XR_EYE_TRACKING_MODE_BOTH_ANDROID`	Indicates that both the left and right eyes are tracking.

Example code for eye tracking

The following example code demonstrates how to get eye information relative to a view space.

XrSession session; // previously initialized, e.g. created at app startup.
XrSpace viewSpace; // space created for XR_REFERENCE_SPACE_TYPE_VIEW.

// The function pointers are previously initialized using xrGetInstanceProcAddr.
PFN_xrCreateEyeTrackerANDROID xrCreateEyeTrackerANDROID; // previously initialized
PFN_xrDestroyEyeTrackerANDROID xrDestroyEyeTrackerANDROID; // previously initialized
PFN_xrGetCoarseTrackingEyesInfoANDROID xrGetCoarseTrackingEyesInfoANDROID; // previously initialized
PFN_xrGetFineTrackingEyesInfoANDROID xrGetFineTrackingEyesInfoANDROID; // previously initialized

// This will use the XrSession that is bound to the eye tracker done at time of creation.
XrEyeTrackerANDROID eyeTracker;
XrEyeTrackerCreateInfoANDROID createInfo{
    .type = XR_TYPE_EYE_TRACKER_CREATE_INFO_ANDROID,
    .next = nullptr};
CHK_XR(xrCreateEyeTrackerANDROID(session, &createInfo, &eyeTracker));

while (1) {
    // ...
    // For every frame in frame loop
    // ...

    XrFrameState frameState;  // previously returned from xrWaitFrame
    const XrTime time = frameState.predictedDisplayTime;
    XrEyesANDROID fineEyesInfo{.type = XR_TYPE_EYES_ANDROID,
                               .next = nullptr,
                               .mode = XR_EYE_TRACKING_MODE_BOTH_ANDROID};
    XrEyesANDROID coarseEyesInfo{.type = XR_TYPE_EYES_ANDROID,
                                 .next = nullptr,
                                 .mode = XR_EYE_TRACKING_MODE_BOTH_ANDROID};
    XrEyesGetInfoANDROID eyesGetInfo{.type = XR_TYPE_EYES_GET_INFO_ANDROID,
                                     .next = nullptr,
                                     .time = time,
                                     .baseSpace = viewSpace};
    CHK_XR(xrGetCoarseTrackingEyesInfoANDROID(eyeTracker, &eyesGetInfo, &coarseEyesInfo));
    CHK_XR(xrGetFineTrackingEyesInfoANDROID(eyeTracker, &eyesGetInfo, &fineEyesInfo));

    // eyes tracking information is now available:
    // drawLeftEye(eyesInfo.eyes[XR_EYE_INDEX_LEFT_ANDROID].eyePose);
    // drawRightEye(eyesInfo.eyes[XR_EYE_INDEX_RIGHT_ANDROID].eyePose);

    // ...
    // Finish frame loop
    // ...
}

// after usage
CHK_XR(xrDestroyEyeTrackerANDROID(eyeTracker));