XR_ANDROID_eye_tracking

Name String

XR_ANDROID_eye_tracking

Extension Type

Instance extension

Registered Extension Number

457

Revision

1

Ratification Status

Not ratified

Extension and Version Dependencies

OpenXR 1.0

Last Modified Date

2025-01-17

IP Status

No known IP claims.

Contributors

Spencer Quin, Google
Jared Finder, Google
Levana Chen, Google
Kenny Vercaemer, Google
Prasanthi Gurumurthy, Google
Nihav Jain, Google

Overview

This extension enables applications to obtain position and orientation of the user’s eyes, as well as eye tracking status.

Eye tracking data is provided in two modes: coarse and fine. Coarse tracking provides a coarse estimate of the user’s eyes, while fine tracking provides a more accurate estimate. Coarse tracking is meant for applications that want to provide a basic avatar-like representation, while fine tracking is meant for more precise applications.

For interaction, XR_EXT_eye_gaze_interaction should be used.

Permissions

Android applications must have the android.permission.EYE_TRACKING_COARSE or the android.permission.EYE_TRACKING_FINE permission listed in their manifest. These permissions are considered dangerous permissions. The application must request the permission at runtime to use these functions:

(protection level: dangerous)

Inspect system capability

The XrSystemEyeTrackingPropertiesANDROID structure is defined as:

typedef struct XrSystemEyeTrackingPropertiesANDROID {
    XrStructureType    type;
    void*              next;
    XrBool32           supportsEyeTracking;
} XrSystemEyeTrackingPropertiesANDROID;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
  • supportsEyeTracking is an XrBool32 indicating if current system supports eye tracking.

An application can inspect whether the system is capable of eye tracking by extending the XrSystemProperties with XrSystemEyeTrackingPropertiesANDROID structure when calling xrGetSystemProperties .

If and only if a runtime returns XR_FALSE for supportsEyeTracking , the runtime must return XR_ERROR_FEATURE_UNSUPPORTED from xrCreateEyeTrackerANDROID .

Valid Usage (Implicit)

Creating an eye tracker handle

XR_DEFINE_HANDLE(XrEyeTrackerANDROID)

The XrEyeTrackerANDROID handle represents an eye tracker for tracking eyes and accurately maps where the user is looking.

Eye tracking data is sensitive personal information and is closely linked to personal privacy and integrity. It is strongly recommended that applications that store or transfer eye tracking data always ask the user for active and specific acceptance to do so.

The application uses this handle to access eye tracking data using other functions in this extension.

Eye tracking provides eye pose and status representation in the scene.

The xrCreateEyeTrackerANDROID function is defined as:

XrResult xrCreateEyeTrackerANDROID(
    XrSession                                   session,
    const XrEyeTrackerCreateInfoANDROID*        createInfo,
    XrEyeTrackerANDROID*                        eyeTracker);

Parameter Descriptions

An application creates an XrEyeTrackerANDROID handle using the xrCreateEyeTrackerANDROID function.

If the system does not support eye tracking, the runtime must return XR_ERROR_FEATURE_UNSUPPORTED from xrCreateEyeTrackerANDROID .

Valid Usage (Implicit)

Return Codes

Success

  • XR_SUCCESS
  • XR_SESSION_LOSS_PENDING

Failure

  • XR_ERROR_FEATURE_UNSUPPORTED
  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_HANDLE_INVALID
  • XR_ERROR_INSTANCE_LOST
  • XR_ERROR_LIMIT_REACHED
  • XR_ERROR_OUT_OF_MEMORY
  • XR_ERROR_RUNTIME_FAILURE
  • XR_ERROR_SESSION_LOST
  • XR_ERROR_VALIDATION_FAILURE

The XrEyeTrackerCreateInfoANDROID structure is defined as:

typedef struct XrEyeTrackerCreateInfoANDROID {
    XrStructureType    type;
    const void*        next;
} XrEyeTrackerCreateInfoANDROID;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.

The XrEyeTrackerCreateInfoANDROID structure describes the information to create an XrEyeTrackerANDROID handle.

Valid Usage (Implicit)

The xrDestroyEyeTrackerANDROID function is defined as:

XrResult xrDestroyEyeTrackerANDROID(
    XrEyeTrackerANDROID                         eyeTracker);

Parameter Descriptions

xrDestroyEyeTrackerANDROID function releases the eyeTracker and the underlying resources when finished with eye tracking experiences.

Valid Usage (Implicit)

Thread Safety

  • Access to eyeTracker , and any child handles, must be externally synchronized

Return Codes

Success

  • XR_SUCCESS

Failure

  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_HANDLE_INVALID

Getting eyes information

The xrGetCoarseTrackingEyesInfoANDROID function is defined as:

XrResult xrGetCoarseTrackingEyesInfoANDROID(
    XrEyeTrackerANDROID                         eyeTracker,
    const XrEyesGetInfoANDROID*                 getInfo,
    XrEyesANDROID*                              eyesOutput);

Parameter Descriptions

xrGetCoarseTrackingEyesInfoANDROID function gets the information for eye states and poses in a way that preserves user privacy.

The runtime must return XR_ERROR_PERMISSION_INSUFFICIENT if the application does not have the android.permission.EYE_TRACKING_COARSE permission.

The eye poses are expressed in XrEyesGetInfoANDROID :: baseSpace at XrEyesGetInfoANDROID :: time .

At any point of time both the position and direction of the eye pose is tracked or untracked. This means that the runtime must set both XR_SPACE_LOCATION_POSITION_TRACKED_BIT and XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT or clear both XR_SPACE_LOCATION_POSITION_TRACKED_BIT and XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT on the supplied XrEyesANDROID :: eyes , and set XrEyesANDROID :: mode to indicate the tracking states.

Valid Usage (Implicit)

Return Codes

Success

  • XR_SUCCESS
  • XR_SESSION_LOSS_PENDING

Failure

  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_HANDLE_INVALID
  • XR_ERROR_INSTANCE_LOST
  • XR_ERROR_LIMIT_REACHED
  • XR_ERROR_OUT_OF_MEMORY
  • XR_ERROR_PERMISSION_INSUFFICIENT
  • XR_ERROR_RUNTIME_FAILURE
  • XR_ERROR_SESSION_LOST
  • XR_ERROR_TIME_INVALID
  • XR_ERROR_VALIDATION_FAILURE

The xrGetFineTrackingEyesInfoANDROID function is defined as:

XrResult xrGetFineTrackingEyesInfoANDROID(
    XrEyeTrackerANDROID                         eyeTracker,
    const XrEyesGetInfoANDROID*                 getInfo,
    XrEyesANDROID*                              eyesOutput);

Parameter Descriptions

xrGetFineTrackingEyesInfoANDROID function gets the information for eye states and poses with higher precision than xrGetCoarseTrackingEyesInfoANDROID .

The runtime must return XR_ERROR_PERMISSION_INSUFFICIENT if the application does not have the android.permission.EYE_TRACKING_FINE permission.

The eyes information is resolved and relative to the base space at the time of the call to xrGetFineTrackingEyesInfoANDROID using XrEyesGetInfoANDROID :: time , XrEyesGetInfoANDROID :: baseSpace . The runtime must resolve the location defined by XrEyesGetInfoANDROID :: baseSpace at the time of the call.

At any point of time both the position and direction of the eye pose is tracked or untracked. This means that the runtime must set both XR_SPACE_LOCATION_POSITION_TRACKED_BIT and XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT or clear both XR_SPACE_LOCATION_POSITION_TRACKED_BIT and XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT on the supplied XrEyesANDROID :: eyes , and set XrEyesANDROID :: mode to indicate the tracking states.

Valid Usage (Implicit)

Return Codes

Success

  • XR_SUCCESS
  • XR_SESSION_LOSS_PENDING

Failure

  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_HANDLE_INVALID
  • XR_ERROR_INSTANCE_LOST
  • XR_ERROR_LIMIT_REACHED
  • XR_ERROR_OUT_OF_MEMORY
  • XR_ERROR_PERMISSION_INSUFFICIENT
  • XR_ERROR_RUNTIME_FAILURE
  • XR_ERROR_SESSION_LOST
  • XR_ERROR_TIME_INVALID
  • XR_ERROR_VALIDATION_FAILURE

XrEyesGetInfoANDROID structure contains the information required to retrieve eye poses and states.

typedef struct XrEyesGetInfoANDROID {
    XrStructureType    type;
    const void*        next;
    XrTime             time;
    XrSpace            baseSpace;
} XrEyesGetInfoANDROID;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
  • time is the XrTime at which to evaluate the coordinates relative to the baseSpace .
  • baseSpace is the XrSpace the eye poses will be relative to at time .

Valid Usage (Implicit)

The XrEyesANDROID structure contains information about the tracked eyes.

typedef struct XrEyesANDROID {
    XrStructureType             type;
    void*                       next;
    XrEyeANDROID                eyes[XR_EYE_MAX_ANDROID];
    XrEyeTrackingModeANDROID    mode;
} XrEyesANDROID;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
  • eyes is an array of XrEyeANDROID , of size XR_EYE_MAX_ANDROID , for the left and right eyes as indexed by XrEyeIndexANDROID .
  • mode is the XrEyeTrackingModeANDROID indicating which of the eyes are currently tracking.

Valid Usage (Implicit)

The XrEyeANDROID structure describes the state, position and orientation of an eye.

typedef struct XrEyeANDROID {
    XrEyeStateANDROID    eyeState;
    XrPosef              eyePose;
} XrEyeANDROID;

Member Descriptions

  • eyeState is the XrEyeStateANDROID of an eye.
  • eyePose is an XrPosef defining the position and orientation of the pupil of an eye within the reference frame of the corresponding XrEyesGetInfoANDROID :: baseSpace . An identity orientation here represents a coordinate axes with +Z into the user’s eyes, +X to the right and +Y up.

Valid Usage (Implicit)

The XrEyeStateANDROID enumeration identifies the different state of tracked eyes.

typedef enum XrEyeStateANDROID {
    XR_EYE_STATE_INVALID_ANDROID = 0,
    XR_EYE_STATE_GAZING_ANDROID = 1,
    XR_EYE_STATE_SHUT_ANDROID = 2,
    XR_EYE_STATE_MAX_ENUM_ANDROID = 0x7FFFFFFF
} XrEyeStateANDROID;

The enums have the following meanings:

Enum Description

XR_EYE_STATE_INVALID_ANDROID

Indicates that the eye is in an error state or not present.

XR_EYE_STATE_GAZING_ANDROID

Indicates that the eye is currently gazing.

XR_EYE_STATE_SHUT_ANDROID

Indicates that the eye is currently shut due to a wink or a blink.

The XrEyeIndexANDROID enumeration identifies the index of the left or right eyes.

typedef enum XrEyeIndexANDROID {
    XR_EYE_INDEX_LEFT_ANDROID = 0,
    XR_EYE_INDEX_RIGHT_ANDROID = 1,
    XR_EYE_INDEX_MAX_ENUM_ANDROID = 0x7FFFFFFF
} XrEyeIndexANDROID;

The enums have the following meanings:

Enum Description

XR_EYE_INDEX_LEFT_ANDROID

Left eye.

XR_EYE_INDEX_RIGHT_ANDROID

Right eye.

The XrEyeTrackingModeANDROID enumeration identifies the different state of tracked eyes.

typedef enum XrEyeTrackingModeANDROID {
    XR_EYE_TRACKING_MODE_NOT_TRACKING_ANDROID = 0,
    XR_EYE_TRACKING_MODE_RIGHT_ANDROID = 1,
    XR_EYE_TRACKING_MODE_LEFT_ANDROID = 2,
    XR_EYE_TRACKING_MODE_BOTH_ANDROID = 3,
    XR_EYE_TRACKING_MODE_MAX_ENUM_ANDROID = 0x7FFFFFFF
} XrEyeTrackingModeANDROID;

The enums have the following meanings:

Enum Description

XR_EYE_TRACKING_MODE_NOT_TRACKING_ANDROID

Indicates that eye tracking is not currently active.

XR_EYE_TRACKING_MODE_RIGHT_ANDROID

Indicates that only the right eye is tracking.

XR_EYE_TRACKING_MODE_LEFT_ANDROID

Indicates that only the left eye is tracking.

XR_EYE_TRACKING_MODE_BOTH_ANDROID

Indicates that both the left and right eyes are tracking.

Example code for eye tracking

The following example code demonstrates how to get eyes information relative to a view space.

XrSession session; // previously initialized, e.g. created at app startup.
XrSpace viewSpace; // space created for XR_REFERENCE_SPACE_TYPE_VIEW.

// The function pointers are previously initialized using xrGetInstanceProcAddr.
PFN_xrCreateEyeTrackerANDROID xrCreateEyeTrackerANDROID; // previously initialized
PFN_xrDestroyEyeTrackerANDROID xrDestroyEyeTrackerANDROID; // previously initialized
PFN_xrGetCoarseTrackingEyesInfoANDROID xrGetCoarseTrackingEyesInfoANDROID; // previously initialized
PFN_xrGetFineTrackingEyesInfoANDROID xrGetFineTrackingEyesInfoANDROID; // previously initialized

// This will use the XrSession that is bound to the eye tracker done at time of creation.
XrEyeTrackerANDROID eyeTracker;
XrEyeTrackerCreateInfoANDROID createInfo{
    .type = XR_TYPE_EYE_TRACKER_CREATE_INFO_ANDROID,
    .next = nullptr};
CHK_XR(xrCreateEyeTrackerANDROID(session, &createInfo, &eyeTracker));

while (1) {
    // ...
    // For every frame in frame loop
    // ...

    XrFrameState frameState;  // previously returned from xrWaitFrame
    const XrTime time = frameState.predictedDisplayTime;
    XrEyesANDROID fineEyesInfo{.type = XR_TYPE_EYES_ANDROID,
                               .next = nullptr,
                               .mode = XR_EYE_TRACKING_MODE_BOTH_ANDROID};
    XrEyesANDROID coarseEyesInfo{.type = XR_TYPE_EYES_ANDROID,
                                 .next = nullptr,
                                 .mode = XR_EYE_TRACKING_MODE_BOTH_ANDROID};
    XrEyesGetInfoANDROID eyesGetInfo{.type = XR_TYPE_EYES_GET_INFO_ANDROID,
                                     .next = nullptr,
                                     .time = time,
                                     .baseSpace = viewSpace};
    CHK_XR(xrGetCoarseTrackingEyesInfoANDROID(eyeTracker, &eyesGetInfo, &coarseEyesInfo));
    CHK_XR(xrGetFineTrackingEyesInfoANDROID(eyeTracker, &eyesGetInfo, &fineEyesInfo));

    // eyes tracking information is now available:
    // drawLeftEye(eyesInfo.eyes[XR_EYE_INDEX_LEFT_ANDROID].eyePose);
    // drawRightEye(eyesInfo.eyes[XR_EYE_INDEX_RIGHT_ANDROID].eyePose);

    // ...
    // Finish frame loop
    // ...
}

// after usage
CHK_XR(xrDestroyEyeTrackerANDROID(eyeTracker));

New Object Types

New Commands

New Structures

New Enums

New Enum Constants

  • XR_ANDROID_EYE_TRACKING_EXTENSION_NAME
  • XR_ANDROID_eye_tracking_SPEC_VERSION
  • XR_EYE_MAX_ANDROID

  • Extending XrObjectType :

    • XR_OBJECT_TYPE_EYE_TRACKER_ANDROID

  • Extending XrStructureType :

    • XR_TYPE_EYES_ANDROID
    • XR_TYPE_EYES_GET_INFO_ANDROID
    • XR_TYPE_EYE_TRACKER_CREATE_INFO_ANDROID
    • XR_TYPE_SYSTEM_EYE_TRACKING_PROPERTIES_ANDROID

Issues

Version History

  • Revision 1, 2025-01-17 (Kenny Vercaemer)

    • Initial extension description