XR_ANDROID_light_estimation OpenXR extension

Name String

XR_ANDROID_light_estimation

Extension Type

Instance extension

Registered Extension Number

701

Revision

1

Extension and Version Dependencies

OpenXR 1.0

Last Modified Date

2024-09-18

IP Status

No known IP claims.

Contributors

Jared Finder, Google

Cairn Overturf, Google

Spencer Quin, Google

Levana Chen, Google

Nihav Jain, Google

Overview

This extension allows the application to request data representing the lighting of the real-world environment around the headset. This information can be used when rendering virtual objects to light them under the same conditions as the scene they're placed in.

Inspect system capability

An application can inspect whether the system supports light estimation by chaining an XrSystemLightEstimationPropertiesANDROID structure to the XrSystemProperties when calling xrGetSystemProperties. 

typedef struct XrSystemLightEstimationPropertiesANDROID {
    XrStructureType    type;
    void*              next;
    XrBool32           supportsLightEstimation;
} XrSystemLightEstimationPropertiesANDROID;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
  • supportsLightEstimation is an XrBool32, indicating if the current system supports light estimation.

An application can inspect whether the system is capable of supporting light estimation by extending the XrSystemProperties with XrSystemLightEstimationPropertiesANDROID structure when calling xrGetSystemProperties.

If the system is not capable of supporting light estimation, then it will return XR_FALSE for supportsLightEstimation, and XR_ERROR_FEATURE_UNSUPPORTED from xrCreateLightEstimatorANDROID.

Valid Usage (Implicit)

Create a light estimator handle

XR_DEFINE_HANDLE(XrLightEstimatorANDROID)

The XrLightEstimatorANDROID handle represents a light estimator. This handle can be used to access light estimation information using other functions in this extension.

The xrCreateLightEstimatorANDROID function is defined as: 

XrResult xrCreateLightEstimatorANDROID(
    XrSession                                   session,
    XrLightEstimatorCreateInfoANDROID*          createInfo,
    XrLightEstimatorANDROID*                    outHandle);

Parameter Descriptions

The application can use xrCreateLightEstimatorANDROID function to create a light estimator.

The returned light estimator handle may be subsequently used in API calls. If an application wants to indicate to the runtime that it is done accessing light estimation data it must destroy the handle using xrDestroyLightEstimatorANDROID.

Valid Usage (Implicit)

Return Codes

Success

  • XR_SUCCESS
  • XR_SESSION_LOSS_PENDING

Failure

  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_RUNTIME_FAILURE
  • XR_ERROR_INSTANCE_LOST
  • XR_ERROR_SESSION_LOST
  • XR_ERROR_OUT_OF_MEMORY
  • XR_ERROR_HANDLE_INVALID
  • XR_ERROR_LIMIT_REACHED

The XrLightEstimatorCreateInfoANDROID structure describes the information to create an XrLightEstimatorANDROID handle. 

typedef struct XrLightEstimatorCreateInfoANDROID {
    XrStructureType    type;
    void*              next;
} XrLightEstimatorCreateInfoANDROID;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.

Valid Usage (Implicit)

xrDestroyLightEstimatorANDROID function releases the estimator and any underlying resources. 

XrResult xrDestroyLightEstimatorANDROID(
    XrLightEstimatorANDROID                     estimator);

Parameter Descriptions

Valid Usage (Implicit)

Thread Safety

  • Access to estimator, and any child handles, must be externally synchronized

Return Codes

Success

  • XR_SUCCESS

Failure

  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_HANDLE_INVALID

Access light estimation data

The xrGetLightEstimateANDROID function is defined as: 

XrResult xrGetLightEstimateANDROID(
    XrLightEstimatorANDROID                     estimator,
    const XrLightEstimateGetInfoANDROID*        input,
    XrLightEstimateANDROID*                     output);

Parameter Descriptions

Valid Usage (Implicit)

Return Codes

Success

  • XR_SUCCESS
  • XR_SESSION_LOSS_PENDING

Failure

  • XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_VALIDATION_FAILURE
  • XR_ERROR_RUNTIME_FAILURE
  • XR_ERROR_HANDLE_INVALID
  • XR_ERROR_INSTANCE_LOST
  • XR_ERROR_SESSION_LOST
  • XR_ERROR_OUT_OF_MEMORY
  • XR_ERROR_TIME_INVALID

A XrLightEstimateGetInfoANDROID describes the information required to get light estimation data. 

typedef struct XrLightEstimateGetInfoANDROID {
    XrStructureType    type;
    void*              next;
    XrSpace            space;
    XrTime             time;
} XrLightEstimateGetInfoANDROID;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain.
  • space is the XrSpace defining the reference space in which the returned light direction and spherical harmonics are expressed in.
  • time is the XrTime that describes the time at which the application wishes to query the light estimation.

Valid Usage (Implicit)

XrLightEstimateANDROID structure contains light estimation data. 

typedef struct XrLightEstimateANDROID {
    XrStructureType                type;
    void*                          next;
    XrLightEstimateStateANDROID    state;
    XrTime                         lastUpdatedTime;
} XrLightEstimateANDROID;

Member Descriptions

To obtain light estimate information for ambient light, spherical harmonics and the primary directional light, applications can chain instances of the following structures, XrAmbientLightANDROID, XrSphericalHarmonicsANDROID and XrDirectionalLightANDROID respectively onto XrLightEstimateANDROID::next.

Valid Usage (Implicit)

XrAmbientLightANDROID structure contains light estimation data about the ambient light in the scene. 

typedef struct XrAmbientLightANDROID {
    XrStructureType                type;
    void*                          next;
    XrLightEstimateStateANDROID    state;
    XrVector3f                     intensity;
    XrVector3f                     colorCorrection;
} XrAmbientLightANDROID;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain. Valid structures are XrSphericalHarmonicsANDROID and XrDirectionalLightANDROID.
  • state is the XrLightEstimateStateANDROID representing the state of the light estimate.
  • intensity is a XrVector3 representing the intensity of the ambient light. Each component of the vector corresponds to the red, green and blue channels.
  • colorCorrection is a XrVector3 with values in gamma space. Componentwise multiply the color of your gamma corrected render with these values.

Valid Usage (Implicit)

XrSphericalHarmonicsANDROID structure contains spherical harmonics representing the lighting of the scene. 

typedef struct XrSphericalHarmonicsANDROID {
    XrStructureType                    type;
    void*                              next;
    XrLightEstimateStateANDROID        state;
    XrSphericalHarmonicsKindANDROID    kind;
    float                              coefficients[9][3];
} XrSphericalHarmonicsANDROID;

Member Descriptions

Valid Usage (Implicit)

XrDirectionalLightANDROID structure contains light estimation data. 

typedef struct XrDirectionalLightANDROID {
    XrStructureType                type;
    void*                          next;
    XrLightEstimateStateANDROID    state;
    XrVector3f                     intensity;
    XrVector3f                     direction;
} XrDirectionalLightANDROID;

Member Descriptions

Valid Usage (Implicit)

The XrSphericalHarmonicsKindANDROID enumeration identifies to the runtime, what type of spherical harmonics the application is requesting. 

typedef enum XrSphericalHarmonicsKindANDROID {
    XR_SPHERICAL_HARMONICS_KIND_TOTAL_ANDROID = 0,
    XR_SPHERICAL_HARMONICS_KIND_AMBIENT_ANDROID = 1
} XrSphericalHarmonicsKindANDROID;

The enums have the following meanings:

Enum

Description

XR_SPHERICAL_HARMONICS_KIND_TOTAL_ANDROID

The spherical harmonic coefficients represent the environment light radiance function, excluding the contribution from the main light.

XR_SPHERICAL_HARMONICS_KIND_AMBIENT_ANDROID

The spherical harmonic coefficients represent the environment light radiance function, including the contribution from the main light.

The XrLightEstimateStateANDROID enumeration identifies to the runtime, what type of spherical harmonics the application is requesting. 

typedef enum XrLightEstimateStateANDROID {
    XR_LIGHT_ESTIMATE_STATE_VALID_ANDROID = 0,
    XR_LIGHT_ESTIMATE_STATE_INVALID_ANDROID = 1
} XrLightEstimateStateANDROID;

The enums have the following meanings:

Enum

Description

XR_LIGHT_ESTIMATE_STATE_VALID_ANDROID

The light estimate is valid

XR_LIGHT_ESTIMATE_STATE_INVALID_ANDROID

The light estimate is invalid

Example code for light estimation

The following example code demonstrates how to get all possible light estimation quantities from the runtime.

XrSession session;  // Created at app startup
XrSpace appSpace;   // Created previously.

XrLightEstimatorANDROID estimator;
XrLightEstimatorCreateInfoANDROID createInfo = {
    .type = XR_TYPE_LIGHT_ESTIMATOR_CREATE_INFO_ANDROID};
CHK_XR(xrCreateLightEstimatorANDROID(session, &createInfo, &estimator));

// Every frame
XrTime updateTime;  // Time used for the current frame's simulation update.

XrLightEstimateGetInfoANDROID info = {
    .type = XR_TYPE_LIGHT_ESTIMATE_GET_INFO_ANDROID,
    .space = appSpace,
    .time = updateTime,
};

XrDirectionalLightANDROID directionalLight = {
    .type = XR_TYPE_DIRECTIONAL_LIGHT_ANDROID,
};

XrSphericalHarmonicsANDROID totalSh = {
    .type = XR_TYPE_SPHERICAL_HARMONICS_ANDROID,
    .kind = XR_SPHERICAL_HARMONICS_KIND_TOTAL_ANDROID,
    .next = &directionalLight,
};

XrSphericalHarmonicsANDROID ambientSh = {
    .type = XR_TYPE_SPHERICAL_HARMONICS_ANDROID,
    .kind = XR_SPHERICAL_HARMONICS_KIND_AMBIENT_ANDROID,
    .next = &totalSh,
};

XrAmbientLightANDROID ambientLight = {
    .type = XR_TYPE_AMBIENT_LIGHT_ANDROID,
    .next = &ambientSh,
};

XrLightEstimateANDROID estimate = {
    .type = XR_TYPE_LIGHT_ESTIMATE_ANDROID,
    .next = &ambientLight,
};

XrResult result = xrGetLightEstimateANDROID(estimator, &info, &estimate);
if (result == XR_SUCCESS &&
    estimate.state == XR_LIGHT_ESTIMATE_STATE_VALID_ANDROID) {
  // use directionalLight, totalSh, ambientSh, ambientLight if each
  // struct has a valid state field
}

// When you want to disable light estimation
CHK_XR(xrDestroyLightEstimatorANDROID(estimator));

New Object Types

  • XrLightEstimator

XrObjectType enumeration is extended with:

  • XR_OBJECT_TYPE_LIGHT_ESTIMATOR_ANDROID

XrStructureType enumeration is extended with:

  • XR_TYPE_LIGHT_ESTIMATOR_CREATE_INFO_ANDROID
  • XR_TYPE_LIGHT_ESTIMATE_GET_INFO_ANDROID
  • XR_TYPE_LIGHT_ESTIMATE_ANDROID
  • XR_TYPE_DIRECTIONAL_LIGHT_ANDROID
  • XR_TYPE_SPHERICAL_HARMONICS_ANDROID
  • XR_TYPE_AMBIENT_LIGHT_ANDROID

New Enums

New Structures

New Functions

Issues

Version History

  • Revision 1, 2024-09-16 (Cairn Overturf)
    • Initial extension description