XR_ANDROID_trackables OpenXR extension

Name String

XR_ANDROID_trackables_object

Extension Type

Instance extension

Registered Extension Number

467

Revision

1

Extension and Version Dependencies

XR_ANDROID_trackables

Last Modified Date

2024-10-03

IP Status

No known IP claims.

Contributors

Spencer Quin, Google

Nihav Jain, Google

John Pursey, Google

Jared Finder, Google

Levana Chen, Google

Kenny Vercaemer, Google

Overview

This extension enables physical object tracking. For example, keyboards, mice, and other objects in the environment.

Caution:

Permissions

Android applications must have the android.permission.SCENE_UNDERSTANDING permission listed in their manifest as this extension depends on XR_ANDROID_trackables and exposes the geometry of the environment. The android.permission.SCENE_UNDERSTANDING permission is considered a dangerous permission.
(protection level: dangerous)

Track objects

This extension adds XR_TRACKABLE_TYPE_OBJECT_ANDROID to XrTrackableTypeANDROID.

The application may create an XrTrackableTrackerANDROID by calling xrCreateTrackableTrackerANDROID and specifying XR_TRACKABLE_TYPE_OBJECT_ANDROID as the trackable type in XrTrackableTrackerCreateInfoANDROID::trackableType to track objects.

Get trackable object

The xrGetTrackableObjectANDROID function is defined as:

XrResult xrGetTrackableObjectANDROID(
    XrTrackableTrackerANDROID                   tracker,
    const XrTrackableGetInfoANDROID*            getInfo,
    XrTrackableObjectANDROID*                   objectOutput);

Parameter Descriptions

• tracker is the XrTrackableTrackerANDROID to query.
  • getInfo is the XrTrackableGetInfoANDROID with the information used to get the trackable object.
  • objectOutput is a pointer to the XrTrackableObjectANDROID structure in which the trackable object is returned.

XR_ERROR_MISMATCHING_TRACKABLE_TYPE_ANDROID will be returned if the trackable type of the XrTrackableANDROID is not XR_TRACKABLE_TYPE_OBJECT_ANDROID, or if the trackable type of the XrTrackableTrackerANDROID is not XR_TRACKABLE_TYPE_OBJECT_ANDROID.

Valid Usage (Implicit)

• The XR_ANDROID_trackables_object extension must be enabled prior to calling xrGetTrackableObjectANDROID
  • tracker must be a valid XrTrackableTrackerANDROID handle
  • getInfo must be a pointer to a valid XrTrackableGetInfoANDROID structure
  • objectOutput must be a pointer to an XrTrackableObjectANDROID structure

Return Codes

Success

• XR_SUCCESS
  • XR_SESSION_LOSS_PENDING

Failure

• XR_ERROR_FUNCTION_UNSUPPORTED
  • XR_ERROR_VALIDATION_FAILURE
  • XR_ERROR_RUNTIME_FAILURE
  • XR_ERROR_HANDLE_INVALID
  • XR_ERROR_INSTANCE_LOST
  • XR_ERROR_SESSION_LOST
  • XR_ERROR_SESSION_NOT_RUNNING
  • XR_ERROR_TIME_INVALID

The XrTrackableObjectANDROID structure is defined as:

typedef struct XrTrackableObjectANDROID {
    XrStructureType           type;
    void*                     next;
    XrTrackingStateANDROID    trackingState;
    XrPosef                   centerPose;
    XrExtent3DfEXT            extents;
    XrObjectLabelANDROID      objectLabel;
    XrTime                    lastUpdatedTime;
} XrTrackableObjectANDROID;

Member Descriptions

• type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
  • trackingState is the XrTrackingStateANDROID of the object.
  • centerPose is the XrPosef of the object located in XrTrackableGetInfoANDROID::baseSpace.
  • extents is the XrExtent3DfEXT dimension of the object.
  • objectLabel is the XrObjectLabelANDROID that the runtime has determined for this object.
  • lastUpdatedTime is the XrTime of the last update of the object.

Valid Usage (Implicit)

• The XR_ANDROID_trackables_object extension must be enabled prior to using XrTrackableObjectANDROID
  • type must be XR_TYPE_TRACKABLE_OBJECT_ANDROID
  • next must be NULL or a valid pointer to the next structure in a structure chain
  • trackingState must be a valid XrTrackingStateANDROID value
  • objectLabel must be a valid XrObjectLabelANDROID value

The XrObjectLabelANDROID enum is a label for a XrTrackableANDROID object.

typedef enum XrObjectLabelANDROID {
    XR_OBJECT_LABEL_UNKNOWN_ANDROID = 0,
    XR_OBJECT_LABEL_KEYBOARD_ANDROID = 1,
    XR_OBJECT_LABEL_MOUSE_ANDROID = 2,
    XR_OBJECT_LABEL_LAPTOP_ANDROID = 3,
    XR_OBJECT_LABEL_MAX_ENUM_ANDROID = 0x7FFFFFFF
} XrObjectLabelANDROID;

Example code for getting trackable objects

The following example code demonstrates how to get trackable objects.

XrSession session; // previously initialized

// The function pointers are previously initialized using xrGetInstanceProcAddr.
PFN_xrCreateTrackableTrackerANDROID xrCreateTrackableTrackerANDROID; // previously initialized
PFN_xrGetAllTrackablesANDROID xrGetAllTrackablesANDROID; // previously initialized
PFN_xrGetTrackableObjectANDROID xrGetTrackableObjectANDROID; // previously initialized
PFN_xrDestroyTrackableTrackerANDROID xrDestroyTrackableTrackerANDROID; // previously initialized

XrTime updateTime; // Time used for the current frame's simulation update.
XrSpace appSpace; // Space created for XR_REFERENCE_SPACE_TYPE_LOCAL.

XrTrackableTrackerCreateInfoANDROID
  createInfo{XR_TYPE_TRACKABLE_TRACKER_CREATE_INFO_ANDROID};
createInfo.trackableType = XR_TRACKABLE_TYPE_OBJECT_ANDROID;
XrTrackableTracker objectTrackableTracker;
XrResult result = xrCreateTrackableTrackerANDROID(
  session,
  &createInfo,
  &objectrackableTracker);
if (result != XR_SUCCESS) { /* Handle failures. */ }

uint32_t trackableCountOutput = 0;
std::vector<XrTrackableANDROID> allObjectTrackables;

// Query the number of trackables available.
result = xrGetAllTrackablesANDROID(
  objectTrackableTracker,
  0,
  &trackableCountOutput,
  nullptr
);

if (result == XR_SUCCESS) {
  allObjectTrackables.resize(trackableCountOutput, XR_NULL_TRACKABLE_ANDROID);

  // Fetch the actual trackable handles in the appropriately resized array.
  result = xrGetAllTrackablesANDROID(
    objectTrackableTracker,
    trackableCountOutput,
    &trackableCountOutput,
    allObjectTrackables.data());

  if (result == XR_SUCCESS) {
    for (XrTrackableANDROID trackable : allObjectTrackables) {

      // Object trackable query information
      XrTrackableGetInfoANDROID objectGetInfo;
      objectGetInfo.type = XR_TYPE_TRACKABLE_GET_INFO_ANDROID;
      objectGetInfo.next = nullptr;
      objectGetInfo.trackable = trackable;
      objectGetInfo.space = appSpace;
      objectGetInfo.time = updateTime;

      // Get the object trackable. Note that the tracker only returns object types.
      XrTrackableObjectANDROID object = { XR_TYPE_TRACKABLE_OBJECT_ANDROID };
      result = xrGetTrackableObjectANDROID(
        objectTrackableTracker,
        &objectGetInfo,
        &object
      );

      if (result == XR_SUCCESS) {
        /** Do Stuff with the object */
      }
    }
  }
}

// Release trackable tracker.
result = xrDestroyTrackableTrackerANDROID(objectTrackableTracker);

New Enum Constants

XrStructureType enumeration is extended with:

• XR_TYPE_TRACKABLE_OBJECT_ANDROID
• XR_TYPE_TRACKABLE_OBJECT_CONFIGURATION_ANDROID

XrTrackableTypeANDROID enumeration is extended with:

• XR_TRACKABLE_TYPE_OBJECT_ANDROID

New Enums

• XrObjectLabelANDROID

New Structures

• XrTrackableObjectANDROID

New Functions

• xrGetTrackableObjectANDROID

Issues

Version History

• Revision 1, 2024-10-03 (Kenny Vercaemer)
  • Initial extension description.