XR_ANDROID_raycast OpenXR extension

Name String

XR_ANDROID_raycast

Extension Type

Instance extension

Registered Extension Number

464

Revision

1

Extension and Version Dependencies

XR_ANDROID_trackables

Last Modified Date

2024-10-02

IP Status

No known IP claims.

Contributors

Spencer Quin, Google

Nihav Jain, Google

John Pursey, Google

Jared Finder, Google

Levana Chen, Google

Kenny Vercaemer, Google

Overview

This extension allows the application to perform raycasts against trackables in the environment. Raycasts can be useful for detecting objects in the environment that a ray would intersect with. For example:

• To determine where a floating object would fall when dropped, by using a vertical raycast.
• To determine where a user is looking, by using a forward-facing raycast.

Caution:
Permissions

Android applications must have the android.permission.SCENE_UNDERSTANDING permission listed in their manifest as this extension depends on XR_ANDROID_trackables and exposes the geometry of the environment. The android.permission.SCENE_UNDERSTANDING permission is considered a dangerous permission. (protection level: dangerous)

Query supported raycast capabilities

The xrEnumerateRaycastSupportedTrackableTypesANDROID function is defined as:

XrResult xrEnumerateRaycastSupportedTrackableTypesANDROID(
    XrInstance                                  instance,
    XrSystemId                                  systemId,
    uint32_t                                    trackableTypeCapacityInput,
    uint32_t*                                   trackableTypeCountOutput,
    XrTrackableTypeANDROID*                     trackableTypes);

Parameter Descriptions

• instance is the XrInstance from which systemId was retrieved.
• systemId is the XrSystemId whose supported trackable types for raycasting are being enumerated.
• trackableTypeCapacityInput is the capacity of the trackableTypes, or 0 to retrieve the required capacity.
• trackableTypeCountOutput is a pointer to the count of the array, or a pointer to the required capacity in the case that trackableTypeCapacityInput is insufficient.
  • trackableTypes is a pointer to an array of XrTrackableTypeANDROID, but can be NULL if trackableTypeCapacityInput is 0.
• See the Buffer Size Parameters section for a detailed description of retrieving the required trackableTypes size.

xrEnumerateRaycastSupportedTrackableTypesANDROID enumerates the trackable types that support raycasting by the current session.

Valid Usage (Implicit)

• The XR_ANDROID_raycast extension must be enabled prior to calling xrEnumerateRaycastSupportedTrackableTypesANDROID
• instance must be a valid XrInstance handle
• trackableTypeCountOutput must be a pointer to a uint32_t value
• If trackableTypeCapacityInput is not 0, trackableTypes must be a pointer to an array of trackableTypeCapacityInput XrTrackableTypeANDROID values

Return Codes

Success

• XR_SUCCESS

Failure

• XR_ERROR_VALIDATION_FAILURE
• XR_ERROR_RUNTIME_FAILURE
• XR_ERROR_HANDLE_INVALID
• XR_ERROR_INSTANCE_LOST
• XR_ERROR_SIZE_INSUFFICIENT
• XR_ERROR_SYSTEM_INVALID
• XR_ERROR_FUNCTION_UNSUPPORTED

Perform a raycast

The xrRaycastANDROID function is defined as:

XrResult xrRaycastANDROID(
    XrSession                                   session,
    const XrRaycastInfoANDROID*                 rayInfo,
    XrRaycastHitResultsANDROID*                 results);

Parameter Descriptions

• session is the XrSession that performs the raycast.
• rayInfo is the XrRaycastInfoANDROID that describes the ray to cast.
• results is a pointer to the XrRaycastHitResultsANDROID that receives the results of the raycast.

The application can perform raycasts by calling xrRaycastANDROID.

• If a raycast intersects more trackables than XrRaycastHitResultsANDROID::maxResults, the runtime must return the hit results that are closest to the XrRaycastHitResultsANDROID::origin of the ray.
• If a raycast intersects a trackable of type XR_TRACKABLE_TYPE_ANDROID_PLANE, that is subsumed by another plane, the runtime must return the hit result for the subsuming plane only.
• The runtime must return the hit results in closest-to-farthest order from XrRaycastInfoANDROID::origin along the XrRaycastInfoANDROID::trajectory vector.
• The runtime must return XR_ERROR_TRACKABLE_TYPE_NOT_SUPPORTED_ANDROID if the trackable type corresponding to the XrTrackableTrackerANDROID handles in XrRaycastInfoANDROID::trackers are not enumerated by xrEnumerateRaycastSupportedTrackableTypesANDROID.

Valid Usage (Implicit)

• The XR_ANDROID_raycast extension must be enabled prior to calling xrRaycastANDROID
• session must be a valid XrSession handle
• rayInfo must be a pointer to a valid XrRaycastInfoANDROID structure
• results must be a pointer to an XrRaycastHitResultsANDROID structure

Return Codes

Success

• XR_SUCCESS
• XR_SESSION_LOSS_PENDING

Failure

• XR_ERROR_FUNCTION_UNSUPPORTED
• XR_ERROR_TRACKABLE_TYPE_NOT_SUPPORTED_ANDROID
• XR_ERROR_VALIDATION_FAILURE
• XR_ERROR_RUNTIME_FAILURE
• XR_ERROR_HANDLE_INVALID
• XR_ERROR_INSTANCE_LOST
• XR_ERROR_SESSION_LOST
• XR_ERROR_OUT_OF_MEMORY
• XR_ERROR_LIMIT_REACHED
• XR_ERROR_POSE_INVALID
• XR_ERROR_TIME_INVALID
• XR_ERROR_FEATURE_UNSUPPORTED

The XrRaycastInfoANDROID structure is defined as:

typedef struct XrRaycastInfoANDROID {
    XrStructureType                     type;
    void*                               next;
    uint32_t                            maxResults;
    uint32_t                            trackerCount;
    const XrTrackableTrackerANDROID*    trackers;
    XrVector3f                          origin;
    XrVector3f                          trajectory;
    XrSpace                             space;
    XrTime                              time;
} XrRaycastInfoANDROID;

Member Descriptions

• type is the XrStructureType of this structure.
• next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
• maxResults is the uint32_t maximum number of results to return.
• trackerCount is the uint32_t count of the trackers array.
• trackers is the array of XrTrackableTrackerANDROID that the casted ray should be tested against.
• origin is the XrVector3f that the ray is cast from.
• trajectory is the XrVector3f that the ray is targeted at.
• space is the XrSpace that the ray is cast in.
• time is the XrTime the ray is cast at.

The XrRaycastInfoANDROID structure describes the ray to cast.

• The XrRaycastInfoANDROID::trackers array may contain trackers of different types.
• The XrRaycastInfoANDROID::trackers array must not contain multiple trackers of the same type, otherwise the runtime must return XR_ERROR_VALIDATION_FAILURE.

Valid Usage (Implicit)

• The XR_ANDROID_raycast extension must be enabled prior to using XrRaycastInfoANDROID
• type must be XR_TYPE_RAYCAST_INFO_ANDROID
• next must be NULL or a valid pointer to the next structure in a structure chain
• trackers must be a pointer to an array of trackerCount valid XrTrackableTrackerANDROID handles
• space must be a valid XrSpace handle
• The trackerCount parameter must be greater than 0
• Both of space and the elements of trackers must have been created, allocated, or retrieved from the same XrSession

The XrRaycastHitResultsANDROID structure is defined as:

typedef struct XrRaycastHitResultsANDROID {
    XrStructureType               type;
    void*                         next;
    uint32_t                      resultsCapacityInput;
    uint32_t                      resultsCountOutput;
    XrRaycastHitResultANDROID*    results;
} XrRaycastHitResultsANDROID;

Member Descriptions

• type is the XrStructureType of this structure.
• next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
• resultsCapacityInput is the capacity of the results array, or 0 to indicate a request to retrieve the required capacity.
• resultsCountOutput is a pointer to the count of results written, or a pointer to the required capacity in the case that resultsCapacityInput is insufficient.
• results is a pointer to an array of XrRaycastHitResultANDROID structures. It can be NULL if resultsCapacityInput is 0.
• See the Buffer Size Parameters section for a detailed description of retrieving the required results size.

The XrRaycastHitResultsANDROID contains the array of hits of a raycast.

The runtime must set resultsCountOutput to be less than or equal to XrRaycastInfoANDROID::maxResults.

Valid Usage (Implicit)

• The XR_ANDROID_raycast extension must be enabled prior to using XrRaycastHitResultsANDROID
• type must be XR_TYPE_RAYCAST_HIT_RESULTS_ANDROID
• next must be NULL or a valid pointer to the next structure in a structure chain
• results must be a pointer to an array of resultsCapacityInput XrRaycastHitResultANDROID-based structures
• The resultsCapacityInput parameter must be greater than 0

The XrRaycastHitResultANDROID structure is defined as:

typedef struct XrRaycastHitResultANDROID {
    XrTrackableTypeANDROID    type;
    XrTrackableANDROID        trackable;
    XrPosef                   pose;
} XrRaycastHitResultANDROID;

Member Descriptions

• type is the XrTrackableTypeANDROID of the trackable that the raycast hit.
• trackable is the XrTrackableANDROID that the raycast hit, or XR_NULL_TRACKABLE_ANDROID if the trackable type was XR_TRACKABLE_TYPE_DEPTH_ANDROID.
• pose is the XrPosef that the raycast hit.

The XrRaycastHitResultANDROID contains the details of a raycast hit.

The XrRaycastHitResultANDROID::pose for a plane hit must be such that X and Z are parallel to the plane, and the Y axis is normal to the plane.

Type of trackable hit	Description
XR_TRACKABLE_TYPE_PLANE_ANDROID	Hits horizontal and/or vertical surfaces to determine a point's correct depth and orientation.
XR_TRACKABLE_TYPE_DEPTH_ANDROID	Uses depth information from the entire scene to determine a point's correct depth and orientation.

Valid Usage (Implicit)

• The XR_ANDROID_raycast extension must be enabled prior to using XrRaycastHitResultANDROID
• type must be a valid XrTrackableTypeANDROID value

Example code for raycasting

The following example code demonstrates how to perform raycasts.

XrSession session; // previously initialized
XrTime updateTime; // previously initialized
XrSpace appSpace;  // space created for XR_REFERENCE_SPACE_TYPE_LOCAL.
XrPosef headPose;  // latest pose of the HMD.
XrTrackableTrackerANDROID planeTracker; // tracker for plane trackables.
XrTrackableTrackerANDROID depthTracker; // tracker for depth trackables.

// Perform a raycast against multiple trackers.
XrTrackableTrackerANDROID trackers[] = {
  &planeTracker,
  &depthTracker,
};
XrRaycastInfoANDROID rayInfo = {XR_TYPE_RAYCAST_INFO_ANDROID};
rayInfo.trackerCount = sizeof(trackers) / sizeof(XrTrackableTrackerANDROID);
rayInfo.trackers = trackers;
rayInfo.origin = headPose.position;
rayInfo.trajectory = CalculateForwardDirectionFromHeadPose(headPose);
rayInfo.space = appSpace;
rayInfo.time = updateTime;

uint32_t totalHitResults = 0;
constexpr uint32 NUM_DESIRED_RESULTS = 2;
XrRaycastHitResultANDROID hitResult[NUM_DESIRED_RESULTS];
XrRaycastHitResultsANDROID hitResults = {XR_TYPE_RAYCAST_HIT_RESULTS_ANDROID};
hitResults.maxResults = NUM_DESIRED_RESULTS;
hitResults.resultsCapacityInput = NUM_DESIRED_RESULTS;
hitResults.results = hitResult;
XrResult result = xrRaycastANDROID(session, &rayInfo, &hitResults);

if (result == XR_SUCCESS && hitResults.resultsCountOutput >= 1) {
  // Hit results are returned in closest-to-farthest order in
  // hitResults.results[0] .. hitResults.results[hitResults.resultsCountOutput - 1]
}

New Enum Constants

XrStructureType enumeration is extended with:

• XR_TYPE_RAYCAST_INFO_ANDROID
• XR_TYPE_RAYCAST_HIT_RESULTS_ANDROID

New Structures

• XrRaycastInfoANDROID
• XrRaycastHitResultANDROID
• XrRaycastHitResultsANDROID

New Functions

• xrEnumerateRaycastSupportedTrackableTypesANDROID
• xrRaycastANDROID

Issues

Version History

• Revision 1, 2024-10-02 (Kenny Vercaemer)
  • Initial extension description