ImageAnalysis.Analyzer

public interface ImageAnalysis.Analyzer

Known direct subclasses
MlKitAnalyzer

An implementation of ImageAnalysis.Analyzer with ML Kit libraries.

Interface for analyzing images.

Implement Analyzer and pass it to setAnalyzer to receive images and perform custom processing by implementing the analyze function.

Summary

Public methods
abstract void

Analyzes an image to produce a result.
default @Nullable Size

Implement this method to set a default target resolution for the ImageAnalysis.
default int

Implement this method to return the target coordinate system.
default void

Implement this method to receive the Matrix for coordinate transformation.

Public methods

analyze

Added in 1.0.0
abstract void analyze(@NonNull ImageProxy image)

Analyzes an image to produce a result.

This method is called once for each image from the camera, and called at the frame rate of the camera. Each analyze call is executed sequentially.

It is the responsibility of the application to close the image once done with it. If the images are not closed then it may block further images from being produced (causing the preview to stall) or drop images as determined by the configured backpressure strategy. The exact behavior is configurable via setBackpressureStrategy.

Images produced here will no longer be valid after the ImageAnalysis instance that produced it has been unbound from the camera.

The image provided has format YUV_420_888.

The provided image is typically in the orientation of the sensor, meaning CameraX does not perform an internal rotation of the data. The rotationDegrees parameter allows the analysis to understand the image orientation when processing or to apply a rotation. For example, if the target rotation) is natural orientation, rotationDegrees would be the rotation which would align the buffer data ordering to natural orientation.

Timestamps are in nanoseconds and monotonic and can be compared to timestamps from images produced from UseCases bound to the same camera instance. More detail is available depending on the implementation. For example with CameraX using a androidx.camera.camera2 implementation additional detail can be found in android.hardware.camera2.CameraDevice documentation.

Parameters
@NonNull ImageProxy image

The image to analyze
See also
getTimestamp
SENSOR_TIMESTAMP

getDefaultTargetResolution

Added in 1.2.0
default @Nullable Size getDefaultTargetResolution()

Implement this method to set a default target resolution for the ImageAnalysis.

Implement this method if the Analyzer requires a specific resolution to work. The return value will be used as the default target resolution for the ImageAnalysis. Return null if no falling back is needed. By default, this method returns null.

If the app does not set a target resolution for ImageAnalysis, then this value will be used as the target resolution. If the ImageAnalysis has set a target resolution, e.g. if setTargetResolution is called, then the ImageAnalysis will use the app value over this value.

Note that this method is invoked by CameraX at the time of binding to lifecycle. In order for this value to be effective, the Analyzer has to be set before ImageAnalysis is bound to a lifecycle. Otherwise, the value will be ignored.

Returns
@Nullable Size

the default resolution of ImageAnalysis, or null if no specific resolution is needed.

getTargetCoordinateSystem

Added in 1.1.0
default int getTargetCoordinateSystem()

Implement this method to return the target coordinate system.

The coordinates detected by analyzing camera frame usually needs to be transformed. For example, in order to highlight a detected face, the app needs to transform the bounding box from the ImageAnalysis's coordinate system to the View's coordinate system. This method allows the implementer to set a target coordinate system.

The value will be used by CameraX to calculate the transformation Matrix and forward it to the Analyzer via updateTransform. By default, this method returns COORDINATE_SYSTEM_ORIGINAL.

For now, camera-core only supports COORDINATE_SYSTEM_ORIGINAL, please see libraries derived from camera-core, for example, camera-view.

See also
updateTransform

updateTransform

Added in 1.1.0
default void updateTransform(@Nullable Matrix matrix)

Implement this method to receive the Matrix for coordinate transformation.

The value represents the transformation from the camera sensor to the target coordinate system defined in getTargetCoordinateSystem. It should be used by the implementation to transform the coordinates detected in the camera frame. For example, the coordinates of the detected face.

If the value is null, it means that no valid transformation is available. It could have different causes depending on the value of getTargetCoordinateSystem:

  • If the target coordinate system is COORDINATE_SYSTEM_ORIGINAL, it is always invalid because in that case, the coordinate system depends on how the analysis algorithm processes the ImageProxy.
  • It is also invalid if the target coordinate system is not available, for example if the analyzer targets the viewfinder and the view finder is not visible in UI.

This method is invoked whenever a new transformation is ready. For example, when the view finder is first a launched as well as when it's resized.

See also
getTargetCoordinateSystem