ImageAnalysis

class ImageAnalysis : UseCase

kotlin.Any
   ↳ androidx.camera.core.UseCase
     ↳ androidx.camera.core.ImageAnalysis

A use case providing CPU accessible images for an app to perform image analysis on.

ImageAnalysis acquires images from the camera via an ImageReader. Each image is provided to an ImageAnalysis.Analyzer function which can be implemented by application code, where it can access image data for application analysis via an ImageProxy.

The application is responsible for calling close to close the image. Failing to close the image will cause future images to be stalled or dropped depending on the backpressure strategy.

Summary

Nested types

Interface for analyzing images.

Builder for a ImageAnalysis.

Constants
const Int

ImageAnalysis.Analyzer option for returning the original coordinates.
const Int

ImageAnalysis.Analyzer option for returning the sensor coordinates.
const Int

ImageAnalysis.Analyzer option for returning UI coordinates.
const Int

Images sent to the analyzer will have RGBA format.
const Int

Images sent to the analyzer will have YUV format.
const Int

Block the producer from generating new images.
const Int

Only deliver the latest image to the analyzer, dropping images as they arrive.

Public functions
Unit

Removes a previously set analyzer.
Executor?

Returns the executor that will be used for background tasks.
Int

Returns the mode with which images are acquired from the image producer.
Int

Returns the number of images available to the camera pipeline, including the image being analyzed, for the STRATEGY_BLOCK_PRODUCER backpressure mode.
Int

Gets output image format.
ResolutionInfo?

Gets resolution related information of the ImageAnalysis.
ResolutionSelector?

Returns the resolution selector setting.
Int

Returns the rotation of the intended target for images.
Boolean

Checks if output image rotation is enabled.
Unit

Sets an analyzer to receive and analyze images.
Unit

Sets the target rotation.
String

Inherited functions
From androidx.camera.core.UseCase
java-static Int
snapToSurfaceRotation(orientation: @IntRange(from = 0, to = 359) Int)

A utility function that can convert the orientation degrees of OrientationEventListener to the nearest Surface rotation.

Constants

COORDINATE_SYSTEM_ORIGINAL

Added in 1.1.0
const val COORDINATE_SYSTEM_ORIGINAL = 0: Int

ImageAnalysis.Analyzer option for returning the original coordinates.

Use this option if no additional transformation is needed by the Analyzer implementation. The coordinates returned by the Analyzer should be within (0, 0) - (width, height) where width and height are the dimensions of the ImageProxy.

By using this option, CameraX will pass null to updateTransform.

COORDINATE_SYSTEM_SENSOR

Added in 1.4.0
const val COORDINATE_SYSTEM_SENSOR = 2: Int

ImageAnalysis.Analyzer option for returning the sensor coordinates.

Use this option if the app wishes to get the detected objects in camera sensor coordinates. The coordinates returned by the Analyzer should be within (left, right) - (width, height), where the left, right, width and height are bounds of the camera sensor's active array.

By using this option, CameraX will pass getSensorToBufferTransformMatrix's inverse to updateTransform.

COORDINATE_SYSTEM_VIEW_REFERENCED

Added in 1.4.0
const val COORDINATE_SYSTEM_VIEW_REFERENCED = 1: Int

ImageAnalysis.Analyzer option for returning UI coordinates.

When the ImageAnalysis.Analyzer is configured with this option, it will receive a Matrix that will receive a value that represents the transformation from camera sensor to the View, which can be used for highlighting detected result in UI. For example, laying over a bounding box on top of the detected face.

Note this option will only work with an artifact that displays the camera feed in UI. Generally, this is used by higher-level libraries such as the CameraController API that incorporates a viewfinder UI. It will not be effective when used with camera-core directly.

See also
ImageAnalysis.Analyzer

OUTPUT_IMAGE_FORMAT_RGBA_8888

Added in 1.1.0
const val OUTPUT_IMAGE_FORMAT_RGBA_8888 = 2: Int

Images sent to the analyzer will have RGBA format.

All ImageProxy sent to analyze will have format RGBA_8888

The output order is a single-plane with the order of R, G, B, A in increasing byte index in the java.nio.ByteBuffer. The java.nio.ByteBuffer is retrieved from getBuffer.

See also
setOutputImageFormat

OUTPUT_IMAGE_FORMAT_YUV_420_888

Added in 1.1.0
const val OUTPUT_IMAGE_FORMAT_YUV_420_888 = 1: Int

Images sent to the analyzer will have YUV format.

All ImageProxy sent to analyze will have format YUV_420_888

See also
setOutputImageFormat

STRATEGY_BLOCK_PRODUCER

Added in 1.0.0
const val STRATEGY_BLOCK_PRODUCER = 1: Int

Block the producer from generating new images.

Once the producer has produced the number of images equal to the image queue depth, and none have been closed, the producer will stop producing images. Note that images may be queued internally and not be delivered to the analyzer until the last delivered image has been closed with close. These internally queued images will count towards the total number of images that the producer can provide at any one time.

When the producer stops producing images, it may also stop producing images for other use cases, such as Preview, so it is important for the analyzer to keep up with frame rate, on average. Failure to keep up with frame rate may lead to jank in the frame stream and a diminished user experience. If more time is needed for analysis on some frames, consider increasing the image queue depth with setImageQueueDepth.

See also
setImageQueueDepth

STRATEGY_KEEP_ONLY_LATEST

Added in 1.0.0
const val STRATEGY_KEEP_ONLY_LATEST = 0: Int

Only deliver the latest image to the analyzer, dropping images as they arrive.

This strategy ignores the value set by setImageQueueDepth. Only one image will be delivered for analysis at a time. If more images are produced while that image is being analyzed, they will be dropped and not queued for delivery. Once the image being analyzed is closed by calling close, the next latest image will be delivered.

Internally this strategy may make use of an internal Executor to receive and drop images from the producer. A performance-tuned executor will be created internally unless one is explicitly provided by setBackgroundExecutor. In order to ensure smooth operation of this backpressure strategy, any user supplied Executor must be able to quickly respond to tasks posted to it, so setting the executor manually should only be considered in advanced use cases.

See also
setBackgroundExecutor

Public functions

clearAnalyzer

Added in 1.0.0
fun clearAnalyzer(): Unit

Removes a previously set analyzer.

This will stop data from streaming to the ImageAnalysis.

getBackgroundExecutor

Added in 1.1.0
@ExperimentalUseCaseApi
fun getBackgroundExecutor(): Executor?

Returns the executor that will be used for background tasks.

Returns
Executor?

The Executor provided to setBackgroundExecutor. If no Executor has been provided, then returns null

getBackpressureStrategy

Added in 1.0.0
fun getBackpressureStrategy(): Int

Returns the mode with which images are acquired from the image producer.

The backpressure strategy is set when constructing an ImageAnalysis instance using setBackpressureStrategy. If not set, it defaults to STRATEGY_KEEP_ONLY_LATEST.

Returns
Int

The backpressure strategy applied to the image producer.
See also
setBackpressureStrategy

getImageQueueDepth

Added in 1.0.0
fun getImageQueueDepth(): Int

Returns the number of images available to the camera pipeline, including the image being analyzed, for the STRATEGY_BLOCK_PRODUCER backpressure mode.

The image queue depth is set when constructing an ImageAnalysis instance using setImageQueueDepth. If not set, and this option is used by the backpressure strategy, the default will be a queue depth of 6 images.

Returns
Int

The image queue depth for the STRATEGY_BLOCK_PRODUCER backpressure mode.
See also
setImageQueueDepth
setBackpressureStrategy

getOutputImageFormat

Added in 1.1.0
fun getOutputImageFormat(): Int

Gets output image format.

The returned image format will be OUTPUT_IMAGE_FORMAT_YUV_420_888 or OUTPUT_IMAGE_FORMAT_RGBA_8888.

Returns
Int

output image format.
See also
setOutputImageFormat

getResolutionInfo

Added in 1.1.0
fun getResolutionInfo(): ResolutionInfo?

Gets resolution related information of the ImageAnalysis.

The returned ResolutionInfo will be expressed in the coordinates of the camera sensor. It will be the same as the resolution of the ImageProxy received from analyze.

The resolution information might change if the use case is unbound and then rebound or setTargetRotation is called to change the target rotation setting. The application needs to call getResolutionInfo again to get the latest ResolutionInfo for the changes.

Returns
ResolutionInfo?

the resolution information if the use case has been bound by the bindToLifecycle API, or null if the use case is not bound yet.

getResolutionSelector

Added in 1.3.0
fun getResolutionSelector(): ResolutionSelector?

Returns the resolution selector setting.

This setting is set when constructing an ImageAnalysis using setResolutionSelector.

getTargetRotation

Added in 1.0.0
fun getTargetRotation(): Int

Returns the rotation of the intended target for images.

The rotation can be set when constructing an ImageAnalysis instance using setTargetRotation, or dynamically by calling setTargetRotation. If not set, the target rotation defaults to the value of getRotation of the default display at the time the use case is created. The use case is fully created once it has been attached to a camera.

Returns
Int

The rotation of the intended target for images.
See also
setTargetRotation

isOutputImageRotationEnabled

Added in 1.1.0
fun isOutputImageRotationEnabled(): Boolean

Checks if output image rotation is enabled. It returns false by default.

Returns
Boolean

true if enabled, false otherwise.
See also
setOutputImageRotationEnabled

setAnalyzer

Added in 1.0.0
fun setAnalyzer(executor: Executor, analyzer: ImageAnalysis.Analyzer): Unit

Sets an analyzer to receive and analyze images.

Setting an analyzer will signal to the camera that it should begin sending data. The stream of data can be stopped by calling clearAnalyzer.

Applications can process or copy the image by implementing the Analyzer. If frames should be skipped (no analysis), the analyzer function should return, instead of disconnecting the analyzer function completely.

Setting an analyzer function replaces any previous analyzer. Only one analyzer can be set at any time.

Parameters
executor: Executor

The executor in which the analyze will be run.
analyzer: ImageAnalysis.Analyzer

of the images.

setTargetRotation

Added in 1.0.0
fun setTargetRotation(rotation: Int): Unit

Sets the target rotation.

This adjust the getRotationDegrees of the ImageProxy passed to analyze. The rotation value of ImageInfo will be the rotation, which if applied to the output image, will make the image match target rotation specified here.

While rotation can also be set via setTargetRotation, using setTargetRotation allows the target rotation to be set dynamically.

In general, it is best to use an android.view.OrientationEventListener to set the target rotation. This way, the rotation output to the Analyzer will indicate which way is down for a given image. This is important since display orientation may be locked by device default, user setting, or app configuration, and some devices may not transition to a reverse-portrait display orientation. In these cases, set target rotation dynamically according to the android.view.OrientationEventListener, without re-creating the use case. snapToSurfaceRotation is a helper function to convert the orientation of the android.view.OrientationEventListener to a rotation value. See snapToSurfaceRotation for more information and sample code.

When this function is called, value set by setTargetResolution will be updated automatically to make sure the suitable resolution can be selected when the use case is bound.

If not set here or by configuration, the target rotation will default to the value of getRotation of the default display at the time the use case is bound. To return to the default value, set the value to 

context.getSystemService(WindowManager.class).getDefaultDisplay().getRotation();
Parameters
rotation: Int

Target rotation of the output image, expressed as one of ROTATION_0, ROTATION_90, ROTATION_180, or ROTATION_270.

toString

fun toString(): String