SurfaceProcessor

interface SurfaceProcessor

Interface to implement a GPU-based post-processing effect.

This interface is for implementing a GPU effect for the PreviewUseCase. Both the input and the output of the implementation are Surfaces. It's recommended to use graphics API such as OpenGL or Vulkan to access the Surface.

If the implementation fails to process frames, for example, fails to allocate the resources, it should throw a ProcessingException in either onInputSurface or onOutputSurface to notify CameraX. If the implementation encounters an error after the pipeline is running, it should invalidate the input Surface by calling invalidate, then throwing a ProcessingException when onInputSurface is invoked again.

Once the implementation throws an exception, CameraX will treat it as an unrecoverable error and abort the pipeline. If the getTargets is PREVIEW, CameraX will not propagate the error to the app. It's the implementation's responsibility to notify the app. For example: 

class SurfaceProcessorImpl implements SurfaceProcessor {

    ConsumermErrorListener;

    SurfaceProcessorImpl(@NonNull Consumer

Summary

Public functions
Unit

Invoked when CameraX requires an input Surface for reading original frames.
Unit

Invoked when CameraX provides output Surface(s) for drawing processed frames.

Public functions

onInputSurface

Added in 1.3.0
fun onInputSurface(request: SurfaceRequest): Unit

Invoked when CameraX requires an input Surface for reading original frames.

CameraX requests Surfaces when the upstream pipeline is reconfigured. For example, when UseCases are bound to lifecycle.

With OpenGL, the implementation should create a Surface backed by SurfaceTexture with the size of getResolution, then listen for the setOnFrameAvailableListener to get the incoming frames. The Surface should not be released until the callback provided in provideSurface is invoked. CameraX may request new input Surface before releasing the existing one.

If the implementation encounters errors in creating the input Surface, it should throw an ProcessingException to notify CameraX.

The implementation can replace a previously provided Surface by invoking invalidate. Once invoked, CameraX will restart the camera pipeline and call onInputSurface again with another SurfaceRequest.

The value of the getTransformMatrix will need an additional transformation. CameraX calculates the additional transformation based on UseCase configurations such as ViewPort and target rotation, and provide the value via updateTransformMatrix. Code sample: 

// Create Surface based on the request.
SurfaceTexture surfaceTexture = SurfaceTexture(textureName);
surfaceTexture.setDefaultBufferSize(request.resolution.width, request.resolution.height);
Surface surface = Surface(surfaceTexture);

// Provide the Surface to CameraX, and cleanup when it's no longer used.
surfaceRequest.provideSurface(surface, executor, result -> {
    surfaceTexture.setOnFrameAvailableListener(null)
    surfaceTexture.release()
    surface.release()
});

// Listen to the incoming frames.
surfaceTexture.setOnFrameAvailableListener(surfaceTexture -> {
    // Process the incoming frames and draw to the output Surface from #onOutputSurface
}, handler);
Parameters
request: SurfaceRequest

a request to provide Surface for input.
Throws
androidx.camera.core.ProcessingException

if the implementation fails to fulfill the SurfaceRequest.
See also
SurfaceRequest

onOutputSurface

Added in 1.3.0
fun onOutputSurface(surfaceOutput: SurfaceOutput): Unit

Invoked when CameraX provides output Surface(s) for drawing processed frames.

CameraX provides the output Surfaces when the downstream pipeline is reconfigured, for example, when UseCases are bound or the preview viewfinder is reset.

The provided Surfaces are for drawing processed frames. The implementation must get the Surface via getSurface and provide a < listening to the end-of-life event of the Surface. Then, the implementation should call close after it stops drawing to the Surface. CameraX may provide new output Surface before requesting to close the existing one.

If the implementation encounters an error and cannot consume the Surface, it should throw an ProcessingException to notify CameraX.

When drawing to the Surface, the implementation should apply an additional transformation to the input Surface by calling updateTransformMatrix with the value of getTransformMatrix} from the input Surface.

Parameters
surfaceOutput: SurfaceOutput

contains a Surface for drawing processed frames.
Throws
androidx.camera.core.ProcessingException

if the implementation fails to consume the SurfaceOutput.
See also
SurfaceOutput