同时使用多个摄像头画面

注意：本页介绍的是 Camera2 软件包。除非您的应用需要 Camera2 中的特定低级别功能，否则我们建议使用 CameraX。CameraX 和 Camera2 都支持 Android 5.0（API 级别 21）及更高版本。

相机应用可以同时使用多个帧流。在某些情况下，不同的视频流甚至需要不同的帧分辨率或像素格式。一些典型用例包括：

• 视频录制：一个用于预览，另一个用于编码并保存到文件中。
• 条形码扫描：一个用于预览，另一个用于条形码检测。
• 计算摄影：一个用于预览，另一个用于人脸/场景检测。

处理帧时会产生相当大的性能开销，而进行并行流或流水线处理时，开销会成倍增加。

CPU、GPU 和 DSP 等资源可能能够利用框架的预处理功能，但内存等资源会线性增长。

每个请求包含多个目标

多个摄像头画面可以合并为一个 CameraCaptureRequest。 以下代码段展示了如何设置一个用于相机预览的流和一个用于图像处理的流，以创建相机工作会话：

Kotlin

val session: CameraCaptureSession = ...  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
val requestTemplate = CameraDevice.TEMPLATE_PREVIEW
val combinedRequest = session.device.createCaptureRequest(requestTemplate)

// Link the Surface targets with the combined request
combinedRequest.addTarget(previewSurface)
combinedRequest.addTarget(imReaderSurface)

// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
session.setRepeatingRequest(combinedRequest.build(), null, null)

Java

CameraCaptureSession session = …;  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
        CaptureRequest.Builder combinedRequest = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);

// Link the Surface targets with the combined request
        combinedRequest.addTarget(previewSurface);
        combinedRequest.addTarget(imReaderSurface);

// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
        session.setRepeatingRequest(combinedRequest.build(), null, null);

如果您正确配置了目标界面，此代码将仅生成满足由 StreamComfigurationMap.GetOutputMinFrameDuration(int, Size) 和 StreamComfigurationMap.GetOutputStallDuration(int, Size) 确定的最低 FPS 的视频流。实际性能因设备而异，不过 Android 针对支持特定组合提供了一些保证，具体取决于三个变量：输出类型、输出大小和硬件级别。

使用不支持的变量组合可能会以低帧速率运行；如果无法运行，则会触发其中一个失败回调。 createCaptureSession 的文档介绍了保证可正常运行的内容。

输出类型

输出类型是指帧的编码格式。可能的值包括 PRIV、YUV、JPEG 和 RAW。createCaptureSession 的文档中介绍了这些参数。

选择应用的输出类型时，如果目标是最大限度地提高兼容性，请使用 ImageFormat.YUV_420_888 进行帧分析，并使用 ImageFormat.JPEG 处理静态图片。对于预览和录制场景，您可能会使用 SurfaceView、TextureView、MediaRecorder、MediaCodec 或 RenderScript.Allocation。在这种情况下，请勿指定图片格式。为了实现兼容性，无论内部实际使用的格式如何，它都将计为 ImageFormat.PRIVATE。如需查询设备根据其 CameraCharacteristics 支持的格式，请使用以下代码：

Kotlin

val characteristics: CameraCharacteristics = ...
val supportedFormats = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).outputFormats

Java

CameraCharacteristics characteristics = …;
        int[] supportedFormats = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).getOutputFormats();

输出大小

所有可用的输出大小均按 StreamConfigurationMap.getOutputSizes() 列出，但只有两个与兼容性相关：PREVIEW 和 MAXIMUM。这些大小充当上限。如果大小为 PREVIEW 的某事物可行，那么大小小于 PREVIEW 的任何事物也可行。MAXIMUM 也是如此。CameraDevice 的文档中介绍了这些大小。

可用的输出大小取决于所选格式。给定 CameraCharacteristics 和一种格式，您可以按如下方式查询可用的输出尺寸：

Kotlin

val characteristics: CameraCharacteristics = ...
val outputFormat: Int = ...  // such as ImageFormat.JPEG
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(outputFormat)

Java

CameraCharacteristics characteristics = …;
        int outputFormat = …;  // such as ImageFormat.JPEG
Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

在摄像头预览和录制用例中，使用目标类来确定支持的尺寸。该格式将由相机框架本身处理：

Kotlin

val characteristics: CameraCharacteristics = ...
val targetClass: Class <T> = ...  // such as SurfaceView::class.java
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(targetClass)

Java

CameraCharacteristics characteristics = …;
   int outputFormat = …;  // such as ImageFormat.JPEG
   Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

如需获取 MAXIMUM 大小，请按面积对输出大小进行排序，然后返回最大的一个：

Kotlin

fun <T>getMaximumOutputSize(
    characteristics: CameraCharacteristics, targetClass: Class <T>, format: Int? = null):
    Size {
  val config = characteristics.get(
      CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)

  // If image format is provided, use it to determine supported sizes; or else use target class
  val allSizes = if (format == null)
    config.getOutputSizes(targetClass) else config.getOutputSizes(format)
  return allSizes.maxBy { it.height * it.width }
}

Java

 @RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getMaximumOutputSize(CameraCharacteristics characteristics,
                                            Class <T> targetClass,
                                            Integer format) {
        StreamConfigurationMap config = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);

        // If image format is provided, use it to determine supported sizes; else use target class
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }
        return Arrays.stream(allSizes).max(Comparator.comparing(s -> s.getHeight() * s.getWidth())).get();
    }

PREVIEW 指的是与设备的屏幕分辨率或 1080p (1920x1080) 匹配的最佳尺寸（以较低者为准）。宽高比可能与屏幕的宽高比不完全一致，因此您可能需要对视频流应用信箱模式或裁剪，才能以全屏模式显示视频流。为了获得合适的预览尺寸，请比较可用的输出尺寸与显示尺寸，同时考虑到显示屏可能会旋转。

以下代码定义了一个辅助类 SmartSize，该类可让大小比较变得更简单：

Kotlin

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
class SmartSize(width: Int, height: Int) {
    var size = Size(width, height)
    var long = max(size.width, size.height)
    var short = min(size.width, size.height)
    override fun toString() = "SmartSize(${long}x${short})"
}

/** Standard High Definition size for pictures and video */
val SIZE_1080P: SmartSize = SmartSize(1920, 1080)

/** Returns a [SmartSize] object for the given [Display] */
fun getDisplaySmartSize(display: Display): SmartSize {
    val outPoint = Point()
    display.getRealSize(outPoint)
    return SmartSize(outPoint.x, outPoint.y)
}

/**
 * Returns the largest available PREVIEW size. For more information, see:
 * https://d.android.com/reference/android/hardware/camera2/CameraDevice
 */
fun <T>getPreviewOutputSize(
        display: Display,
        characteristics: CameraCharacteristics,
        targetClass: Class <T>,
        format: Int? = null
): Size {

    // Find which is smaller: screen or 1080p
    val screenSize = getDisplaySmartSize(display)
    val hdScreen = screenSize.long >= SIZE_1080P.long || screenSize.short >= SIZE_1080P.short
    val maxSize = if (hdScreen) SIZE_1080P else screenSize

    // If image format is provided, use it to determine supported sizes; else use target class
    val config = characteristics.get(
            CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)!!
    if (format == null)
        assert(StreamConfigurationMap.isOutputSupportedFor(targetClass))
    else
        assert(config.isOutputSupportedFor(format))
    val allSizes = if (format == null)
        config.getOutputSizes(targetClass) else config.getOutputSizes(format)

    // Get available sizes and sort them by area from largest to smallest
    val validSizes = allSizes
            .sortedWith(compareBy { it.height * it.width })
            .map { SmartSize(it.width, it.height) }.reversed()

    // Then, get the largest output size that is smaller or equal than our max size
    return validSizes.first { it.long <= maxSize.long && it.short <= maxSize.short }.size
}

Java

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
    class SmartSize {
        Size size;
        double longSize;
        double shortSize;

        public SmartSize(Integer width, Integer height) {
            size = new Size(width, height);
            longSize = max(size.getWidth(), size.getHeight());
            shortSize = min(size.getWidth(), size.getHeight());
        }

        @Override
        public String toString() {
            return String.format("SmartSize(%sx%s)", longSize, shortSize);
        }
    }

    /** Standard High Definition size for pictures and video */
    SmartSize SIZE_1080P = new SmartSize(1920, 1080);

    /** Returns a [SmartSize] object for the given [Display] */
    SmartSize getDisplaySmartSize(Display display) {
        Point outPoint = new Point();
        display.getRealSize(outPoint);
        return new SmartSize(outPoint.x, outPoint.y);
    }

    /**
     * Returns the largest available PREVIEW size. For more information, see:
     * https://d.android.com/reference/android/hardware/camera2/CameraDevice
     */
    @RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getPreviewOutputSize(
            Display display,
            CameraCharacteristics characteristics,
            Class <T> targetClass,
            Integer format
    ){

        // Find which is smaller: screen or 1080p
        SmartSize screenSize = getDisplaySmartSize(display);
        boolean hdScreen = screenSize.longSize >= SIZE_1080P.longSize || screenSize.shortSize >= SIZE_1080P.shortSize;
        SmartSize maxSize;
        if (hdScreen) {
            maxSize = SIZE_1080P;
        } else {
            maxSize = screenSize;
        }

        // If image format is provided, use it to determine supported sizes; else use target class
        StreamConfigurationMap config = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
        if (format == null)
            assert(StreamConfigurationMap.isOutputSupportedFor(targetClass));
        else
            assert(config.isOutputSupportedFor(format));
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }

        // Get available sizes and sort them by area from largest to smallest
        List <Size> sortedSizes = Arrays.asList(allSizes);
        List <SmartSize> validSizes =
                sortedSizes.stream()
                        .sorted(Comparator.comparing(s -> s.getHeight() * s.getWidth()))
                        .map(s -> new SmartSize(s.getWidth(), s.getHeight()))
                        .sorted(Collections.reverseOrder()).collect(Collectors.toList());

        // Then, get the largest output size that is smaller or equal than our max size
        return validSizes.stream()
                .filter(s -> s.longSize <= maxSize.longSize && s.shortSize <= maxSize.shortSize)
                .findFirst().get().size;
    }

检查支持的硬件级别

如需在运行时确定可用的功能，请使用 CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL 检查受支持的硬件级别。

借助 CameraCharacteristics 对象，您可以使用一条语句检索硬件级别：

Kotlin

val characteristics: CameraCharacteristics = ...

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
val hardwareLevel = characteristics.get(
        CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL)

Java

CameraCharacteristics characteristics = ...;

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
Integer hardwareLevel = characteristics.get(
                CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);

将所有内容整合在一起

通过输出类型、输出大小和硬件级别，您可以确定哪些流组合有效。下图显示了具有 LEGACY 硬件级别的 CameraDevice 支持的配置。

目标 1		目标 2		目标 3		使用场景示例
类型	最大大小	类型	最大大小	类型	最大大小
PRIV	MAXIMUM					简单预览、GPU 视频处理或无预览视频录制。
JPEG	MAXIMUM					无取景器静态图片拍摄。
YUV	MAXIMUM					应用内视频/图片处理。
PRIV	PREVIEW	JPEG	MAXIMUM			标准静态成像。
YUV	PREVIEW	JPEG	MAXIMUM			应用内处理和拍摄。
PRIV	PREVIEW	PRIV	PREVIEW			标准录制。
PRIV	PREVIEW	YUV	PREVIEW			预览和应用内处理。
PRIV	PREVIEW	YUV	PREVIEW	JPEG	MAXIMUM	静态拍摄加应用内处理。

LEGACY 是最低可能的硬件级别。此表显示，支持 Camera2（API 级别 21 及更高级别）的每部设备都可以使用正确的配置同时输出最多三个数据流，前提是没有过多的开销限制性能，例如内存、CPU 或散热限制。

您的应用还需要配置目标输出缓冲区。例如，若要以硬件级别为 LEGACY 的设备为目标，您可以设置两个目标输出界面，一个使用 ImageFormat.PRIVATE，另一个使用 ImageFormat.YUV_420_888。这是使用 PREVIEW 大小时支持的组合。使用本主题前面定义的函数，获取相机 ID 所需的预览大小需要以下代码：

Kotlin

val characteristics: CameraCharacteristics = ...
val context = this as Context  // assuming you are inside of an activity

val surfaceViewSize = getPreviewOutputSize(
    context, characteristics, SurfaceView::class.java)
val imageReaderSize = getPreviewOutputSize(
    context, characteristics, ImageReader::class.java, format = ImageFormat.YUV_420_888)

Java

CameraCharacteristics characteristics = ...;
        Context context = this; // assuming you are inside of an activity

        Size surfaceViewSize = getPreviewOutputSize(
                context, characteristics, SurfaceView.class);
        Size imageReaderSize = getPreviewOutputSize(
                context, characteristics, ImageReader.class, format = ImageFormat.YUV_420_888);

它需要使用提供的回调等待 SurfaceView 准备就绪：

Kotlin

val surfaceView = findViewById <SurfaceView>(...)
surfaceView.holder.addCallback(object : SurfaceHolder.Callback {
  override fun surfaceCreated(holder: SurfaceHolder) {
    // You do not need to specify image format, and it will be considered of type PRIV
    // Surface is now ready and you could use it as an output target for CameraSession
  }
  ...
})

Java

SurfaceView surfaceView = findViewById <SurfaceView>(...);

surfaceView.getHolder().addCallback(new SurfaceHolder.Callback() {
            @Override
            public void surfaceCreated(@NonNull SurfaceHolder surfaceHolder) {
                // You do not need to specify image format, and it will be considered of type PRIV
                // Surface is now ready and you could use it as an output target for CameraSession
            }
            ...
        });

您可以调用 SurfaceHolder.setFixedSize() 强制使 SurfaceView 与相机输出大小保持一致，也可以采用与 GitHub 上相机示例的通用模块中的 AutoFitSurfaceView 类似的方法，该方法会设置绝对大小，同时考虑宽高比和可用空间，并在触发 activity 更改时自动调整。

从 ImageReader 设置具有所需格式的其他 Surface 更简单，因为无需等待任何回调：

Kotlin

val frameBufferCount = 3  // just an example, depends on your usage of ImageReader
val imageReader = ImageReader.newInstance(
    imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
    frameBufferCount)

Java

int frameBufferCount = 3;  // just an example, depends on your usage of ImageReader
ImageReader imageReader = ImageReader.newInstance(
                imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
                frameBufferCount);

使用 ImageReader 等阻塞目标缓冲区时，请在使用帧后将其舍弃：

Kotlin

imageReader.setOnImageAvailableListener({
  val frame =  it.acquireNextImage()
  // Do something with "frame" here
  it.close()
}, null)

Java

imageReader.setOnImageAvailableListener(listener -> {
            Image frame = listener.acquireNextImage();
            // Do something with "frame" here
            listener.close();
        }, null);

LEGACY 硬件级别以最通用的设备为目标。您可以添加条件分支，并针对硬件级别为 LIMITED 的设备，将其中一个输出目标表面的大小设置为 RECORD，甚至针对硬件级别为 FULL 的设备，将其大小增加到 MAXIMUM。