同时使用多个摄像头画面

注意：本页介绍的是 Camera2 软件包。除非您的应用需要 Camera2 的特定低级别功能，否则我们建议您使用 CameraX。CameraX 和 Camera2 都支持 Android 5.0（API 级别 21）及更高版本。

相机应用可以同时使用多个帧流。在某些情况下，不同的数据流甚至需要不同的帧分辨率或像素格式。一些典型用例包括：

• 视频录制：一个数据流用于预览，另一个数据流经过编码并保存到文件中。
• 条形码扫描：一个数据流用于预览，另一个数据流用于条形码检测。
• 计算摄影：一个数据流用于预览，另一个数据流用于人脸或场景检测。

处理帧时会产生不小的性能开销，而在进行并行流或流水线处理时，开销会成倍增加。

CPU、GPU 和 DSP 等资源或许能够利用框架的重新处理功能，但内存等资源会线性增长。

每个请求多个目标

多个摄像头视频传输可以合并为一个 CameraCaptureRequest。以下代码段展示了如何设置一个相机拍摄会话，其中一个数据流用于相机预览，另一个数据流用于图片处理：

Kotlin

val session: CameraCaptureSession = ...  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
val requestTemplate = CameraDevice.TEMPLATE_PREVIEW
val combinedRequest = session.device.createCaptureRequest(requestTemplate)

// Link the Surface targets with the combined request
combinedRequest.addTarget(previewSurface)
combinedRequest.addTarget(imReaderSurface)

// In this basic case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
session.setRepeatingRequest(combinedRequest.build(), null, null)

Java

CameraCaptureSession session = …;  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
        CaptureRequest.Builder combinedRequest = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);

// Link the Surface targets with the combined request
        combinedRequest.addTarget(previewSurface);
        combinedRequest.addTarget(imReaderSurface);

// In this basic case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
        session.setRepeatingRequest(combinedRequest.build(), null, null);

如果您正确配置了目标 Surface，此代码将仅生成满足 StreamComfigurationMap.GetOutputMinFrameDuration(int, Size) 和 StreamComfigurationMap.GetOutputStallDuration(int, Size) 确定的最低 FPS 的数据流。实际性能因设备而异，但 Android 会根据输出类型、输出大小和硬件级别这三个变量，为支持特定组合提供一些保证。

使用不受支持的变量组合可能会在低帧速率下运行；如果无法运行，则会触发其中一个失败回调。createCaptureSession 的文档介绍了保证可以正常运行的内容。

输出类型

输出类型是指帧的编码格式。可能的值包括 PRIV、YUV、JPEG 和 RAW。createCaptureSession 的文档对此进行了说明。

在选择应用的输出类型时，如果目标是最大限度地提高兼容性，请使用 ImageFormat.YUV_420_888 进行帧分析，并使用 ImageFormat.JPEG 处理静态图片。对于预览和录制场景，您可能会使用 SurfaceView、TextureView、MediaRecorder、MediaCodec 或 RenderScript.Allocation。在这些情况下，请勿指定图片格式。为实现兼容性，无论内部使用的实际格式如何，它都将计为 ImageFormat.PRIVATE。如需查询设备在给定 CameraCharacteristics 的情况下支持的格式，请使用以下代码：

Kotlin

val characteristics: CameraCharacteristics = ...
val supportedFormats = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).outputFormats

Java

CameraCharacteristics characteristics = …;
        int[] supportedFormats = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).getOutputFormats();

输出大小

所有可用的输出尺寸均由 StreamConfigurationMap.getOutputSizes() 列出，但只有两个与兼容性相关：PREVIEW 和 MAXIMUM。这些大小用作上限。如果大小为 PREVIEW 的内容有效，则大小小于 PREVIEW 的任何内容也有效。MAXIMUM 也是如此。CameraDevice 的文档介绍了这些大小。

可用的输出尺寸取决于所选格式。给定 CameraCharacteristics 和格式后，您可以按如下方式查询可用的输出大小：

Kotlin

val characteristics: CameraCharacteristics = ...
val outputFormat: Int = ...  // such as ImageFormat.JPEG
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(outputFormat)

Java

CameraCharacteristics characteristics = …;
        int outputFormat = …;  // such as ImageFormat.JPEG
Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

在相机预览和录制用例中，使用目标类来确定支持的尺寸。该格式将由相机框架本身处理：

Kotlin

val characteristics: CameraCharacteristics = ...
val targetClass: Class<T> = ...  // such as SurfaceView::class.java
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(targetClass)

Java

CameraCharacteristics characteristics = …;
   int outputFormat = …;  // such as ImageFormat.JPEG
   Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

如需获取 MAXIMUM 尺寸，请按面积对输出尺寸进行排序，并返回最大尺寸：

Kotlin

fun<T> getMaximumOutputSize(
    characteristics: CameraCharacteristics, targetClass: Class<T>, format: Int? = null):
    Size {
  val config = characteristics.get(
      CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)

  // If image format is provided, use it to determine supported sizes; or else use target class
  val allSizes = if (format == null)
    config.getOutputSizes(targetClass) else config.getOutputSizes(format)
  return allSizes.maxBy { it.height * it.width }
}

Java

@RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getMaximumOutputSize(CameraCharacteristics characteristics,
                                            Class<T> targetClass,
                                            Integer format) {
        StreamConfigurationMap config = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);

        // If image format is provided, use it to determine supported sizes; else use target class
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }
        return Arrays.stream(allSizes).max(Comparator.comparing(s -> s.getHeight() * s.getWidth())).get();
    }

PREVIEW 指的是与设备的屏幕分辨率或 1080p (1920x1080)（以较低者为准）匹配的最佳尺寸。宽高比可能与屏幕的宽高比不完全匹配，因此您可能需要对视频流应用黑边或剪裁，才能以全屏模式显示。如需获取正确的预览尺寸，请将可用的输出尺寸与显示尺寸进行比较，同时考虑到显示屏可能会旋转。

以下代码定义了一个辅助类 SmartSize，该类可使尺寸比较变得更简单：

Kotlin

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
class SmartSize(width: Int, height: Int) {
    var size = Size(width, height)
    var long = max(size.width, size.height)
    var short = min(size.width, size.height)
    override fun toString() = "SmartSize(${long}x${short})"
}

/** Standard High Definition size for pictures and video */
val SIZE_1080P: SmartSize = SmartSize(1920, 1080)

/** Returns a [SmartSize] object for the given [Display] */
fun getDisplaySmartSize(display: Display): SmartSize {
    val outPoint = Point()
    display.getRealSize(outPoint)
    return SmartSize(outPoint.x, outPoint.y)
}

/**
 * Returns the largest available PREVIEW size. For more information, see:
 * https://d.android.com/reference/android/hardware/camera2/CameraDevice
 */
fun<T> getPreviewOutputSize(
        display: Display,
        characteristics: CameraCharacteristics,
        targetClass: Class<T>,
        format: Int? = null
): Size {

    // Find which is smaller: screen or 1080p
    val screenSize = getDisplaySmartSize(display)
    val hdScreen = screenSize.long >= SIZE_1080P.long || screenSize.short >= SIZE_1080P.short
    val maxSize = if (hdScreen) SIZE_1080P else screenSize

    // If image format is provided, use it to determine supported sizes; else use target class
    val config = characteristics.get(
            CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)!!
    if (format == null)
        assert(StreamConfigurationMap.isOutputSupportedFor(targetClass))
    else
        assert(config.isOutputSupportedFor(format))
    val allSizes = if (format == null)
        config.getOutputSizes(targetClass) else config.getOutputSizes(format)

    // Get available sizes and sort them by area from largest to smallest
    val validSizes = allSizes
            .sortedWith(compareBy { it.height * it.width })
            .map { SmartSize(it.width, it.height) }.reversed()

    // Then, get the largest output size that is smaller or equal than our max size
    return validSizes.first { it.long <= maxSize.long && it.short <= maxSize.short }.size
}

Java

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
class SmartSize {
    Size size;
    double longSize;
    double shortSize;

    public SmartSize(Integer width, Integer height) {
        size = new Size(width, height);
        longSize = max(size.getWidth(), size.getHeight());
        shortSize = min(size.getWidth(), size.getHeight());
    }

    @Override
    public String toString() {
        return String.format("SmartSize(%sx%s)", longSize, shortSize);
    }
}

/** Standard High Definition size for pictures and video */
SmartSize SIZE_1080P = new SmartSize(1920, 1080);

/** Returns a [SmartSize] object for the given [Display] */
SmartSize getDisplaySmartSize(Display display) {
    Point outPoint = new Point();
    display.getRealSize(outPoint);
    return new SmartSize(outPoint.x, outPoint.y);
}

/**
 * Returns the largest available PREVIEW size. For more information, see:
 * https://d.android.com/reference/android/hardware/camera2/CameraDevice
 */
@RequiresApi(api = Build.VERSION_CODES.N)
<T> Size getPreviewOutputSize(
        Display display,
        CameraCharacteristics characteristics,
        Class<T> targetClass,
        Integer format
){

    // Find which is smaller: screen or 1080p
    SmartSize screenSize = getDisplaySmartSize(display);
    boolean hdScreen = screenSize.longSize >= SIZE_1080P.longSize || screenSize.shortSize >= SIZE_1080P.shortSize;
    SmartSize maxSize;
    if (hdScreen) {
        maxSize = SIZE_1080P;
    } else {
        maxSize = screenSize;
    }

    // If image format is provided, use it to determine supported sizes; else use target class
    StreamConfigurationMap config = characteristics.get(
            CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
    if (format == null)
        assert(StreamConfigurationMap.isOutputSupportedFor(targetClass));
    else
        assert(config.isOutputSupportedFor(format));
    Size[] allSizes;
    if (format == null) {
        allSizes = config.getOutputSizes(targetClass);
    } else {
        allSizes = config.getOutputSizes(format);
    }

    // Get available sizes and sort them by area from largest to smallest
    List<Size> sortedSizes = Arrays.asList(allSizes);
    List<SmartSize> validSizes =
            sortedSizes.stream()
                    .sorted(Comparator.comparing(s -> s.getHeight() * s.getWidth()))
                    .map(s -> new SmartSize(s.getWidth(), s.getHeight()))
                    .sorted(Collections.reverseOrder()).collect(Collectors.toList());

    // Then, get the largest output size that is smaller or equal than our max size
    return validSizes.stream()
            .filter(s -> s.longSize <= maxSize.longSize && s.shortSize <= maxSize.shortSize)
            .findFirst().get().size;
}

检查支持的硬件级别

如需确定运行时可用的功能，请使用 CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL 检查支持的硬件级别。

借助 CameraCharacteristics 对象，您可以使用单个语句检索硬件级别：

Kotlin

val characteristics: CameraCharacteristics = ...

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
val hardwareLevel = characteristics.get(
        CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL)

Java

CameraCharacteristics characteristics = ...;

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
Integer hardwareLevel = characteristics.get(
                CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);

整合所有部分

通过输出类型、输出大小和硬件级别，您可以确定哪些数据流组合有效。下图是具有 LEGACY 硬件级别的 CameraDevice 支持的配置的快照。

目标 1		目标 2		目标 3		示例用例
类型	最大大小	类型	最大大小	类型	最大大小
PRIV	MAXIMUM					简单预览、GPU 视频处理或无预览视频录制。
JPEG	MAXIMUM					无取景器静态图片拍摄。
YUV	MAXIMUM					应用内视频/图片处理。
PRIV	PREVIEW	JPEG	MAXIMUM			标准静态图片。
YUV	PREVIEW	JPEG	MAXIMUM			应用内处理和静态拍摄。
PRIV	PREVIEW	PRIV	PREVIEW			标准录制。
PRIV	PREVIEW	YUV	PREVIEW			预览和应用内处理。
PRIV	PREVIEW	YUV	PREVIEW	JPEG	MAXIMUM	静态拍摄和应用内处理。

LEGACY 是可能的最低硬件级别。此表显示，如果配置正确，并且没有太多限制性能的开销（例如内存、CPU 或热量限制），则每个支持 Camera2（API 级别 21 及更高级别）的设备最多可以同时输出三个数据流。

您的应用还需要配置定位输出缓冲区。例如，如需定位具有 LEGACY 硬件级别的设备，您可以设置两个目标输出 Surface，一个使用 ImageFormat.PRIVATE，另一个使用 ImageFormat.YUV_420_888。使用 PREVIEW 大小时，此组合受支持。使用本文档前面定义的函数，获取相机 ID 所需的预览尺寸需要以下代码：

Kotlin

val characteristics: CameraCharacteristics = ...
val context = this as Context  // assuming you are inside of an activity

val surfaceViewSize = getPreviewOutputSize(
    context, characteristics, SurfaceView::class.java)
val imageReaderSize = getPreviewOutputSize(
    context, characteristics, ImageReader::class.java, format = ImageFormat.YUV_420_888)

Java

CameraCharacteristics characteristics = ...;
        Context context = this; // assuming you are inside of an activity

        Size surfaceViewSize = getPreviewOutputSize(
                context, characteristics, SurfaceView.class);
        Size imageReaderSize = getPreviewOutputSize(
                context, characteristics, ImageReader.class, format = ImageFormat.YUV_420_888);

它需要使用提供的回调等待 SurfaceView 准备就绪：

Kotlin

val surfaceView = findViewById<SurfaceView>(...)
surfaceView.holder.addCallback(object : SurfaceHolder.Callback {
  override fun surfaceCreated(holder: SurfaceHolder) {
    // You do not need to specify image format, and it will be considered of type PRIV
    // Surface is now ready and you could use it as an output target for CameraSession
  }
  ...
})

Java

SurfaceView surfaceView = findViewById<SurfaceView>(...);

surfaceView.getHolder().addCallback(new SurfaceHolder.Callback() {
            @Override
            public void surfaceCreated(@NonNull SurfaceHolder surfaceHolder) {
                // You do not need to specify image format, and it will be considered of type PRIV
                // Surface is now ready and you could use it as an output target for CameraSession
            }
            ...
        });

您可以通过调用 SurfaceHolder.setFixedSize() 强制 SurfaceView 与相机输出大小匹配，也可以采用与 GitHub 上的相机示例的通用模块中的 AutoFitSurfaceView 类似的方法，该方法会设置绝对大小，同时考虑宽高比和可用空间，并在触发 activity 更改时自动调整。

使用所选格式从 ImageReader 设置其他 Surface 会更容易，因为无需等待回调：

Kotlin

val frameBufferCount = 3  // just an example, depends on your usage of ImageReader
val imageReader = ImageReader.newInstance(
    imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
    frameBufferCount)

Java

int frameBufferCount = 3;  // just an example, depends on your usage of ImageReader
ImageReader imageReader = ImageReader.newInstance(
                imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
                frameBufferCount);

使用阻塞目标缓冲区（例如 ImageReader）时，请在使用帧后将其舍弃：

Kotlin

imageReader.setOnImageAvailableListener({
  val frame =  it.acquireNextImage()
  // Do something with "frame" here
  it.close()
}, null)

Java

imageReader.setOnImageAvailableListener(listener -> {
            Image frame = listener.acquireNextImage();
            // Do something with "frame" here
            listener.close();
        }, null);

LEGACY 硬件级别针对的是最通用的设备。您可以添加条件分支，并针对硬件级别为 LIMITED 的设备，将其中一个输出目标 Surface 的大小设为 RECORD，甚至针对硬件级别为 FULL 的设备，将大小增加到 MAXIMUM。