注意:本页介绍的是 Camera2 软件包。除非您的应用需要 Camera2 的特定低层级功能,否则我们建议您使用 CameraX。CameraX 和 Camera2 都支持 Android 5.0(API 级别 21)及更高版本。
Android 9(API 级别 28)中引入了多摄像头功能。自发布以来,市场上便有支持该 API 的设备。许多多摄像头用例与特定硬件配置紧密相关。也就是说,并非所有用例都与每台设备兼容,因此多摄像头功能非常适合 Play Feature Delivery。
一些典型用例包括:
- 缩放:根据剪裁区域或所需焦距切换摄像头。
- 深度:使用多个摄像头构建深度图。
- 焦外成像:使用推断的深度信息来模拟类似数码单反相机的窄对焦范围。
逻辑摄像头和物理摄像头之间的区别
如需了解多摄像头 API,还需要了解逻辑摄像头与物理摄像头之间的区别。作为参考,假设有一台配有三个后置摄像头的设备。在此示例中,三个后置摄像头中的每一个都被视为物理摄像头。逻辑摄像头是指由两个或更多个物理摄像头组成的分组。逻辑摄像头的输出可以是来自某个底层物理摄像头的流,也可以是同时来自多个底层物理摄像头的融合流。无论采用哪种方式,数据流都由相机硬件抽象层 (HAL) 进行处理。
许多手机制造商会开发第一方相机应用,这些应用通常会预安装在其设备上。为了使用硬件的所有功能,它们可以使用专用 API 或隐藏 API,或者接受其他应用无权访问的驱动程序实现提供的特殊处理。某些设备通过提供来自不同物理摄像头的融合帧流(但仅向某些特权应用)提供逻辑摄像头的概念。通常情况下,只有一个物理摄像头向框架公开。下图说明了 Android 9 之前第三方开发者的情况:
从 Android 9 开始,Android 应用中不再允许使用私有 API。由于 Android 框架中添加了多摄像头支持,因此 Android 最佳实践强烈建议手机制造商为所有物理摄像头提供朝向相同方向的逻辑摄像头。第三方开发者应该会在搭载 Android 9 及更高版本的设备上看到以下内容:
逻辑摄像头提供的内容完全取决于 OEM 的相机 HAL 实现。例如,Pixel 3 等设备以如下方式实现其逻辑摄像头:根据请求的焦距和剪裁区域选择其中一个物理摄像头。
多摄像头 API
新 API 新增了以下常量、类和方法:
CameraMetadata.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA
CameraCharacteristics.getPhysicalCameraIds()
CameraCharacteristics.getAvailablePhysicalCameraRequestKeys()
CameraDevice.createCaptureSession(SessionConfiguration config)
CameraCharacteritics.LOGICAL_MULTI_CAMERA_SENSOR_SYNC_TYPE
OutputConfiguration
和SessionConfiguration
由于 Android 兼容性定义文档 (CDD) 中对多摄像头 API 进行了更改,因此开发者对多摄像头 API 也有一些预期。在 Android 9 之前存在的配有双摄像头的设备,但同时打开多个摄像头会涉及反复试验和错误。在 Android 9 及更高版本中,多摄像头功能提供了一组规则,用于指定何时可以打开属于同一逻辑摄像头的一对物理摄像头。
在大多数情况下,搭载 Android 9 及更高版本的设备会公开所有物理摄像头(可能不包括红外线等不太常见的传感器类型)以及一个更易于使用的逻辑摄像头。对于保证有效的每个数据流组合,可将属于逻辑摄像头的一个数据流替换为来自底层物理摄像头的两个数据流。
同时在线播放多个直播
同时使用多个摄像头视频流介绍了在单个摄像头中同时使用多个视频流的规则。其中一项值得注意的是,同一规则适用于多个摄像头。CameraMetadata.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA
说明了如何将逻辑 YUV_420_888 或原始信息流替换为两个物理信息流。也就是说,每个 YUV 或 RAW 类型的数据流都可以替换为两个类型和大小相同的数据流。对于单摄像头设备,您可以先从采用以下保证配置的摄像头数据流开始:
- 数据流 1:YUV 类型,逻辑摄像头 (
id = 0
) 的大小为MAXIMUM
然后,借助支持多摄像头的设备,您可以创建会话,将该逻辑 YUV 信息流替换为两个物理信息流:
- 数据流 1:YUV 类型,
MAXIMUM
尺寸(来自实体摄像头)id = 1
- 数据流 2:YUV 类型,
MAXIMUM
尺寸(来自实体摄像头)id = 2
当且仅当两个摄像头属于 CameraCharacteristics.getPhysicalCameraIds()
下的逻辑摄像头分组的一部分时,您才可以将 YUV 或 RAW 流替换为两个等效流。
该框架提供的保证只是同时从多个物理摄像头获取帧所需的最低要求。大多数设备都支持额外的视频流,有时甚至允许单独打开多个物理摄像头设备。由于框架并不提供硬性保证,因此要这样做,就需要通过试验和错误测试按设备执行测试和调优。
创建包含多个物理摄像头的会话
在支持多摄像头的设备上使用物理摄像头时,请打开单个 CameraDevice
(逻辑摄像头),并在单个会话中与其互动。使用 API 级别 28 中的新增 API CameraDevice.createCaptureSession(SessionConfiguration config)
创建单个会话。会话配置具有多种输出配置,每种输出配置都有一组输出目标,以及(可选)所需的物理摄像头 ID。
捕获请求具有与其关联的输出目标。框架根据连接的输出目标确定将请求发送到哪个物理(或逻辑)相机。如果输出目标对应于作为输出配置与物理摄像头 ID 一起发送的某个输出目标,则该物理摄像头会接收并处理请求。
使用一对物理摄像头
适用于多摄像头的 Camera API 的另一个新增功能是能够识别逻辑摄像头并查找其背后的物理摄像头。您可以定义一个函数来帮助识别可能的物理摄像头对,这些摄像头可用于替换其中一个逻辑摄像头数据流:
Kotlin
/** * Helper class used to encapsulate a logical camera and two underlying * physical cameras */ data class DualCamera(val logicalId: String, val physicalId1: String, val physicalId2: String) fun findDualCameras(manager: CameraManager, facing: Int? = null): List{ val dualCameras = MutableList () // Iterate over all the available camera characteristics manager.cameraIdList.map { Pair(manager.getCameraCharacteristics(it), it) }.filter { // Filter by cameras facing the requested direction facing == null || it.first.get(CameraCharacteristics.LENS_FACING) == facing }.filter { // Filter by logical cameras // CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA requires API >= 28 it.first.get(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES)!!.contains( CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA) }.forEach { // All possible pairs from the list of physical cameras are valid results // NOTE: There could be N physical cameras as part of a logical camera grouping // getPhysicalCameraIds() requires API >= 28 val physicalCameras = it.first.physicalCameraIds.toTypedArray() for (idx1 in 0 until physicalCameras.size) { for (idx2 in (idx1 + 1) until physicalCameras.size) { dualCameras.add(DualCamera( it.second, physicalCameras[idx1], physicalCameras[idx2])) } } } return dualCameras }
Java
/** * Helper class used to encapsulate a logical camera and two underlying * physical cameras */ final class DualCamera { final String logicalId; final String physicalId1; final String physicalId2; DualCamera(String logicalId, String physicalId1, String physicalId2) { this.logicalId = logicalId; this.physicalId1 = physicalId1; this.physicalId2 = physicalId2; } } ListfindDualCameras(CameraManager manager, Integer facing) { List dualCameras = new ArrayList<>(); List cameraIdList; try { cameraIdList = Arrays.asList(manager.getCameraIdList()); } catch (CameraAccessException e) { e.printStackTrace(); cameraIdList = new ArrayList<>(); } // Iterate over all the available camera characteristics cameraIdList.stream() .map(id -> { try { CameraCharacteristics characteristics = manager.getCameraCharacteristics(id); return new Pair<>(characteristics, id); } catch (CameraAccessException e) { e.printStackTrace(); return null; } }) .filter(pair -> { // Filter by cameras facing the requested direction return (pair != null) && (facing == null || pair.first.get(CameraCharacteristics.LENS_FACING).equals(facing)); }) .filter(pair -> { // Filter by logical cameras // CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA requires API >= 28 IntPredicate logicalMultiCameraPred = arg -> arg == CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA; return Arrays.stream(pair.first.get(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES)) .anyMatch(logicalMultiCameraPred); }) .forEach(pair -> { // All possible pairs from the list of physical cameras are valid results // NOTE: There could be N physical cameras as part of a logical camera grouping // getPhysicalCameraIds() requires API >= 28 String[] physicalCameras = pair.first.getPhysicalCameraIds().toArray(new String[0]); for (int idx1 = 0; idx1 < physicalCameras.length; idx1++) { for (int idx2 = idx1 + 1; idx2 < physicalCameras.length; idx2++) { dualCameras.add( new DualCamera(pair.second, physicalCameras[idx1], physicalCameras[idx2])); } } }); return dualCameras; }
物理摄像头的状态处理由逻辑摄像头控制。如需打开“双摄像头”,请打开与物理摄像头对应的逻辑摄像头:
Kotlin
fun openDualCamera(cameraManager: CameraManager, dualCamera: DualCamera, // AsyncTask is deprecated beginning API 30 executor: Executor = AsyncTask.SERIAL_EXECUTOR, callback: (CameraDevice) -> Unit) { // openCamera() requires API >= 28 cameraManager.openCamera( dualCamera.logicalId, executor, object : CameraDevice.StateCallback() { override fun onOpened(device: CameraDevice) = callback(device) // Omitting for brevity... override fun onError(device: CameraDevice, error: Int) = onDisconnected(device) override fun onDisconnected(device: CameraDevice) = device.close() }) }
Java
void openDualCamera(CameraManager cameraManager, DualCamera dualCamera, Executor executor, CameraDeviceCallback cameraDeviceCallback ) { // openCamera() requires API >= 28 cameraManager.openCamera(dualCamera.logicalId, executor, new CameraDevice.StateCallback() { @Override public void onOpened(@NonNull CameraDevice cameraDevice) { cameraDeviceCallback.callback(cameraDevice); } @Override public void onDisconnected(@NonNull CameraDevice cameraDevice) { cameraDevice.close(); } @Override public void onError(@NonNull CameraDevice cameraDevice, int i) { onDisconnected(cameraDevice); } }); }
除了选择要打开哪个相机外,该过程与在旧版 Android 中打开相机的过程相同。使用新的会话配置 API 创建拍摄会话会指示框架将某些目标与特定的物理摄像头 ID 相关联:
Kotlin
/** * Helper type definition that encapsulates 3 sets of output targets: * * 1. Logical camera * 2. First physical camera * 3. Second physical camera */ typealias DualCameraOutputs = Triple?, MutableList ?, MutableList ?> fun createDualCameraSession(cameraManager: CameraManager, dualCamera: DualCamera, targets: DualCameraOutputs, // AsyncTask is deprecated beginning API 30 executor: Executor = AsyncTask.SERIAL_EXECUTOR, callback: (CameraCaptureSession) -> Unit) { // Create 3 sets of output configurations: one for the logical camera, and // one for each of the physical cameras. val outputConfigsLogical = targets.first?.map { OutputConfiguration(it) } val outputConfigsPhysical1 = targets.second?.map { OutputConfiguration(it).apply { setPhysicalCameraId(dualCamera.physicalId1) } } val outputConfigsPhysical2 = targets.third?.map { OutputConfiguration(it).apply { setPhysicalCameraId(dualCamera.physicalId2) } } // Put all the output configurations into a single flat array val outputConfigsAll = arrayOf( outputConfigsLogical, outputConfigsPhysical1, outputConfigsPhysical2) .filterNotNull().flatMap { it } // Instantiate a session configuration that can be used to create a session val sessionConfiguration = SessionConfiguration( SessionConfiguration.SESSION_REGULAR, outputConfigsAll, executor, object : CameraCaptureSession.StateCallback() { override fun onConfigured(session: CameraCaptureSession) = callback(session) // Omitting for brevity... override fun onConfigureFailed(session: CameraCaptureSession) = session.device.close() }) // Open the logical camera using the previously defined function openDualCamera(cameraManager, dualCamera, executor = executor) { // Finally create the session and return via callback it.createCaptureSession(sessionConfiguration) } }
Java
/** * Helper class definition that encapsulates 3 sets of output targets: ** 1. Logical camera * 2. First physical camera * 3. Second physical camera */ final class DualCameraOutputs { private final List
logicalCamera; private final List firstPhysicalCamera; private final List secondPhysicalCamera; public DualCameraOutputs(List logicalCamera, List firstPhysicalCamera, List third) { this.logicalCamera = logicalCamera; this.firstPhysicalCamera = firstPhysicalCamera; this.secondPhysicalCamera = third; } public List getLogicalCamera() { return logicalCamera; } public List getFirstPhysicalCamera() { return firstPhysicalCamera; } public List getSecondPhysicalCamera() { return secondPhysicalCamera; } } interface CameraCaptureSessionCallback { void callback(CameraCaptureSession cameraCaptureSession); } void createDualCameraSession(CameraManager cameraManager, DualCamera dualCamera, DualCameraOutputs targets, Executor executor, CameraCaptureSessionCallback cameraCaptureSessionCallback) { // Create 3 sets of output configurations: one for the logical camera, and // one for each of the physical cameras. List outputConfigsLogical = targets.getLogicalCamera().stream() .map(OutputConfiguration::new) .collect(Collectors.toList()); List outputConfigsPhysical1 = targets.getFirstPhysicalCamera().stream() .map(s -> { OutputConfiguration outputConfiguration = new OutputConfiguration(s); outputConfiguration.setPhysicalCameraId(dualCamera.physicalId1); return outputConfiguration; }) .collect(Collectors.toList()); List outputConfigsPhysical2 = targets.getSecondPhysicalCamera().stream() .map(s -> { OutputConfiguration outputConfiguration = new OutputConfiguration(s); outputConfiguration.setPhysicalCameraId(dualCamera.physicalId2); return outputConfiguration; }) .collect(Collectors.toList()); // Put all the output configurations into a single flat array List outputConfigsAll = Stream.of( outputConfigsLogical, outputConfigsPhysical1, outputConfigsPhysical2 ) .filter(Objects::nonNull) .flatMap(Collection::stream) .collect(Collectors.toList()); // Instantiate a session configuration that can be used to create a session SessionConfiguration sessionConfiguration = new SessionConfiguration( SessionConfiguration.SESSION_REGULAR, outputConfigsAll, executor, new CameraCaptureSession.StateCallback() { @Override public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) { cameraCaptureSessionCallback.callback(cameraCaptureSession); } // Omitting for brevity... @Override public void onConfigureFailed(@NonNull CameraCaptureSession cameraCaptureSession) { cameraCaptureSession.getDevice().close(); } }); // Open the logical camera using the previously defined function openDualCamera(cameraManager, dualCamera, executor, (CameraDevice c) -> // Finally create the session and return via callback c.createCaptureSession(sessionConfiguration)); }
如需了解受支持的数据流组合,请参阅 createCaptureSession
。合并视频流适用于单个逻辑摄像头上的多个视频流。兼容性可扩展到使用相同的配置,并将其中一个数据流替换为来自属于同一逻辑摄像头的两个物理摄像头的两个数据流。
准备好相机会话后,分派所需的拍摄请求。拍摄请求的每个目标都会从其关联的物理摄像头(如果有正在使用)接收其数据,或者回退到逻辑摄像头。
Zoom 应用场景示例
可以将物理摄像头合并到单个视频流中,以便用户能够在不同的物理摄像头之间切换,体验不同的视野范围,从而有效地捕获不同的“缩放级别”。
首先,选择一对物理摄像头,以便用户进行切换。为了达到最佳效果,您可以选择提供最小和最大可用焦距的一对摄像头。
Kotlin
fun findShortLongCameraPair(manager: CameraManager, facing: Int? = null): DualCamera? { return findDualCameras(manager, facing).map { val characteristics1 = manager.getCameraCharacteristics(it.physicalId1) val characteristics2 = manager.getCameraCharacteristics(it.physicalId2) // Query the focal lengths advertised by each physical camera val focalLengths1 = characteristics1.get( CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS) ?: floatArrayOf(0F) val focalLengths2 = characteristics2.get( CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS) ?: floatArrayOf(0F) // Compute the largest difference between min and max focal lengths between cameras val focalLengthsDiff1 = focalLengths2.maxOrNull()!! - focalLengths1.minOrNull()!! val focalLengthsDiff2 = focalLengths1.maxOrNull()!! - focalLengths2.minOrNull()!! // Return the pair of camera IDs and the difference between min and max focal lengths if (focalLengthsDiff1 < focalLengthsDiff2) { Pair(DualCamera(it.logicalId, it.physicalId1, it.physicalId2), focalLengthsDiff1) } else { Pair(DualCamera(it.logicalId, it.physicalId2, it.physicalId1), focalLengthsDiff2) } // Return only the pair with the largest difference, or null if no pairs are found }.maxByOrNull { it.second }?.first }
Java
// Utility functions to find min/max value in float[] float findMax(float[] array) { float max = Float.NEGATIVE_INFINITY; for(float cur: array) max = Math.max(max, cur); return max; } float findMin(float[] array) { float min = Float.NEGATIVE_INFINITY; for(float cur: array) min = Math.min(min, cur); return min; } DualCamera findShortLongCameraPair(CameraManager manager, Integer facing) { return findDualCameras(manager, facing).stream() .map(c -> { CameraCharacteristics characteristics1; CameraCharacteristics characteristics2; try { characteristics1 = manager.getCameraCharacteristics(c.physicalId1); characteristics2 = manager.getCameraCharacteristics(c.physicalId2); } catch (CameraAccessException e) { e.printStackTrace(); return null; } // Query the focal lengths advertised by each physical camera float[] focalLengths1 = characteristics1.get( CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS); float[] focalLengths2 = characteristics2.get( CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS); // Compute the largest difference between min and max focal lengths between cameras Float focalLengthsDiff1 = findMax(focalLengths2) - findMin(focalLengths1); Float focalLengthsDiff2 = findMax(focalLengths1) - findMin(focalLengths2); // Return the pair of camera IDs and the difference between min and max focal lengths if (focalLengthsDiff1 < focalLengthsDiff2) { return new Pair<>(new DualCamera(c.logicalId, c.physicalId1, c.physicalId2), focalLengthsDiff1); } else { return new Pair<>(new DualCamera(c.logicalId, c.physicalId2, c.physicalId1), focalLengthsDiff2); } }) // Return only the pair with the largest difference, or null if no pairs are found .max(Comparator.comparing(pair -> pair.second)).get().first; }
合理的架构是设置两个 SurfaceViews
- 每个数据流各一个。这些 SurfaceViews
会根据用户互动进行交换,因此在任何给定时间都只有一个可见。
以下代码展示了如何打开逻辑摄像头、配置摄像头输出、创建摄像头会话以及启动两个预览流:
Kotlin
val cameraManager: CameraManager = ... // Get the two output targets from the activity / fragment val surface1 = ... // from SurfaceView val surface2 = ... // from SurfaceView val dualCamera = findShortLongCameraPair(manager)!! val outputTargets = DualCameraOutputs( null, mutableListOf(surface1), mutableListOf(surface2)) // Here you open the logical camera, configure the outputs and create a session createDualCameraSession(manager, dualCamera, targets = outputTargets) { session -> // Create a single request which has one target for each physical camera // NOTE: Each target receive frames from only its associated physical camera val requestTemplate = CameraDevice.TEMPLATE_PREVIEW val captureRequest = session.device.createCaptureRequest(requestTemplate).apply { arrayOf(surface1, surface2).forEach { addTarget(it) } }.build() // Set the sticky request for the session and you are done session.setRepeatingRequest(captureRequest, null, null) }
Java
CameraManager manager = ...; // Get the two output targets from the activity / fragment Surface surface1 = ...; // from SurfaceView Surface surface2 = ...; // from SurfaceView DualCamera dualCamera = findShortLongCameraPair(manager, null); DualCameraOutputs outputTargets = new DualCameraOutputs( null, Collections.singletonList(surface1), Collections.singletonList(surface2)); // Here you open the logical camera, configure the outputs and create a session createDualCameraSession(manager, dualCamera, outputTargets, null, (session) -> { // Create a single request which has one target for each physical camera // NOTE: Each target receive frames from only its associated physical camera CaptureRequest.Builder captureRequestBuilder; try { captureRequestBuilder = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW); Arrays.asList(surface1, surface2).forEach(captureRequestBuilder::addTarget); // Set the sticky request for the session and you are done session.setRepeatingRequest(captureRequestBuilder.build(), null, null); } catch (CameraAccessException e) { e.printStackTrace(); } });
接下来,您只需提供一个界面,以便用户在两个 surface 之间切换,例如点按一个按钮或点按两次 SurfaceView
。您甚至可以执行某种形式的场景分析,并自动在两个数据流之间切换。
镜头失真
所有镜头都会产生一定程度的失真。在 Android 中,您可以使用 CameraCharacteristics.LENS_DISTORTION
(取代现已废弃的 CameraCharacteristics.LENS_RADIAL_DISTORTION
)查询镜头产生的失真率。对于逻辑摄像头,失真很小,并且您的应用可以根据来自摄像头的帧多使用或减少使用帧。对于物理摄像头,其镜头配置可能截然不同,尤其是在广角镜头上。
某些设备可以通过 CaptureRequest.DISTORTION_CORRECTION_MODE
实现自动失真校正。对于大多数设备,失真校正默认处于开启状态。
Kotlin
val cameraSession: CameraCaptureSession = ... // Use still capture template to build the capture request val captureRequest = cameraSession.device.createCaptureRequest( CameraDevice.TEMPLATE_STILL_CAPTURE ) // Determine if this device supports distortion correction val characteristics: CameraCharacteristics = ... val supportsDistortionCorrection = characteristics.get( CameraCharacteristics.DISTORTION_CORRECTION_AVAILABLE_MODES )?.contains( CameraMetadata.DISTORTION_CORRECTION_MODE_HIGH_QUALITY ) ?: false if (supportsDistortionCorrection) { captureRequest.set( CaptureRequest.DISTORTION_CORRECTION_MODE, CameraMetadata.DISTORTION_CORRECTION_MODE_HIGH_QUALITY ) } // Add output target, set other capture request parameters... // Dispatch the capture request cameraSession.capture(captureRequest.build(), ...)
Java
CameraCaptureSession cameraSession = ...; // Use still capture template to build the capture request CaptureRequest.Builder captureRequestBuilder = null; try { captureRequestBuilder = cameraSession.getDevice().createCaptureRequest( CameraDevice.TEMPLATE_STILL_CAPTURE ); } catch (CameraAccessException e) { e.printStackTrace(); } // Determine if this device supports distortion correction CameraCharacteristics characteristics = ...; boolean supportsDistortionCorrection = Arrays.stream( characteristics.get( CameraCharacteristics.DISTORTION_CORRECTION_AVAILABLE_MODES )) .anyMatch(i -> i == CameraMetadata.DISTORTION_CORRECTION_MODE_HIGH_QUALITY); if (supportsDistortionCorrection) { captureRequestBuilder.set( CaptureRequest.DISTORTION_CORRECTION_MODE, CameraMetadata.DISTORTION_CORRECTION_MODE_HIGH_QUALITY ); } // Add output target, set other capture request parameters... // Dispatch the capture request cameraSession.capture(captureRequestBuilder.build(), ...);
在此模式下设置拍摄请求可能会影响相机可产生的帧速率。您可以选择仅针对静态图片拍摄设置失真校正。