注意:本页介绍的是 Camera2 软件包。除非您的应用需要 Camera2 中的特定低级功能,否则我们建议您使用 CameraX。CameraX 和 Camera2 都支持 Android 5.0(API 级别 21)及更高版本。
相机应用可以同时使用多个帧流。在 在某些情况下,不同的视频流甚至需要不同的帧分辨率或像素 格式。一些典型用例包括:
- 视频录制:一个视频流用于预览,另一个视频流正在编码和保存 创建一个文件
- 条形码扫描:一个用于预览的流,另一个用于条形码检测。
- 计算摄影:一个数据流用于预览,另一个用于人脸/场景 检测。
处理帧时会产生非常重要的性能开销, 会成倍增加。
CPU、GPU 和 DSP 等资源或许能够利用 框架的再处理 但内存等资源将线性增长。
每个请求多个目标
多个摄像头视频流可合并为单个视频流
CameraCaptureRequest
。
以下代码段说明了如何使用
用于相机预览的流和用于图片处理的流:
val session: CameraCaptureSession = ... // from CameraCaptureSession.StateCallback
// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
val requestTemplate = CameraDevice.TEMPLATE_PREVIEW
val combinedRequest = session.device.createCaptureRequest(requestTemplate)
// Link the Surface targets with the combined request
combinedRequest.addTarget(previewSurface)
combinedRequest.addTarget(imReaderSurface)
// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
session.setRepeatingRequest(combinedRequest.build(), null, null)
CameraCaptureSession session = …; // from CameraCaptureSession.StateCallback
// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
CaptureRequest.Builder combinedRequest = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
// Link the Surface targets with the combined request
combinedRequest.addTarget(previewSurface);
combinedRequest.addTarget(imReaderSurface);
// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
session.setRepeatingRequest(combinedRequest.build(), null, null);
如果您正确配置目标平台,此代码将仅生成
只要视频流达到 FPS 的
StreamComfigurationMap.GetOutputMinFrameDuration(int, Size)
和
StreamComfigurationMap.GetOutputStallDuration(int, Size)
。
实际性能因设备而异,不过 Android 提供了一些
可保证根据三个变量支持特定组合:
输出类型、输出大小和硬件级别。
使用不受支持的变量组合可能会在低帧速率下运行;如果
否则,将触发其中一个失败回调。
createCaptureSession
的文档
说明了保证什么有效
输出类型
“输出类型”是指帧的编码格式。通过
可能的值为 PRIV、YUV、JPEG 和 RAW。有关
createCaptureSession
对它们进行了说明。
在选择应用的输出类型时,如果目标是尽可能增加
兼容性,然后使用
ImageFormat.YUV_420_888
进行帧分析
ImageFormat.JPEG
表示静态图片
图片。对于预览和录制场景,您可能会使用
SurfaceView
、
TextureView
、
MediaRecorder
,
MediaCodec
或
RenderScript.Allocation
。在
在此类情况下,请勿指定图片格式。为确保兼容性,将被视为
ImageFormat.PRIVATE
、
无论内部实际使用的格式如何。查询支持的格式
获得最大收益
CameraCharacteristics
、
请使用以下代码:
val characteristics: CameraCharacteristics = ...
val supportedFormats = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).outputFormats
CameraCharacteristics characteristics = …;
int[] supportedFormats = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).getOutputFormats();
输出大小
所有可用的输出大小均按
StreamConfigurationMap.getOutputSizes()
、
但只有 2 个与兼容性相关:PREVIEW
和 MAXIMUM
。尺寸
用作上限如果 PREVIEW
大小的某个元素有效,那么任何具有
小于 PREVIEW
的尺寸也是不错的选择。MAXIMUM
也是如此。通过
文档
CameraDevice
介绍了这些尺寸
可用的输出大小取决于所选格式。由于存在
CameraCharacteristics
和格式,您可以查询可用的输出大小,如下所示:
val characteristics: CameraCharacteristics = ...
val outputFormat: Int = ... // such as ImageFormat.JPEG
val sizes = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
.getOutputSizes(outputFormat)
CameraCharacteristics characteristics = …;
int outputFormat = …; // such as ImageFormat.JPEG
Size[] sizes = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
.getOutputSizes(outputFormat);
在相机预览和录制用例中,使用目标类来确定 支持的尺寸格式将由相机框架本身处理:
val characteristics: CameraCharacteristics = ...
val targetClass: Class <T> = ... // such as SurfaceView::class.java
val sizes = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
.getOutputSizes(targetClass)
CameraCharacteristics characteristics = …;
int outputFormat = …; // such as ImageFormat.JPEG
Size[] sizes = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
.getOutputSizes(outputFormat);
如需获取 MAXIMUM
大小,请按面积对输出大小进行排序,然后返回
一:
fun <T>getMaximumOutputSize(
characteristics: CameraCharacteristics, targetClass: Class <T>, format: Int? = null):
Size {
val config = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
// If image format is provided, use it to determine supported sizes; or else use target class
val allSizes = if (format == null)
config.getOutputSizes(targetClass) else config.getOutputSizes(format)
return allSizes.maxBy { it.height * it.width }
}
@RequiresApi(api = Build.VERSION_CODES.N)
<T> Size getMaximumOutputSize(CameraCharacteristics characteristics,
Class <T> targetClass,
Integer format) {
StreamConfigurationMap config = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
// If image format is provided, use it to determine supported sizes; else use target class
Size[] allSizes;
if (format == null) {
allSizes = config.getOutputSizes(targetClass);
} else {
allSizes = config.getOutputSizes(format);
}
return Arrays.stream(allSizes).max(Comparator.comparing(s -> s.getHeight() * s.getWidth())).get();
}
PREVIEW
是指与设备的屏幕分辨率或
1080p (1920x1080),以较小者为准。宽高比可能与
与屏幕的宽高比完全一致,因此您可能需要对
剪裁到信息流,以全屏模式显示。要获得正确的
将可用的输出尺寸与显示尺寸进行比较
并考虑到显示屏可能会旋转。
以下代码定义了一个辅助类 SmartSize
,用于设置大小
更轻松地进行比较:
/** Helper class used to pre-compute shortest and longest sides of a [Size] */
class SmartSize(width: Int, height: Int) {
var size = Size(width, height)
var long = max(size.width, size.height)
var short = min(size.width, size.height)
override fun toString() = "SmartSize(${long}x${short})"
}
/** Standard High Definition size for pictures and video */
val SIZE_1080P: SmartSize = SmartSize(1920, 1080)
/** Returns a [SmartSize] object for the given [Display] */
fun getDisplaySmartSize(display: Display): SmartSize {
val outPoint = Point()
display.getRealSize(outPoint)
return SmartSize(outPoint.x, outPoint.y)
}
/**
* Returns the largest available PREVIEW size. For more information, see:
* https://d.android.com/reference/android/hardware/camera2/CameraDevice
*/
fun <T>getPreviewOutputSize(
display: Display,
characteristics: CameraCharacteristics,
targetClass: Class <T>,
format: Int? = null
): Size {
// Find which is smaller: screen or 1080p
val screenSize = getDisplaySmartSize(display)
val hdScreen = screenSize.long >= SIZE_1080P.long || screenSize.short >= SIZE_1080P.short
val maxSize = if (hdScreen) SIZE_1080P else screenSize
// If image format is provided, use it to determine supported sizes; else use target class
val config = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)!!
if (format == null)
assert(StreamConfigurationMap.isOutputSupportedFor(targetClass))
else
assert(config.isOutputSupportedFor(format))
val allSizes = if (format == null)
config.getOutputSizes(targetClass) else config.getOutputSizes(format)
// Get available sizes and sort them by area from largest to smallest
val validSizes = allSizes
.sortedWith(compareBy { it.height * it.width })
.map { SmartSize(it.width, it.height) }.reversed()
// Then, get the largest output size that is smaller or equal than our max size
return validSizes.first { it.long <= maxSize.long && it.short <= maxSize.short }.size
}
/** Helper class used to pre-compute shortest and longest sides of a [Size] */
class SmartSize {
Size size;
double longSize;
double shortSize;
public SmartSize(Integer width, Integer height) {
size = new Size(width, height);
longSize = max(size.getWidth(), size.getHeight());
shortSize = min(size.getWidth(), size.getHeight());
}
@Override
public String toString() {
return String.format("SmartSize(%sx%s)", longSize, shortSize);
}
}
/** Standard High Definition size for pictures and video */
SmartSize SIZE_1080P = new SmartSize(1920, 1080);
/** Returns a [SmartSize] object for the given [Display] */
SmartSize getDisplaySmartSize(Display display) {
Point outPoint = new Point();
display.getRealSize(outPoint);
return new SmartSize(outPoint.x, outPoint.y);
}
/**
* Returns the largest available PREVIEW size. For more information, see:
* https://d.android.com/reference/android/hardware/camera2/CameraDevice
*/
@RequiresApi(api = Build.VERSION_CODES.N)
<T> Size getPreviewOutputSize(
Display display,
CameraCharacteristics characteristics,
Class <T> targetClass,
Integer format
){
// Find which is smaller: screen or 1080p
SmartSize screenSize = getDisplaySmartSize(display);
boolean hdScreen = screenSize.longSize >= SIZE_1080P.longSize || screenSize.shortSize >= SIZE_1080P.shortSize;
SmartSize maxSize;
if (hdScreen) {
maxSize = SIZE_1080P;
} else {
maxSize = screenSize;
}
// If image format is provided, use it to determine supported sizes; else use target class
StreamConfigurationMap config = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
if (format == null)
assert(StreamConfigurationMap.isOutputSupportedFor(targetClass));
else
assert(config.isOutputSupportedFor(format));
Size[] allSizes;
if (format == null) {
allSizes = config.getOutputSizes(targetClass);
} else {
allSizes = config.getOutputSizes(format);
}
// Get available sizes and sort them by area from largest to smallest
List <Size> sortedSizes = Arrays.asList(allSizes);
List <SmartSize> validSizes =
sortedSizes.stream()
.sorted(Comparator.comparing(s -> s.getHeight() * s.getWidth()))
.map(s -> new SmartSize(s.getWidth(), s.getHeight()))
.sorted(Collections.reverseOrder()).collect(Collectors.toList());
// Then, get the largest output size that is smaller or equal than our max size
return validSizes.stream()
.filter(s -> s.longSize <= maxSize.longSize && s.shortSize <= maxSize.shortSize)
.findFirst().get().size;
}
检查支持的硬件级别
如需确定在运行时可用的功能,请检查支持的硬件
使用
CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL
。
使用
CameraCharacteristics
对象,您可以使用一个语句来检索硬件级别:
val characteristics: CameraCharacteristics = ...
// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
val hardwareLevel = characteristics.get(
CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL)
CameraCharacteristics characteristics = ...;
// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
Integer hardwareLevel = characteristics.get(
CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);
将各部分整合在一起
通过输出类型、输出大小和硬件级别,您可以确定
一组有效的数据流。下图是
具有 CameraDevice
支持的
LEGACY
硬件级别。
目标 1 | 目标 2 | 目标 3 | 应用场景示例 | |||
---|---|---|---|---|---|---|
类型 | 最大大小 | 类型 | 最大大小 | 类型 | 最大大小 | |
PRIV |
MAXIMUM |
简单预览、GPU 视频处理或不预览视频录制。 | ||||
JPEG |
MAXIMUM |
无取景器静态图片拍摄。 | ||||
YUV |
MAXIMUM |
应用内视频/图片处理。 | ||||
PRIV |
PREVIEW |
JPEG |
MAXIMUM |
标准静态成像。 | ||
YUV |
PREVIEW |
JPEG |
MAXIMUM |
应用内处理加静态拍摄。 | ||
PRIV |
PREVIEW |
PRIV |
PREVIEW |
标准录制。 | ||
PRIV |
PREVIEW |
YUV |
PREVIEW |
预览 + 应用内处理。 | ||
PRIV |
PREVIEW |
YUV |
PREVIEW |
预览 + 应用内处理。 | ||
PRIV |
PREVIEW |
YUV |
PREVIEW |
JPEG |
MAXIMUM |
静态拍摄加上应用内处理。 |
LEGACY
是可能的最低硬件级别。此表显示
支持 Camera2(API 级别 21 及更高级别)的设备最多可输出
使用正确的配置同时在线播放多个视频
限制性能的开销,例如内存、CPU 或散热约束。
您的应用还需要配置目标输出缓冲区。例如,
以硬件级别为 LEGACY
的设备为目标,您可以设置两个目标输出
surface,一个使用 ImageFormat.PRIVATE
,另一个使用
ImageFormat.YUV_420_888
。这是使用
PREVIEW
大小。使用本主题前面定义的函数,获取
相机 ID 所需的预览尺寸需要以下代码:
val characteristics: CameraCharacteristics = ...
val context = this as Context // assuming you are inside of an activity
val surfaceViewSize = getPreviewOutputSize(
context, characteristics, SurfaceView::class.java)
val imageReaderSize = getPreviewOutputSize(
context, characteristics, ImageReader::class.java, format = ImageFormat.YUV_420_888)
CameraCharacteristics characteristics = ...;
Context context = this; // assuming you are inside of an activity
Size surfaceViewSize = getPreviewOutputSize(
context, characteristics, SurfaceView.class);
Size imageReaderSize = getPreviewOutputSize(
context, characteristics, ImageReader.class, format = ImageFormat.YUV_420_888);
它需要使用提供的回调等待 SurfaceView
准备就绪:
val surfaceView = findViewById <SurfaceView>(...)
surfaceView.holder.addCallback(object : SurfaceHolder.Callback {
override fun surfaceCreated(holder: SurfaceHolder) {
// You do not need to specify image format, and it will be considered of type PRIV
// Surface is now ready and you could use it as an output target for CameraSession
}
...
})
SurfaceView surfaceView = findViewById <SurfaceView>(...);
surfaceView.getHolder().addCallback(new SurfaceHolder.Callback() {
@Override
public void surfaceCreated(@NonNull SurfaceHolder surfaceHolder) {
// You do not need to specify image format, and it will be considered of type PRIV
// Surface is now ready and you could use it as an output target for CameraSession
}
...
});
您可以通过调用SurfaceView
SurfaceHolder.setFixedSize()
或者,您可以采用
AutoFitSurfaceView
来自 Common
单元
GitHub 上提供了相机示例,该示例设置了绝对尺寸,
同时考虑宽高比和可用空间
调整何时触发 activity 更改。
从以下位置设置另一个 surface
所需格式的 ImageReader
为
因为没有需要等待的回调:
val frameBufferCount = 3 // just an example, depends on your usage of ImageReader
val imageReader = ImageReader.newInstance(
imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
frameBufferCount)
int frameBufferCount = 3; // just an example, depends on your usage of ImageReader
ImageReader imageReader = ImageReader.newInstance(
imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
frameBufferCount);
使用 ImageReader
等阻塞目标缓冲区时,请丢弃经过训练的
使用它们:
imageReader.setOnImageAvailableListener({
val frame = it.acquireNextImage()
// Do something with "frame" here
it.close()
}, null)
imageReader.setOnImageAvailableListener(listener -> {
Image frame = listener.acquireNextImage();
// Do something with "frame" here
listener.close();
}, null);
LEGACY
硬件级别针对的是最低标准 设备。您可以
添加条件分支并对其中一个输出目标使用 RECORD
大小
甚至在硬件级别为 LIMITED
的设备中显示,甚至将该级别提高到
硬件级别为 FULL
的设备的 MAXIMUM
大小。