同時使用多個相機串流畫面

注意：本頁面所述是指 Camera2 套件。除非應用程式需要 Camera2 的特定低階功能，否則建議使用 CameraX。CameraX 和 Camera2 均支援 Android 5.0 (API 級別 21) 以上版本。

相機應用程式可以同時使用多個影格串流。在某些情況下，不同串流甚至需要不同的影格解析度或像素格式。常見的用途包括：

• 錄製影片：一個串流用於預覽，另一個串流則用於編碼並儲存至檔案。
• 條碼掃描：一個串流用於預覽，另一個用於偵測條碼。
• 計算攝影：一個串流用於預覽，另一個用於臉部/場景偵測。

處理影格時會產生相當可觀的效能成本，而平行串流或管道處理作業會使成本倍增。

CPU、GPU 和 DSP 等資源或許能運用架構的重新處理功能，但記憶體等資源會線性成長。

每個要求有多個目標

多個攝影機串流可以合併為單一CameraCaptureRequest。以下程式碼片段說明如何設定攝影機工作階段，其中一個串流用於攝影機預覽，另一個串流用於影像處理：

Kotlin

val session: CameraCaptureSession = ...  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
val requestTemplate = CameraDevice.TEMPLATE_PREVIEW
val combinedRequest = session.device.createCaptureRequest(requestTemplate)

// Link the Surface targets with the combined request
combinedRequest.addTarget(previewSurface)
combinedRequest.addTarget(imReaderSurface)

// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
session.setRepeatingRequest(combinedRequest.build(), null, null)

Java

CameraCaptureSession session = …;  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
        CaptureRequest.Builder combinedRequest = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);

// Link the Surface targets with the combined request
        combinedRequest.addTarget(previewSurface);
        combinedRequest.addTarget(imReaderSurface);

// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
        session.setRepeatingRequest(combinedRequest.build(), null, null);

如果正確設定目標介面，這段程式碼只會產生符合 StreamComfigurationMap.GetOutputMinFrameDuration(int, Size) 和 StreamComfigurationMap.GetOutputStallDuration(int, Size) 決定的最低 FPS 串流。實際效能會因裝置而異，但 Android 會根據三項變數 (輸出類型、輸出大小和硬體層級)，保證支援特定組合。

使用不支援的變數組合可能可以運作，但影格速率會很低；如果無法運作，系統會觸發其中一個失敗回呼。createCaptureSession 的文件說明保證可運作的項目。

輸出類型

輸出類型是指影格的編碼格式。可能的值為 PRIV、YUV、JPEG 和 RAW。createCaptureSession 的說明文件會說明這些項目。

選擇應用程式的輸出類型時，如果目標是盡量提高相容性，請使用 ImageFormat.YUV_420_888 進行影格分析，並使用 ImageFormat.JPEG 產生靜態圖片。在預覽和錄製情境中，您可能會使用 SurfaceView、TextureView、MediaRecorder、MediaCodec 或 RenderScript.Allocation。在這種情況下，請勿指定圖片格式。為求相容性，無論內部使用的實際格式為何，系統都會將其視為 ImageFormat.PRIVATE。如要查詢裝置支援的格式 (以 CameraCharacteristics 為準)，請使用下列程式碼：

Kotlin

val characteristics: CameraCharacteristics = ...
val supportedFormats = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).outputFormats

Java

CameraCharacteristics characteristics = …;
        int[] supportedFormats = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).getOutputFormats();

輸出大小

所有可用的輸出大小都會列在 StreamConfigurationMap.getOutputSizes() 中，但只有 PREVIEW 和 MAXIMUM 與相容性有關。這些大小會做為上限。如果大小為 PREVIEW 的項目可以運作，那麼小於 PREVIEW 的項目也都能運作。MAXIMUM 也是如此。如需這些大小的說明，請參閱 CameraDevice 的文件。

可用的輸出大小取決於所選格式。指定 CameraCharacteristics 和格式後，即可查詢可用的輸出大小，如下所示：

Kotlin

val characteristics: CameraCharacteristics = ...
val outputFormat: Int = ...  // such as ImageFormat.JPEG
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(outputFormat)

Java

CameraCharacteristics characteristics = …;
        int outputFormat = …;  // such as ImageFormat.JPEG
Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

在相機預覽和錄影用途中，使用目標類別判斷支援的大小。格式會由攝影機架構本身處理：

Kotlin

val characteristics: CameraCharacteristics = ...
val targetClass: Class <T> = ...  // such as SurfaceView::class.java
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(targetClass)

Java

CameraCharacteristics characteristics = …;
   int outputFormat = …;  // such as ImageFormat.JPEG
   Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

如要取得 MAXIMUM 大小，請依面積排序輸出大小，然後傳回最大的大小：

Kotlin

fun <T>getMaximumOutputSize(
    characteristics: CameraCharacteristics, targetClass: Class <T>, format: Int? = null):
    Size {
  val config = characteristics.get(
      CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)

  // If image format is provided, use it to determine supported sizes; or else use target class
  val allSizes = if (format == null)
    config.getOutputSizes(targetClass) else config.getOutputSizes(format)
  return allSizes.maxBy { it.height * it.width }
}

Java

 @RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getMaximumOutputSize(CameraCharacteristics characteristics,
                                            Class <T> targetClass,
                                            Integer format) {
        StreamConfigurationMap config = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);

        // If image format is provided, use it to determine supported sizes; else use target class
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }
        return Arrays.stream(allSizes).max(Comparator.comparing(s -> s.getHeight() * s.getWidth())).get();
    }

PREVIEW 是指符合裝置螢幕解析度或 1080p (1920x1080) 的最佳尺寸，以較小者為準。顯示比例可能與螢幕的顯示比例不完全相符，因此您可能需要對串流套用上下黑邊或裁剪，才能以全螢幕模式顯示。如要取得正確的預覽大小，請比較可用的輸出大小與螢幕大小，同時考量螢幕可能會旋轉。

下列程式碼定義了輔助類別 SmartSize，可稍微簡化大小比較作業：

Kotlin

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
class SmartSize(width: Int, height: Int) {
    var size = Size(width, height)
    var long = max(size.width, size.height)
    var short = min(size.width, size.height)
    override fun toString() = "SmartSize(${long}x${short})"
}

/** Standard High Definition size for pictures and video */
val SIZE_1080P: SmartSize = SmartSize(1920, 1080)

/** Returns a [SmartSize] object for the given [Display] */
fun getDisplaySmartSize(display: Display): SmartSize {
    val outPoint = Point()
    display.getRealSize(outPoint)
    return SmartSize(outPoint.x, outPoint.y)
}

/**
 * Returns the largest available PREVIEW size. For more information, see:
 * https://d.android.com/reference/android/hardware/camera2/CameraDevice
 */
fun <T>getPreviewOutputSize(
        display: Display,
        characteristics: CameraCharacteristics,
        targetClass: Class <T>,
        format: Int? = null
): Size {

    // Find which is smaller: screen or 1080p
    val screenSize = getDisplaySmartSize(display)
    val hdScreen = screenSize.long >= SIZE_1080P.long || screenSize.short >= SIZE_1080P.short
    val maxSize = if (hdScreen) SIZE_1080P else screenSize

    // If image format is provided, use it to determine supported sizes; else use target class
    val config = characteristics.get(
            CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)!!
    if (format == null)
        assert(StreamConfigurationMap.isOutputSupportedFor(targetClass))
    else
        assert(config.isOutputSupportedFor(format))
    val allSizes = if (format == null)
        config.getOutputSizes(targetClass) else config.getOutputSizes(format)

    // Get available sizes and sort them by area from largest to smallest
    val validSizes = allSizes
            .sortedWith(compareBy { it.height * it.width })
            .map { SmartSize(it.width, it.height) }.reversed()

    // Then, get the largest output size that is smaller or equal than our max size
    return validSizes.first { it.long <= maxSize.long && it.short <= maxSize.short }.size
}

Java

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
    class SmartSize {
        Size size;
        double longSize;
        double shortSize;

        public SmartSize(Integer width, Integer height) {
            size = new Size(width, height);
            longSize = max(size.getWidth(), size.getHeight());
            shortSize = min(size.getWidth(), size.getHeight());
        }

        @Override
        public String toString() {
            return String.format("SmartSize(%sx%s)", longSize, shortSize);
        }
    }

    /** Standard High Definition size for pictures and video */
    SmartSize SIZE_1080P = new SmartSize(1920, 1080);

    /** Returns a [SmartSize] object for the given [Display] */
    SmartSize getDisplaySmartSize(Display display) {
        Point outPoint = new Point();
        display.getRealSize(outPoint);
        return new SmartSize(outPoint.x, outPoint.y);
    }

    /**
     * Returns the largest available PREVIEW size. For more information, see:
     * https://d.android.com/reference/android/hardware/camera2/CameraDevice
     */
    @RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getPreviewOutputSize(
            Display display,
            CameraCharacteristics characteristics,
            Class <T> targetClass,
            Integer format
    ){

        // Find which is smaller: screen or 1080p
        SmartSize screenSize = getDisplaySmartSize(display);
        boolean hdScreen = screenSize.longSize >= SIZE_1080P.longSize || screenSize.shortSize >= SIZE_1080P.shortSize;
        SmartSize maxSize;
        if (hdScreen) {
            maxSize = SIZE_1080P;
        } else {
            maxSize = screenSize;
        }

        // If image format is provided, use it to determine supported sizes; else use target class
        StreamConfigurationMap config = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
        if (format == null)
            assert(StreamConfigurationMap.isOutputSupportedFor(targetClass));
        else
            assert(config.isOutputSupportedFor(format));
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }

        // Get available sizes and sort them by area from largest to smallest
        List <Size> sortedSizes = Arrays.asList(allSizes);
        List <SmartSize> validSizes =
                sortedSizes.stream()
                        .sorted(Comparator.comparing(s -> s.getHeight() * s.getWidth()))
                        .map(s -> new SmartSize(s.getWidth(), s.getHeight()))
                        .sorted(Collections.reverseOrder()).collect(Collectors.toList());

        // Then, get the largest output size that is smaller or equal than our max size
        return validSizes.stream()
                .filter(s -> s.longSize <= maxSize.longSize && s.shortSize <= maxSize.shortSize)
                .findFirst().get().size;
    }

查看支援的硬體層級

如要在執行階段判斷可用的功能，請使用 CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL 檢查支援的硬體層級。

使用 CameraCharacteristics 物件時，您可以使用單一陳述式擷取硬體層級：

Kotlin

val characteristics: CameraCharacteristics = ...

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
val hardwareLevel = characteristics.get(
        CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL)

Java

CameraCharacteristics characteristics = ...;

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
Integer hardwareLevel = characteristics.get(
                CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);

整合所有資訊

您可以根據輸出類型、輸出大小和硬體層級，判斷哪些串流組合有效。下表列出 CameraDevice 在 LEGACY 硬體層級支援的設定。

目標 1		目標 2		目標 3		應用實例
類型	大小上限	類型	大小上限	類型	大小上限
PRIV	MAXIMUM					簡單預覽、GPU 影片處理或無預覽影片錄製。
JPEG	MAXIMUM					不使用觀景窗拍攝靜態影像。
YUV	MAXIMUM					在應用程式中處理影片/圖片。
PRIV	PREVIEW	JPEG	MAXIMUM			標準靜態影像。
YUV	PREVIEW	JPEG	MAXIMUM			應用程式內處理作業加上仍擷取。
PRIV	PREVIEW	PRIV	PREVIEW			標準錄影。
PRIV	PREVIEW	YUV	PREVIEW			預覽加上應用程式內處理。
PRIV	PREVIEW	YUV	PREVIEW	JPEG	MAXIMUM	擷取靜態相片，並在應用程式中處理。

LEGACY 是最低的硬體層級。下表顯示，只要裝置支援 Camera2 (API 級別 21 以上)，且沒有太多會限制效能的額外負荷 (例如記憶體、CPU 或熱限制)，就能使用正確的設定同時輸出最多三個串流。

應用程式也需要設定目標輸出緩衝區。舉例來說，如要以 LEGACY 硬體層級的裝置為目標，您可以設定兩個目標輸出介面，一個使用 ImageFormat.PRIVATE，另一個使用 ImageFormat.YUV_420_888。使用 PREVIEW 大小時，系統支援這種組合。使用本主題稍早定義的函式，取得攝影機 ID 的必要預覽大小需要下列程式碼：

Kotlin

val characteristics: CameraCharacteristics = ...
val context = this as Context  // assuming you are inside of an activity

val surfaceViewSize = getPreviewOutputSize(
    context, characteristics, SurfaceView::class.java)
val imageReaderSize = getPreviewOutputSize(
    context, characteristics, ImageReader::class.java, format = ImageFormat.YUV_420_888)

Java

CameraCharacteristics characteristics = ...;
        Context context = this; // assuming you are inside of an activity

        Size surfaceViewSize = getPreviewOutputSize(
                context, characteristics, SurfaceView.class);
        Size imageReaderSize = getPreviewOutputSize(
                context, characteristics, ImageReader.class, format = ImageFormat.YUV_420_888);

您必須使用提供的回呼，等待 SurfaceView 準備就緒：

Kotlin

val surfaceView = findViewById <SurfaceView>(...)
surfaceView.holder.addCallback(object : SurfaceHolder.Callback {
  override fun surfaceCreated(holder: SurfaceHolder) {
    // You do not need to specify image format, and it will be considered of type PRIV
    // Surface is now ready and you could use it as an output target for CameraSession
  }
  ...
})

Java

SurfaceView surfaceView = findViewById <SurfaceView>(...);

surfaceView.getHolder().addCallback(new SurfaceHolder.Callback() {
            @Override
            public void surfaceCreated(@NonNull SurfaceHolder surfaceHolder) {
                // You do not need to specify image format, and it will be considered of type PRIV
                // Surface is now ready and you could use it as an output target for CameraSession
            }
            ...
        });

您可以呼叫 SurfaceView 來強制比對攝影機輸出大小，SurfaceHolder.setFixedSize() 也可以採取類似於 GitHub 上攝影機範例的「Common」模組中 AutoFitSurfaceView 的做法，設定絕對大小，同時考量長寬比和可用空間，並在觸發活動變更時自動調整。

從 ImageReader 設定其他介面時，由於沒有要等待的回呼，因此會比較簡單：

Kotlin

val frameBufferCount = 3  // just an example, depends on your usage of ImageReader
val imageReader = ImageReader.newInstance(
    imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
    frameBufferCount)

Java

int frameBufferCount = 3;  // just an example, depends on your usage of ImageReader
ImageReader imageReader = ImageReader.newInstance(
                imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
                frameBufferCount);

使用 ImageReader 等封鎖目標緩衝區時，請在使用影格後捨棄影格：

Kotlin

imageReader.setOnImageAvailableListener({
  val frame =  it.acquireNextImage()
  // Do something with "frame" here
  it.close()
}, null)

Java

imageReader.setOnImageAvailableListener(listener -> {
            Image frame = listener.acquireNextImage();
            // Do something with "frame" here
            listener.close();
        }, null);

LEGACY 硬體層級目標是最低標準的裝置。您可以新增條件式分支，並在硬體層級為 LIMITED 的裝置中，將其中一個輸出目標介面的 RECORD 大小設為 MAXIMUM，甚至在硬體層級為 FULL 的裝置中，將大小設為 MAXIMUM。