It's common to run tests in CI without Gradle, or locally if you're using a different build system. This topic explains how to configure the Benchmark library at runtime without Gradle.
To learn about why Benchmarks are useful in CI and how to use benchmarking data to detect regressions, see Fighting Regressions with Benchmarks in CI on the Android Developers blog.
Benchmark outputs a JSON file with results and metadata from a test run. This JSON file is written to external storage on the device. You must to pull the file from the device with each run.
Output is disabled by default, but you can enable it with the following
instrumentation argument, passed to
am instrument command:
-e androidx.benchmark.output.enable true
By default, the reports are saved on-device in the download directory of the tested app's external storage, which you can find with the following query:
adb shell content query --uri content://media/external/file --projection _data --where "\"_data LIKE '%Android'\""
You can configure the path Benchmark writes to using the following instrumentation argument:
-e additionalTestOutputDir "device_path_you_can_write_to"
Storage and Android 10
If, like in Android Gradle, it is difficult to pull output data from the device before a test is uninstalled, you can pass the following instrumentation argument. On devices running Android 10 and higher, this argument lets the files persist after uninstall:
-e no-isolated-storage 1
If you are also targeting API 29 and higher, you also must explicitly allow legacy storage options in your benchmark's manifest:
<application android:requestLegacyExternalStorage="true" ... >
See Opt out of filtered view for more information on opting out of scoped storage.
The Benchmark Gradle plugin provides the command
./gradlew lockClocks to lock
the CPU clocks of a rooted device. This is useful for ensuring stability when
you have access to rooted devices, such as "userdebug" builds. You can
replicate this with the
lockClocks.sh shell script, available in the
You can either run the script directly from a Linux or Mac host, or you can push to the device with a few adb commands:
adb push path/lockClocks.sh /data/local/tmp/lockClocks.sh adb shell /data/local/tmp/lockClocks.sh adb shell rm /data/local/tmp/lockClocks.sh
If you run the shell script directly on a host, it dispatches these commands to a connected device.
You can configure the behavior of the library with the following instrumentation arguments:
- Reconfigures looping behavior to support benchmarking code during startup
- Disables warmup looping
- Captures 10 measurements
- Disables loop averaging, which minimize overhead in microbenchmarks
- Defaults to
- Enables writing the result JSON file to external storage
- Defaults to
- Comma-separated list of errors, such as
DEBUGGABLE,LOW-BATTERY, to turn into warnings
- Defaults to an empty list
- Configures where full JSON benchmark reports are saved on device
- Defaults to test APK's external download directory