On Android, you can deliver rich generative AI experiences without needing a network connection or sending data to the cloud. On-device AI is a great solution for use-cases where low latency, low cost, and privacy safeguards are your primary concerns.
For on-device use-cases, you can take advantage of Google's Gemini Nano foundation model. While it is smaller than other Gemini models running inference in the cloud, you can fine-tune Gemini Nano to perform specialized tasks as well as its larger counterparts. Gemini Nano runs in Android's AICore system service, which leverages device hardware to enable low inference latency and keeps the model up-to-date.
Gemini Nano and AICore are available on Google Pixel 9 series devices, Pixel 8 Pro, Pixel 81, Pixel 8a2, and Samsung S24 series devices, with support for more products coming soon.
Access to Gemini Nano API and AICore is provided by the Google AI Edge SDK. Google AI Edge is a comprehensive suite of tools for on-device ML. Learn more about the Google AI Edge.
Architecture
As a system-level module, you access AICore through a series of APIs in order to run inference on-device. You can also provide a LoRA fine-tuning block to further optimize the performance of Gemini Nano for your application. In addition, AICore has several built-in safety features, ensuring a thorough evaluation against our safety filters. The following diagram outlines how an app accesses AICore to run Gemini Nano on-device.
![Google AI Edge SDK, AICore, and Gemini Nano.](https://developer.android.google.cn/static/images/ai/aicore/gemini-architecture-google-ai-edge-sdk.png?authuser=19)
Keep user data private and secure
AICore adheres to the Private Compute Core principles, with the following key characteristics:
Restricted Package Binding: AICore is isolated from most other packages, with limited exceptions for specific system packages. Any modifications to this allowed list can only occur during a full Android OTA update.
Indirect Internet Access: AICore does not have direct internet access. All internet requests, including model downloads, are routed through the open-source Private Compute Services companion APK. APIs within Private Compute Services must explicitly demonstrate their privacy-centric nature.
![Illustration of the AICore architecture](https://developer.android.google.cn/static/images/ai/aicore-architecture.png?authuser=19)
Benefits of accessing AI foundation models with AICore
AICore enables the Android OS to provide and manage AI foundation models. This significantly reduces the cost of using these large models in your app, principally due to the following:
Ease of deployment: AICore manages the distribution of Gemini Nano and handles future updates. You don't need to worry about downloading or updating large models over the network, nor impact on your app's disk and runtime memory budget.
Accelerated inference: AICore leverages on-device hardware to accelerate inference. Your app gets the best performance on each device, and you don't need to worry about the underlying hardware interfaces.
Supported functionality
AICore supports the following devices and modalities:
- Supported Devices: AICore is currently available on Google Pixel 8 Pro and Samsung S24 Series devices.
- Supported Modalities: AICore currently supports text modality for Gemini Nano.
Additional device and modality support are areas of active investment.
Use cases
Some tasks you can accomplish with Gemini Nano through AICore are the following:
- AI-enhanced content consumption: text summarization, answering questions, and entity extraction.
- AI-assisted content generation: proofreading, grammar correction, writing assistance, and contextual smart replies.
- Classifying text: sentiment detection or mood analysis
- Privacy: unlock generative AI use-cases to privacy focus applications
Several Google products already use AICore on Pixel, including Pixel Voice Recorder and Gboard.
Pixel voice recorder
The Pixel Voice Recorder application uses Gemini Nano and AICore to power an on-device summarization feature.
Gboard smart reply
Gboard leverages on-device Gemini Nano with AICore to provide accurate smart replies.
-
Gemini Nano can be enabled on Pixel 8 devices as a developer option. ↩
-
Gemini Nano can be enabled on Pixel 8a devices as a developer option. ↩