Once you identify your in-app functionality and corresponding built-in intent
(BII) to implement, declare the BIIs your functionality supports by defining
a capability
element in a shortcuts.xml
resource file. Declaring a BII
as a capability
registers support for that semantic intent in your app, and
enables voice query fulfillment of the intent using Google Assistant.
Assistant uses natural language processing to extract parameters from a user
query. The built-in intents reference lists the fields that each BII is
capable of extracting from an associated user query. For example, if a user
invokes the [actions.intent.GET_FOOD_OBSERVATION
][] capability in your app by
saying, "Hey Google, Ask ExampleApp what did I eat for Lunch last Friday", Assistant
extracts the following BII parameters from the user request:
foodObservation.forMeal
= "https://schema.googleapis.com/MealTypeLunch"foodObservation.startTime
= "2024-09-06T00:00:00"foodObservation.endTime
= "2024-09-06T23:59:59"
Assistant passes BII parameters to the fulfillment intent
defined in the
capability
. One or more intent
elements can be defined in a capability to
accommodate the different ways a user might invoke a BII. For instance, you
could define a fulfillment intent
that requires both BII parameters in the
above example. You could then define a second intent that requires a single BII
parameter, foodObservation.forMeal
, that reports for all meals on a particular day, like "Hey Google, Ask ExampleApp what did I eat for Lunch."
Overview
You configure App Actions using a shortcuts.xml
file placed in your app
project's res/xml
directory, and then creating a reference to shortcuts.xml
in your app manifest. Add a reference to shortcuts.xml
in your app manifest
by following these steps:
In your app's manifest file (
AndroidManifest.xml
), find an activity whose intent filters are set to theandroid.intent.action.MAIN
action and theandroid.intent.category.LAUNCHER
category.Add a reference to
shortcuts.xml
inAndroidManifest.xml
using a<meta-data>
tag in theActivity
that has intent filters for bothMAIN
andLAUNCHER
, as follows:<meta-data android:name="android.app.shortcuts" android:resource="@xml/shortcuts" />
The above example declares an XML resource for the xml/shortcuts.xml
file in
the APK. For more details on configuring shortcuts, see
Create static shortcuts in the Android developer documentation.
The Jetpack library androidx.core:core:1.6.0
(or greater) is required in your Android project to avoid compilation errors
when defining App Actions capabilities in shortcuts.xml
. For details, see
Getting started with Android Jetpack.
Static shortcuts
When defining your capability
, you can declare static shortcut
elements in
shortcuts.xml
to extend the functionality of the capability. Static shortcuts
are ingested by Assistant when you upload a release to Google Play Console.
Since static shortcuts can only be created and updated by creating new releases,
they are most useful to highlight common activities and content in your app.
You can enable the following App Actions functionality with static shortcuts:
Capability shortcuts. Create shortcuts that launch an instance of your
capability
containing predefinedintent
parameter values. For example, you could declare an app shortcut "Start a run" which invokes theSTART_EXERCISE
BII capability in your fitness app.These shortcuts contain
intent
,shortLabel
, andlongLabel
attributes, making them eligible to be suggested and fulfilled as chips in proactive surfaces, such as Assistant or when long-pressing an app icon on Android launchers. An action shortcut can also serve as an entity shortcut, detailed below, by associating it to acapability
using a<capability-binding>
tag.Entity shortcuts. Entity shortcuts provide a list of supported parameter values for voice query fulfillment of a
capability
. For example, an entity shortcut with a list of exercise types ("hike," "run," etc.) bound to theexercise.name
BII parameter of theSTART_EXERCISE
capability. If a user utterance matches an entity, theshortcutId
ID is passed to the intent instead of the raw user query value.Entity
shortcuts don't defineintent
,shortLabel
, orlongLabel
attributes, and as such are not suggested on proactive surfaces. For details, see Inline inventory for App Actions.
Capability Schema
The following table describes the App Actions schema for capability
elements
in shortcuts.xml
. When including a tag, all of its attributes are required
unless marked "optional".
Shortcuts.xml tag | Contained in | Attributes |
---|---|---|
<capability> |
<shortcuts> |
|
<intent> |
<capability> |
|
<url-template> |
<intent> |
|
<extra> |
<intent> |
Only applicable for foreground app invocation |
<parameter> |
<intent> |
|
<shortcut-fulfillment> |
<capability> |
Only applicable for inline inventory |
<parameter> |
<shortcut-fulfillment> |
android:name |
<slice> |
<capability> |
Only applicable for Android Slices |
Capability schema description
This section describes the capability
schema elements.
<capability>
A capability
that defines the App Action intent your app supports. Each
<capability>
element in your shortcuts.xml
file must provide at least one
<intent>
to handle fulfillment of the action.
Attributes:
android:name
: Built-in intent Action ID (for example, [actions.intent.GET_FOOD_OBSERVATION
][]). For a list of supported built-in intents, see the built-in intent reference.app:queryPatterns
: A string array resource of queries expected from the user for this intent. This attribute is only applicable to custom intents, as BIIs already include models of the common ways users express the tasks they're trying to do, or the information they seek.
<intent>
Android intent
element defining how a user query should be
fulfilled using in-app functionality. Developers may provide multiple <intent>
tags in a capability
. Assistant attempts to fulfill a user query using the
first <intent>
in a capability
for which all required parameters are
provided.
Attributes:
android:action
: the intentAction
type. Defaults toACTION_VIEW
.android:targetClass
: Target Activity class, for example:"com.example.exercise.ExerciseActivity"
android:targetPackage
: Package containing the target Activity class, for example:"com.example.exercise
android:data
: This field is overwritten by<url-template>
if that tag is declared in theintent
.
<url-template>
Template for constructing a deep link URI to be opened on the device. The template may be expanded with built-in intent parameters if all required parameters for the template are available. For examples of the HTTP URL template, see the Wikipedia article on URL templates. The template format follows the RFC6570 URI template specification.
The following are some examples of URL template values:
Template | Values | Expanded value |
---|---|---|
https://example.com/test{?foo,bar} |
"foo": "123"
|
https://example.com/test?foo=123&bar=456 |
https://example.com/test?utm_campaign=appactions{&foo,bar} |
"foo": "123"
|
https://example.com/test?utm_campaign=appactions&foo=123&bar=456 |
https://example.com/test?utm_campaign=appactions{#foo} |
"foo": "123" |
https://example.com/test?utm_campaign=appactions#foo=123 |
myapp://example/{foo} |
"foo": "123" |
myapp://example/123 |
For more about configuring URL templates, see URL templates in fulfillment.
<extra>
Defines extra data for an intent
. For App Actions, this field is only used to
enable [foreground app invocation][] for a capability
.
<parameter>
Maps a BII parameter to intent parameter values. For more information, see Parameter data and matching.
Attributes:
android:name
: Name of the BII parameter to associate with thisintent
parameter. The name should be a leaf-level field of the BII parameter (for example,foodObservation.aboutFood.name
).android:key
: Developer-defined key of a BII parameter value. For example, you might definecontact_name
for themessage.recipient.name
BII parameter.android:mimeType
: The mimeType of the parameter, such astext/*
. This field is only required for parameters of custom intents.android:required
: Declares whether the user query needs to include this parameter for this intent to be used for fulfillment. If the parameter is not available, Assistant attempts to fulfill the user query using the nextintent
defined for thecapability
.
<shortcut-fulfillment>
Specifies that an intent
defined in an inline inventory shortcut for a
specified parameter be used for fulfillment.
For details, see Fulfillment using shortcut intents.
<parameter> (for <shortcut-fulfillment>
)
Optional attribute that maps a single BII parameter to inline inventory shortcut fulfillment. For details, see Fulfillment using shortcut intents.
Attribute:
android:name
: Name of the BII parameter to associate to inline inventory shortcut fulfillment. The name should be a leaf-level field of the BII parameter (for example,menuItem.name
).
<slice>
Enables Assistant to embed the result of a query matching this capability
as
an Android Slice. For details, see
Integrate App Actions with Android Slices.
Shortcut schema
The following table describes attributes of shortcut
elements that are used to
enable App Actions functionality. When including a tag, all of its attributes
are required unless marked "optional".
Shortcuts.xml tag | Contained in | Attributes |
---|---|---|
<shortcut> |
<shortcuts> |
|
<intent> |
<shortcut> |
|
<capability-binding> |
|
|
<parameter-binding> |
<capability-binding> |
|
<extra> |
<shortcut> |
Only applicable for Enum parameter matching. |
Shortcut schema description
This section describes the shortcut
schema elements.
<shortcut>
An Android <shortcut>
defined in shortcuts.xml
with certain attributes
that are relevant for App Actions. String values for the shortcutShortLabel
and shortcutLongLabel
fields are referenced via the APK's
string resources.
Attributes:
android:shortcutId
: Identifier for this shortcut.android:shortcutShortLabel
: String resource representing a brief shortcut phrase. For example,"@string/callDavidShort"
representing the value "Call David."android:shortcutLongLabel
: String resource representing a long shortcut phrase. For example,"@string/callDavidLong"
representing the value "Make an audio call to David."
<intent>
Android intent associated to this shortcut. This intent
is executed when a
user launches this shortcut using voice or touch.
shortcut
intent attributes are identical to capability
intent
attributes.
<capability-binding>
Associates a shortcut
to an App Actions capability
. Adding this element to
a shortcut
enables it for voice fulfillment using Assistant
.
Attributes:
android:key
: Theandroid:name
attribute of thecapability
thisshortcut
is bound to. For example,actions.intent.START_EXERCISE
.
<parameter-binding>
Optional attribute that associates a shortcut
to a single parameter of an App
Actions capability
. If a parameter-binding
is defined for a shortcut
, the
shortcut can be used to provide an inline inventory entity to a BII parameter.
For more details, see Inline inventory for App Actions.
Attributes:
android:key
: The name of thecapability
BII parameter to associate this shortcut to. For example,exercise.name
.android:value
: theentity
value. This can be a singleentity
or a resource list.
<extra>
The extra
bundle data for the shortcut. sameAs is the only data
relevant to App Actions shortcut
elements. The sameAs URL refers to a
reference web page that unambiguously identifies the entity. Used to specify an
enum value if and only if the intent parameter type is a subtype of
schema.org/Enumeration. It is required for parameter fields
whose types are subtypes of schema.org/Enumeration
(for example:
MealTypeBreakfast
).
Attributes:
android:key
: The supported value for App Actions is:sameAs
android:value
: ThesameAs
URL value
For more details, see Matching enumerated parameter values.
Intent fulfillment options
You define intent
elements within a <capability>
to declare how Assistant
responds to, or fulfills, user voice commands that match that capability. There
are several ways to configure how an intent
launches a fulfillment destination
in your app, depending on how your app navigation is structured.
The following fulfillment options are available:
Explicit intents: Launch a specific app component by defining the
targetClass
andtargetPackage
attributes for theintent
. This is the recommended App Actions fulfillment method.Deep links: Launch app destinations using Android deep links by defining a
<url-template>
tag within theintent
element. This method is useful if your app navigation already relies on deep links.Intent data: You can provide a fulfillment URI in the
intent
android:data
attribute. This field is overwritten by<url-template>
data if that tag is also defined within theintent
.
Parameter data and matching
By default, Assistant sends BII parameters extracted from the user query to your
app as extra
data of the Android intent
defined in the capability
.
Alternately, you can declare a <url-template>
tag in the
capability
that contains placeholders for dynamic parameters. This template
maps to one of your Android activities, using an App Links URL,
a custom scheme, or an Intent-based URL.
Using intent Extras
The following example demonstrates an explicit intent defined for a capability
fulfillment:
<capability android:name="actions.intent.START_EXERCISE">
<intent
android:targetPackage="com.example.myapp"
android:targetClass="com.example.myapp.ExerciseActivity">
<parameter android:name="exercise.name" android:key="exercise" />
</intent>
</capability>
Given the above sample, for a user query like, "Hey Google, order a latte from
ExampleApp," the app receives an intent
that invokes the component:
targetPackage
, targetClass
. The component receives an Extra with
key = "exercise"
, value = "Running"
.
Using a URL template for Android deep links
If your app is already able to handle app-linked URLs, with dynamic parameters,
you can define a <url-template>
in the intent
to generate Android
deep links for fulfillment. The following sample defines a <url-template>
:
<capability android:name="actions.intent.START_EXERCISE">
<intent>
<url-template android:value="myapp://start{?exercise}" />
<parameter android:name="exercise.name" android:key="exercise" />
</intent>
</capability>
Given the above sample, for a user query like, “Hey Google, order a latte from ExampleApp," the app receives the generated URL: "myapp://start?exercise=Running".
To map the BII parameter to a position in your URL, you use the
android:name
attribute of the <parameter>
tag. This attribute
corresponds to the android:key
value in the URL template that you want to
substitute with information from the user. The android:key
value must be present
in your <url-template>
and enclosed by curly braces ({}
).
Match enumerated parameter values
Some BII parameters provide enumerated values to your fulfillment intent, for
example, the supported text values of the RECORD_FOOD_OBSERVATION
BII. For
these parameters, Assistant matches the user's query ("Breakfast") to an
entity whose sameAs
value matches the enum schema URL
(https://schema.googleapis.com/MealTypeBreakfast
). To associate enum
values for a supported entity
, you declare a sameAs
association in
your shortcut
. The following sample demonstrates a sameAs
association for an
inline entity shortcut:
<shortcut android:shortcutId="meal_breakfast" >
<capability-binding android:key="actions.intent.RECORD_FOOD_OBSERVATION">
<parameter-binding android:key="foodObservation.forMeal" />
</capability-binding>
<extra
android:key="sameAs"
android:value="http://schema.googleapis.com/MealTypeBreakfast" />
</shortcut>
<capability android:name="actions.intent.RECORD_FOOD_OBSERVATION">
<intent targetPackage="com.example.app" targetClass="com.example.app.Class">
<parameter android:name="foodObservation.forMeal" android:key="for_meal" />
</intent>
</capability>
In the above example, if the RECORD_FOOD_OBSERVATION
capability triggers a
match for the "breakfast" meal type, the following Extra is sent with the
fulfillment intent
:
key = "for_meal"
value = "meal_breakfast"
Features
The following App Actions features are available in shortcuts.xml
.
Inline inventory for App Actions
For some BII parameters, shortcuts can be used to guide entity
extraction to a set of supported entities specified in shortcuts.xml
, known as
inline inventory. For details, see Inline inventory.
Custom intents
Custom intents can be declared in shortcuts.xml
to voice enable features in
your app that don't match available BIIs. While similar in
functionality to a BII definition, custom intents require two additional
attributes in shortcuts.xml
:
app:queryPatterns
: Array resource that declares the different query patterns for a custom intent.android:mimeType
: Parameter type of a custom intent. This field is not required for BIIs, where the parameter type is known. For custom intent parameters, a supported semantic type must be declared.
For more details, see Custom intents.