Provides app functions related functionalities, allowing apps to execute, check, and set the enabled state of specific system functionalities.
Google is developing new capabilities for its AI assistant Gemini in Android 16, allowing it to perform actions within apps, similar to Apple's plans for iOS 18.
Google is developing new features for its AI assistant, Gemini, through an API called "app functions" in the Android 16 developer preview. This API allows app developers to expose specific functionalities to the system, enabling Gemini to perform actions within apps without needing to open them directly. For example, users could order food from a restaurant using Gemini without launching a food delivery app.
This development is similar to Apple's efforts in iOS 18, where Siri is gaining the ability to take actions in apps. While Apple's update is expected in spring 2025, Google's integration could provide users with a more integrated and useful AI assistant experience. Currently, Gemini can access information in some apps and Siri can handle more complex queries, but both assistants are yet to fully realize their potential to "do things for you."
The advancements hint at a significant evolution in how AI assistants function on smartphones in 2025.
Google is developing new features for its AI assistant, Gemini, through an API called "app functions" in the Android 16 developer preview. This API allows app developers to expose specific functionalities to the system, enabling Gemini to perform actions within apps without needing to open them directly. For example, users could order food from a restaurant using Gemini without launching a food delivery app.
This development is similar to Apple's efforts in iOS 18, where Siri is gaining the ability to take actions in apps. While Apple's update is expected in spring 2025, Google's integration could provide users with a more integrated and useful AI assistant experience. Currently, Gemini can access information in some apps and Siri can handle more complex queries, but both assistants are yet to fully realize their potential to "do things for you."
The advancements hint at a significant evolution in how AI assistants function on smartphones in 2025.
The article discusses the role of AI agents in generative AI, focusing on tool calling and reasoning abilities, and how they can be evaluated using benchmarks like BFCL and Nexus Function Calling Benchmark.
Hugging Face introduces a unified tool use API across multiple model families, making it easier to implement tool use in language models.
Hugging Face has extended chat templates to support tools, offering a unified approach to tool use with the following features:
- Defining tools: Tools can be defined using JSON schema or Python functions with clear names, accurate type hints, and complete docstrings.
- Adding tool calls to the chat: Tool calls are added as a field of assistant messages, including the tool type, name, and arguments.
- Adding tool responses to the chat: Tool responses are added as tool messages containing the tool name and content.
Hugging Face has extended chat templates to support tools, offering a unified approach to tool use with the following features:
- Defining tools: Tools can be defined using JSON schema or Python functions with clear names, accurate type hints, and complete docstrings.
- Adding tool calls to the chat: Tool calls are added as a field of assistant messages, including the tool type, name, and arguments.
- Adding tool responses to the chat: Tool responses are added as tool messages containing the tool name and content.
Extract structured data from remote or local LLM models. Predictable output is essential for any serious use of LLMs.
Extract data into Pydantic objects, dataclasses or simple types.
Same API for local file models and remote OpenAI, Mistral AI and other models.
Model management: download models, manage configuration, quickly switch between models.
Tools for evaluating output across local/remote models, for chat-like interaction and more.
No matter how well you craft a prompt begging a model for the output you need, it can always respond something else. Extracting structured data can be a big step into getting predictable behavior from your models.
Extract data into Pydantic objects, dataclasses or simple types.
Same API for local file models and remote OpenAI, Mistral AI and other models.
Model management: download models, manage configuration, quickly switch between models.
Tools for evaluating output across local/remote models, for chat-like interaction and more.
No matter how well you craft a prompt begging a model for the output you need, it can always respond something else. Extracting structured data can be a big step into getting predictable behavior from your models.
Powerful Data Analysis and Plotting via Natural Language Requests by Giving LLMs Access to LibrariesGetting LLMs to analyze and plot data for you, right in your web browser
The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). It provides a simple yet robust interface using llama-cpp-python, allowing users to chat with LLM models, execute structured function calls and get structured output.
