feat(live): Support live mode of sequential agent

Add a `task_completed` function to the agent so when a model finished the task, it can send a signal and the program knows it can go to next agent.

This cl include:
* Implements the `_run_live_impl` in `sequential_agent` so it can handle live case.
* Add an example for sequential agent.
* Improve error message for unimplemented _run_live_impl in other agents.

Note:
1. Compared to non-live case, live agents process a continuous streams of audio
or video, so it doesn't have a native way to tell if it's finished and should pass
to next agent or not. So we introduce a task_compelted() function so the
model can call this function to signal that it's finished the task and we
can move on to next agent.

2. live agents doesn't seems to be very useful or natural in parallel or loop agents so we don't implement it for now. If there is user demand, we can implement it easily using similar approach.

PiperOrigin-RevId: 758315430
This commit is contained in:
Hangfei Lin
2025-05-13 11:55:50 -07:00
committed by Copybara-Service
parent 39f78dc28f
commit 4188673b0f
7 changed files with 180 additions and 19 deletions

View File

@@ -254,13 +254,13 @@ class Runner:
"""Runs the agent in live mode (experimental feature).
Args:
session: The session to use. This parameter is deprecated, please use
`user_id` and `session_id` instead.
user_id: The user ID for the session. Required if `session` is None.
session_id: The session ID for the session. Required if `session` is
None.
live_request_queue: The queue for live requests.
run_config: The run config for the agent.
session: The session to use. This parameter is deprecated, please use
`user_id` and `session_id` instead.
Yields:
AsyncGenerator[Event, None]: An asynchronous generator that yields
@@ -302,22 +302,24 @@ class Runner:
invocation_context.active_streaming_tools = {}
# TODO(hangfei): switch to use canonical_tools.
for tool in invocation_context.agent.tools:
# replicate a LiveRequestQueue for streaming tools that relis on
# LiveRequestQueue
from typing import get_type_hints
# for shell agents, there is no tools associated with it so we should skip.
if hasattr(invocation_context.agent, 'tools'):
for tool in invocation_context.agent.tools:
# replicate a LiveRequestQueue for streaming tools that relis on
# LiveRequestQueue
from typing import get_type_hints
type_hints = get_type_hints(tool)
for arg_type in type_hints.values():
if arg_type is LiveRequestQueue:
if not invocation_context.active_streaming_tools:
invocation_context.active_streaming_tools = {}
active_streaming_tools = ActiveStreamingTool(
stream=LiveRequestQueue()
)
invocation_context.active_streaming_tools[tool.__name__] = (
active_streaming_tools
)
type_hints = get_type_hints(tool)
for arg_type in type_hints.values():
if arg_type is LiveRequestQueue:
if not invocation_context.active_streaming_tools:
invocation_context.active_streaming_tools = {}
active_streaming_tools = ActiveStreamingTool(
stream=LiveRequestQueue()
)
invocation_context.active_streaming_tools[tool.__name__] = (
active_streaming_tools
)
async for event in invocation_context.agent.run_live(invocation_context):
self.session_service.append_event(session=session, event=event)