AI Foundry Missing Output Traces For Tool-Based Questions

Hey guys! Let's dive into a tricky issue we've been seeing with the Python Agent Framework. Specifically, we're noticing that when using the code to upload tracing data to AI Foundry, the traces are captured and uploaded correctly for general conversational questions. However, when the agent invokes a tool – like when a user asks a tool-related question – only the input trace gets uploaded, and the crucial output trace goes missing. This can make debugging and understanding the agent's behavior in these scenarios super challenging, so let's break it down and see what's going on.

The Problem: Missing Output Traces

Imagine you're building an intelligent agent that can answer questions and perform tasks using various tools. You've set up tracing to monitor the agent's activities and upload the data to AI Foundry for analysis. For simple questions, everything works like a charm. But when a user asks something that requires the agent to use a tool – say, fetching weather information – you notice that only the initial question (the input trace) shows up in AI Foundry. The agent's response and the results from the tool (the output trace) are nowhere to be found. This is the core of the issue we're tackling.

Why are Output Traces Important?

Output traces are vital for several reasons:

• Debugging: They provide a clear picture of what the agent did after receiving the input, including how it processed the information and what actions it took.
• Understanding Agent Behavior: By examining the output traces, you can see exactly how the agent used the tool, what data it received, and how it formulated its response. This is crucial for ensuring the agent is behaving as expected.
• Performance Monitoring: Output traces can help identify bottlenecks or inefficiencies in the agent's workflow, especially when dealing with tools.
• Auditing and Compliance: In some applications, it's essential to have a complete record of the agent's interactions, including both input and output data, for auditing and compliance purposes.

Without output traces, you're essentially flying blind when it comes to tool-based interactions. You know the question was asked, but you have no insight into how the agent handled it.

Visualizing the Issue

To illustrate the problem, here are a couple of screenshots that highlight the missing output traces.

This image shows that while the input is captured, the corresponding output is absent.

This second image further emphasizes the issue within AI Foundry, where the expected traces are not visible.

Code Example: Setting Up Tracing

To better understand the context, let's look at a code example that demonstrates how tracing is set up in the Python Agent Framework.

import os
os.environ["AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED"] = "true"
import asyncio
from random import randint
from typing import Annotated
import dotenv
from agent_framework import ChatAgent
from agent_framework.azure import AzureAIAgentClient
from agent_framework.observability import get_tracer, setup_observability
from azure.ai.projects.aio import AIProjectClient
from azure.identity.aio import DefaultAzureCredential
from azure.monitor.opentelemetry import configure_azure_monitor
from opentelemetry.trace import SpanKind, format_trace_id
from pydantic import Field
 
dotenv.load_dotenv()
 
async def get_weather(
    location: Annotated[str, Field(description="The location to get the weather for.")],
) -> str:
    await asyncio.sleep(randint(0, 10) / 10.0)
    conditions = ["sunny", "cloudy", "rainy", "stormy"]
    return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
 
 
async def main():
    conn_str = os.environ.get("APPLICATIONINSIGHTS_CONNECTION_STRING")
    if not conn_str:
        raise RuntimeError("Please set APPLICATIONINSIGHTS_CONNECTION_STRING env var")
 

    configure_azure_monitor(connection_string=conn_str, enable_live_metrics=True)
    setup_observability()
 
    async with (
        DefaultAzureCredential() as credential,
        AIProjectClient(endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"], credential=credential) as client,
    ):
        with get_tracer().start_as_current_span("SessionID_"+"1111", kind=SpanKind.CLIENT) as current_span:
            print(f"Trace ID: {format_trace_id(current_span.get_span_context().trace_id)}")
 
            async with ChatAgent(
                chat_client=AzureAIAgentClient(project_client=client, agent_id="asst_zdyA5Qiw07ShZlldW9J6LSAD"),
                instructions="You are a helpful weather agent.",
                tools=get_weather,
            ) as agent:
                result = await agent.run("What's the weather in Amsterdam?")
                print("Result:", result)
 
if __name__ == "__main__":
    asyncio.run(main())

Let's break down what this code does:

1. Environment Setup: The code starts by setting the AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED environment variable to `