Python: [Bug]: AG-UI InputContent in Messages Lost During Conversion Due to `source.value` Not Being Checked

[savannahknight]

Description

What happened?

When input content are sent using the AG-UI discriminated union format (with type: "document", type: "image", etc.), the attachments are completely lost during the AG-UI → MAF message conversion in agent-framework-ag-ui.
Specifically, when a document is sent with this structure per the ag-ui-protocol specification:

{
  "type": "document",
  "source": {
    "type": "url",
    "value": "https://example.com/files/document.pdf",
    "mime_type": "application/pdf"
  },
  "metadata": {
    "filename": "document.pdf",
    "id": "file-123"
  }
}

When the AgentFrameworkWorkflow implementation provided by agent-framework-ag-ui runs, it first adapts the message to MAF format. The conversion logic in _extract_multimodal_source_fields() does not extract the URL from source.value, causing the entire file attachment to be dropped (returns None from _parse_multimodal_media_part()).

What did you expect to happen?

The AG-UI → MAF converter should correctly extract the URL from source.value when source.type is "url" or "uri", creating a proper MAF Content.from_uri() object that preserves the file attachment through the conversion process.

This behavior should be consistent with:

1. The ag-ui-protocol specification which defines InputContentUrlSource with a value field (not url or uri or data)
2. How the converter handles base64 data, where it correctly checks source.value when source.type is "base64" (line 383 in agent_framework_ag_ui/_message_adapters.py)

• According to the AG-UI discriminated union pattern, when source.type identifies the source format, the actual content goes in source.value - this should be consistent for both base64 payloads and URLs.

Backward compatibility note
The old AG-UI format (with type: "binary" and a top-level url field) works correctly because line 368 extracts the top-level url:

{
  "type": "binary",
  "url": "https://example.com/files/document.pdf",
  "mimeType": "application/pdf"
}

This format is still supported and works. The issue only affects the newer discriminated union format with nested source objects.

Issues with Current Implementation

1. Mismatch with ag-ui-protocol Dependency
   The agent-framework-ag-ui package declares a dependency on ag-ui-protocol>=0.1.19,<0.2 (

   agent-framework/python/packages/ag-ui/pyproject.toml

   Line 26 in e926ad2

   "ag-ui-protocol>=0.1.19,<0.2",

   ), but the conversion code does not follow the types defined in that protocol.

According to the ag-ui-protocol specification:

• All the InputContent* types in the AGUI protocol use value field:

class InputContentUrlSource(ConfiguredBaseModel):
    """URL-referenced source."""
    type: Literal["url"] = "url"
    value: str  # ← The URL goes in 'value'
    mime_type: Optional[str] = None

2. Tests Use Wrong Field Names
   The test suite in test_message_adapters.py contains multiple tests that use incorrect field names (source.data instead of source.value), which explains why this bug was not caught.

• Example 1: link:

{
    "type": "audio",
    "source": {"type": "base64", "data": payload, "mimeType": "audio/wav"},
    #                             ^^^^ Should be "value" per ag-ui-protocol
}

Question: Why does agent-framework-ag-ui use raw dict[str, Any] types for AG-UI message parsing instead of importing and using the actual typed classes from the ag-ui-protocol package it depends on?

Current implementation:

def _parse_multimodal_media_part(part: dict[str, Any]) -> Content | None:
    # Manual field extraction with string keys
    part_type = str(part.get("type", "")).lower()
    url, data, binary_id, mime_type = _extract_multimodal_source_fields(part)
    # ...

If using ag-ui-protocol types:

from ag_ui.core.types import (
    DocumentInputContent,
    ImageInputContent,
    InputContentUrlSource,
    InputContentDataSource
)

def _parse_multimodal_media_part(
    part: DocumentInputContent | ImageInputContent | AudioInputContent | VideoInputContent
) -> Content | None:
    # Type-safe field access with proper validation
    if isinstance(part.source, InputContentUrlSource):
        return Content.from_uri(uri=part.source.value, media_type=part.source.mime_type)
    elif isinstance(part.source, InputContentDataSource):
        # ... handle base64 data

Benefits of using protocol types:

1. Compile-time type checking would have caught the source.url vs source.value bug
2. Runtime validation via Pydantic would reject malformed inputs
3. Guaranteed compatibility with ag-ui-protocol spec

Summary

The legacy type: "binary" format for AGUI input conversion continues to work because it uses a top-level url field, which is correctly extracted at line 368. However, applications following the current AG-UI specification (using discriminated unions with nested source objects as defined in the ag-ui-protocol package) cannot successfully send file attachments.

Code Sample

Error Messages / Stack Traces

Package Versions

agent-framework-core: 1.13.0, agent-framework-ag-ui: 1.0.1

Python Version

Python >=3.12,<3.13

Additional Context

No response

[github-actions]

🤖 Automated triage reproduction notes (agent-authored — trust but verify)

Agent analysis

Bug is in python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py::_extract_multimodal_source_fields() at line 381. When source_type in {"url", "uri"}, the code extracts source_dict.get("url") or source_dict.get("uri") but does not check source_dict.get("value"), which is what ag-ui-protocol's InputContentUrlSource uses. Minimal repro: call _extract_multimodal_source_fields({"type": "document", "source": {"type": "url", "value": "https://example.com/doc.pdf", "mime_type": "application/pdf"}}) and observe url is None. The existing tests at lines 504/537/1437/1456 in test_message_adapters.py all use the non-spec "url" field name instead of "value", masking the defect.

• Failing test: python/packages/ag-ui/tests/ag_ui/test_repro_7653.py
• Files examined: python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py, python/packages/ag-ui/pyproject.toml, python/packages/ag-ui/tests/ag_ui/test_message_adapters.py
• Tests run: tests/ag_ui/test_repro_7653.py (4 tests: 2 FAILED demonstrating the bug, 2 PASSED showing base64 and legacy formats work)
• Reported version: 1.0.1
• Current version: 1.0.1

[atty57]

I'd like to take this one - I have a fix ready locally.

_extract_multimodal_source_fields reads source.value first for url-typed sources, falling back to source.url/source.uri so the existing non-spec shape keeps working. Added tests for both shapes; poe test -P ag-ui is green. PR to follow.