bug #838 [Agent] Fix sources metadata unavailable during streaming (Copilot)

OskarStark · OskarStark · commit 34fd7940587f · 2025-11-14T08:33:39.000+01:00
This PR was squashed before being merged into the main branch. Discussion ---------- [Agent] Fix sources metadata unavailable during streaming | Q | A | ------------- | --- | Bug fix? | yes | New feature? | no | Docs? | no | Issues | Fix #833 | License | MIT - [x] Analyze the issue with sources metadata being null during streaming - [x] Identify the root cause in ToolboxStreamResult - [x] Implement fix to propagate metadata from inner result to outer result - [x] Add test for streaming with sources - [x] Fix trailing whitespace in test (code quality) - [x] Fix "yield from" error with non-iterable content - [x] Simplify content type check for better maintainability - [x] Improve code comments and comparison consistency - [x] Add safety check for non-iterable objects - [x] Revert unnecessary .gitignore change - [x] Apply Symfony code style conventions ## Summary Fixed issue where sources metadata was inaccessible when using streaming with the Agent. The problem was that metadata was set on the inner result but never propagated to the outer StreamResult object. **Changes:** - Modified `StreamResult::getContent()` to capture and propagate metadata from inner result - Added comprehensive type checking to safely handle all content types - Added test `testSourcesEndUpInResultMetadataWithStreaming()` to verify the fix works correctly - Applied Symfony code style conventions (Yoda conditions, fully qualified function names)  <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>[Agent] Unable to access sources metadata with streaming</issue_title> > <issue_description>I am implementing the Symfony AI Agent in combination with the [ai-sdk](https://ai-sdk.dev/getting-started). I want to display the used sources in the UI, so I want to dispatch these in an event stream using the ai-sdk Data Stream Protocol. However, during streaming, the metadata seems to reset and sources are always `null`. Without the stream option this works fine. > > Should this be done in a different order? The sources are present at some point, but seem to reset while actually streaming the content back. > > This is basically all that I am doing: > > > ```php > public function __invoke(ChatRequest $request): StreamedResponse > { > $platform = PlatformFactory::create(env('OPENAI_API_KEY'), HttpClient::create()); > > $toolBox = new ToolBox([ > new RandomTool(), > ]); > $processor = new AgentProcessor($toolBox, includeSources: true); > $agent = new Agent($platform, 'gpt-4o-mini', [$processor], [$processor]); > > $messages = new MessageBag(); > foreach ($request->messages() as $message) { > foreach ($message['parts'] as $part) { > if ($part['type'] !== 'text') { > continue; > } > $messages->add(Message::ofUser($part['text'])); > } > } > > $result = $agent->call($messages, [ > 'stream' => true > ]); > > $response = new StreamedResponse(function () use ($result) { > $messageId = 'msg_' . bin2hex(random_bytes(16)); > $textBlockId = 'text_' . bin2hex(random_bytes(16)); > > $this->sendSSE(['type' => 'start', 'messageId' => $messageId]); > $this->sendSSE(['type' => 'text-start', 'id' => $textBlockId]); > > foreach ($result->getContent() as $chunk) { > $text = (string)$chunk; > if ($text !== '') { > $this->sendSSE([ > 'type' => 'text-delta', > 'id' => $textBlockId, > 'delta' => $text > ]); > } > } > > $this->sendSSE(['type' => 'text-end', 'id' => $textBlockId]); > > // This is always null > $sources = $result->getMetadata()->get('sources', []); > foreach ($sources as $source) { > if (!$source instanceof Source) { > continue; > } > $this->sendSSE([ > 'type' => 'source-url', > 'sourceId' => $source->getReference(), > 'url' => $source->getReference(), > 'title' => $source->getName() > ]); > } > > $this->sendSSE(['type' => 'finish']); > > echo "data: [DONE]\n\n"; > flush(); > }); > > > $response->headers->set('Content-Type', 'text/event-stream'); > $response->headers->set('Cache-Control', 'no-cache'); > $response->headers->set('Connection', 'keep-alive'); > $response->headers->set('X-Accel-Buffering', 'no'); > $response->headers->set('x-vercel-ai-ui-message-stream', 'v1'); > > return $response; > } > > private function sendSSE(array $data): void > { > echo 'data: ' . json_encode($data, JSON_UNESCAPED_UNICODE) . "\n\n"; > flush(); > } > ``` > > > </issue_description> > > <agent_instructions>Try to fix this issue. do it as minimal as possible</agent_instructions> > > ## Comments on the Issue (you are `@copilot` in this section) > > <comments> > <comment_new><author>`@chr`-hertel</author><body> > Sounds like a legit bug to me - didn't test that yet as well. PR ofc welcome :)</body></comment_new> > </comments> > </details> - Fixes #833  --- ✨ Let Copilot coding agent [set things up for you](https://github.com/symfony/ai/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo. Commits ------- cfdc2d8 [Agent] Fix sources metadata unavailable during streaming
diff --git a/src/agent/src/Toolbox/StreamResult.php b/src/agent/src/Toolbox/StreamResult.php
@@ -31,7 +31,22 @@ public function getContent(): \Generator
         $streamedResult = '';
         foreach ($this->generator as $value) {
             if ($value instanceof ToolCallResult) {
-                yield from ($this->handleToolCallsCallback)($value, Message::ofAssistant($streamedResult))->getContent();
+                $innerResult = ($this->handleToolCallsCallback)($value, Message::ofAssistant($streamedResult));
+
+                // Propagate metadata from inner result to this result
+                foreach ($innerResult->getMetadata()->all() as $key => $metadataValue) {
+                    $this->getMetadata()->add($key, $metadataValue);
+                }
+
+                $content = $innerResult->getContent();
+                // Strings are iterable in PHP but yield from would iterate character-by-character.
+                // We need to yield the complete string as a single value to preserve streaming behavior.
+                // null should also be yielded as-is.
+                if (\is_string($content) || null === $content || !is_iterable($content)) {
+                    yield $content;
+                } else {
+                    yield from $content;
+                }
 
                 break;
             }
diff --git a/src/agent/tests/Toolbox/AgentProcessorTest.php b/src/agent/tests/Toolbox/AgentProcessorTest.php
@@ -26,6 +26,7 @@
 use Symfony\AI\Platform\PlatformInterface;
 use Symfony\AI\Platform\Result\DeferredResult;
 use Symfony\AI\Platform\Result\InMemoryRawResult;
+use Symfony\AI\Platform\Result\StreamResult as GenericStreamResult;
 use Symfony\AI\Platform\Result\TextResult;
 use Symfony\AI\Platform\Result\ToolCall;
 use Symfony\AI\Platform\Result\ToolCallResult;
@@ -233,4 +234,52 @@ public function testSourcesGetCollectedAcrossConsecutiveToolCalls()
         $this->assertCount(2, $metadata->get('sources'));
         $this->assertSame([$source1, $source2], $metadata->get('sources'));
     }
+
+    public function testSourcesEndUpInResultMetadataWithStreaming()
+    {
+        $toolCall = new ToolCall('call_1234', 'tool_sources', ['arg1' => 'value1']);
+        $source1 = new Source('Relevant Article 1', 'http://example.com/article1', 'Content of article about the topic');
+        $source2 = new Source('Relevant Article 2', 'http://example.com/article2', 'More content of article about the topic');
+        $toolbox = $this->createMock(ToolboxInterface::class);
+        $toolbox
+            ->expects($this->once())
+            ->method('execute')
+            ->willReturn(new ToolResult($toolCall, 'Response based on the two articles.', [$source1, $source2]));
+
+        $messageBag = new MessageBag();
+
+        // Create a generator that yields chunks and then a ToolCallResult
+        $generator = (function () use ($toolCall) {
+            yield 'chunk1';
+            yield 'chunk2';
+            yield new ToolCallResult($toolCall);
+        })();
+
+        $result = new GenericStreamResult($generator);
+
+        $agent = $this->createMock(AgentInterface::class);
+        $agent
+            ->expects($this->once())
+            ->method('call')
+            ->willReturn(new TextResult('Final response based on the two articles.'));
+
+        $processor = new AgentProcessor($toolbox, includeSources: true);
+        $processor->setAgent($agent);
+
+        $output = new Output('gpt-4', $result, $messageBag);
+
+        $processor->processOutput($output);
+
+        // Consume the stream
+        $content = '';
+        foreach ($output->getResult()->getContent() as $chunk) {
+            $content .= $chunk;
+        }
+
+        // After consuming the stream, metadata should be available
+        $metadata = $output->getResult()->getMetadata();
+        $this->assertTrue($metadata->has('sources'));
+        $this->assertCount(2, $metadata->get('sources'));
+        $this->assertSame([$source1, $source2], $metadata->get('sources'));
+    }
 }
