feat: add Active Agent monitoring plugin

[joshuap]

This PR adds Active Support instrumentation to capture the following Active Agent events:

• Core events: prompt.active_agent, embed.active_agent (excluding .provider.active_agent events)
• Streaming events: stream_open.active_agent, stream_close.active_agent, stream_chunk.active_agent
• Tool and Processing events: tool_call.active_agent
• Infrastructure events: process.active_agent

The raw events look like this:

Open questions

• Do we need special handling for exceptions? Honeybadger reports unhandled exceptions in jobs and web requests automatically, so additional error reporting my be unnecessary—unless we can provide additional context or catch exceptions that are rescued internally. (For example, Sidekiq provides its own API for error handling.)
• Do we want to subscribe to .provider.active_agent events for individual API calls in multi-turn conversions? If so, we may want to unsubscribe from prompt.active_agent and embed.active_agent events to avoid duplicates. I'm not sure if that would prevent us from showing the total duration for prompts, though.
• Do we want to collect metrics? We can create charts from the the values in the instrumentation events, but rolling up metrics can sometimes make charting easier (and users can fine-tune the data they wish to send).
• What charts/data would be useful for the majority of users to see on a default monitoring dashboard? I'm not sure if a full dashboard for Active Agent makes sense yet.

TODO

• Add some simple unit tests to verify the plugin installation/subscriptions
• Create a Honeybadger automatic dashboard template

Reference

• Active Agent instrumentation docs
• Active Agent error handling docs
• Honeybadger event reporting docs
• Honeybadger metric reporting docs
• Honeybadger plugin guide

[Copilot]

Pull request overview

This PR adds a new plugin for monitoring Active Agent, a framework for building AI agents. The plugin provides both exception tracking and insights collection capabilities similar to other Honeybadger integrations.

• Adds a new Active Agent plugin with error handling and event subscription
• Configures insights collection for Active Agent events (prompts, embeddings, streaming, tool calls, and processing)
• Adds configuration option to enable/disable Active Agent insights

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
lib/honeybadger/plugins/active_agent.rb	New plugin that registers Active Agent monitoring, subscribes to Active Agent notification events, and provides hooks for exception reporting
lib/honeybadger/config/defaults.rb	Adds active_agent.insights.enabled configuration option (defaults to true) for controlling automatic data collection

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[joshuap]

@TonsOfFun do you have a sample Rails app that uses all of Active Agent's features by any chance? That would be useful for testing (I made a sample app, but it doesn't do tool calls, embeds, or streaming, and I'm not sure how to simulate errors).

[TonsOfFun]

https://github.com/TonsOfFun/writebook this should have a few feature examples you can check out. Let me know if you have any questions!

[sirwolfgang]

The main application I'm using ActiveAgents for is also supported by HB; I can test some of this stuff in prod once it's ready.

[joshuap]

The main application I'm using ActiveAgents for is also supported by HB; I can test some of this stuff in prod once it's ready.

Amazing, I'll let you know when it's ready to test!

@TonsOfFun I got your writebook running, but I'll need to upgrade active agent to 1.0 since that's what I'm targeting for telemetry (I'm assuming that's the right target).

[sirwolfgang]

Do we need special handling for exceptions?

Currently AA is pretty agnostic on the exception handling. Prior to v1, there was some generic retry logic. This has been removed in favor of native retry logic built inside of the LLM provider's native gems. So right now the errors bubble and report to HB just like any other error.

It's not clear to me how much AA needs/should be involved with error handling. A lot of it seems out of scope to me, but I'm also getting errors from the MCPs bubble back to the LLM which effectively blow the entire conversational stack. Admittedly I don't really use HB insights at the moment, some people on my team do; but this area likely the area exploration when it comes to errors, since this can make errors pretty expensive.

[joshuap]

It's not clear to me how much AA needs/should be involved with error handling. A lot of it seems out of scope to me, but I'm also getting errors from the MCPs bubble back to the LLM which effectively blow the entire conversational stack. Admittedly I don't really use HB insights at the moment, some people on my team do; but this area likely the area exploration when it comes to errors, since this can make errors pretty expensive.

Yeah, that's what I'm thinking too. I'm going to research some of the callbacks and integration points — we may be able to add some additional context to errors that happen in Active Agent actions (similar to how we do for Action Controller in Rails). But it sounds like reporting the errors themselves should be the responsibility of our standard Rails middleware and/or the user if they are handling errors internally.