Skip to content

Commit 94eae24

Browse files
Python: DevUI: Add OpenAI Responses API proxy support + HIL for Workflows (#1737)
* DevUI: Add OpenAI Responses API proxy support with enhanced UI features This commit adds support for proxying requests to OpenAI's Responses API, allowing DevUI to route conversations to OpenAI models when configured to enable testing. Backend changes: - Add OpenAI proxy executor with conversation routing logic - Enhance event mapper to support OpenAI Responses API format - Extend server endpoints to handle OpenAI proxy mode - Update models with OpenAI-specific response types - Remove emojis from logging and CLI output for cleaner text Frontend changes: - Add settings modal with OpenAI proxy configuration UI - Enhance agent and workflow views with improved state management - Add new UI components (separator, switch) for settings - Update debug panel with better event filtering - Improve message renderers for OpenAI content types - Update types and API client for OpenAI integration * update ui, settings modal and workflow input form, add register cleanup hooks. * add workflow HIL support, user mode, other fixes * feat(devui): add human-in-the-loop (HIL) support with dynamic response schemas Implement HIL workflow support allowing workflows to pause for user input with dynamically generated JSON schemas based on response handler type hints. Key Features: - Automatic response schema extraction from @response_handler decorators - Dynamic form generation in UI based on Pydantic/dataclass response types - Checkpoint-based conversation storage for HIL requests/responses - Resume workflow execution after user provides HIL response Backend Changes: - Add extract_response_type_from_executor() to introspect response handlers - Enrich RequestInfoEvent with response_schema via _enrich_request_info_event_with_response_schema() - Map RequestInfoEvent to response.input.requested OpenAI event format - Store HIL responses in conversation history and restore checkpoints Frontend Changes: - Add HILInputModal component with SchemaFormRenderer for dynamic forms - Support Pydantic BaseModel and dataclass response types - Render enum fields as dropdowns, strings as text/textarea, numbers, booleans, arrays, objects - Display original request context alongside response form Testing: - Add tests for checkpoint storage (test_checkpoints.py) - Add schema generation tests for all input types (test_schema_generation.py) - Validate end-to-end HIL flow with spam workflow sample This enables workflows to seamlessly pause execution and request structured user input with type-safe, validated forms generated automatically from response type annotations. * improve HIL support, improve workflow execution view * ui updates * ui updates * improve HIL for workflows, add auth and view modes * update workflow * security improvements , ui fixes * fix mypy error * update loading spinner in ui --------- Co-authored-by: Mark Wallace <127216156+markwallace-microsoft@users.noreply.github.com>
1 parent 85484c0 commit 94eae24

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+10177
-1598
lines changed

python/packages/devui/README.md

Lines changed: 78 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,27 @@ serve(entities=[agent])
6262

6363
MCP tools use lazy initialization and connect automatically on first use. DevUI attempts to clean up connections on shutdown
6464

65+
## Resource Cleanup
66+
67+
Register cleanup hooks to properly close credentials and resources on shutdown:
68+
69+
```python
70+
from azure.identity.aio import DefaultAzureCredential
71+
from agent_framework import ChatAgent
72+
from agent_framework.azure import AzureOpenAIChatClient
73+
from agent_framework_devui import register_cleanup, serve
74+
75+
credential = DefaultAzureCredential()
76+
client = AzureOpenAIChatClient()
77+
agent = ChatAgent(name="MyAgent", chat_client=client)
78+
79+
# Register cleanup hook - credential will be closed on shutdown
80+
register_cleanup(agent, credential.close)
81+
serve(entities=[agent])
82+
```
83+
84+
Works with multiple resources and file-based discovery. See tests for more examples.
85+
6586
## Directory Structure
6687

6788
For your agents to be discovered by the DevUI, they must be organized in a directory structure like below. Each agent/workflow must have an `__init__.py` that exports the required variable (`agent` or `workflow`).
@@ -150,6 +171,22 @@ response2 = client.responses.create(
150171
151172
**How it works:** DevUI automatically retrieves the conversation's message history from the stored thread and passes it to the agent. You don't need to manually manage message history - just provide the same `conversation` ID for follow-up requests.
152173
174+
### OpenAI Proxy Mode
175+
176+
DevUI provides an **OpenAI Proxy** feature for testing OpenAI models directly through the interface without creating custom agents. Enable via Settings → OpenAI Proxy tab.
177+
178+
**How it works:** The UI sends requests to the DevUI backend (with `X-Proxy-Backend: openai` header), which then proxies them to OpenAI's Responses API (and Conversations API for multi-turn chats). This proxy approach keeps your `OPENAI_API_KEY` secure on the server—never exposed in the browser or client-side code.
179+
180+
**Example:**
181+
182+
```bash
183+
curl -X POST http://localhost:8080/v1/responses \
184+
-H "X-Proxy-Backend: openai" \
185+
-d '{"model": "gpt-4.1-mini", "input": "Hello"}'
186+
```
187+
188+
**Note:** Requires `OPENAI_API_KEY` environment variable configured on the backend.
189+
153190
## CLI Options
154191
155192
```bash
@@ -162,6 +199,21 @@ Options:
162199
--config YAML config file
163200
--tracing none|framework|workflow|all
164201
--reload Enable auto-reload
202+
--mode developer|user (default: developer)
203+
--auth Enable Bearer token authentication
204+
```
205+
206+
### UI Modes
207+
208+
- **developer** (default): Full access - debug panel, entity details, hot reload, deployment
209+
- **user**: Simplified UI with restricted APIs - only chat and conversation management
210+
211+
```bash
212+
# Development
213+
devui ./agents
214+
215+
# Production (user-facing)
216+
devui ./agents --mode user --auth
165217
```
166218
167219
## Key Endpoints
@@ -187,18 +239,23 @@ Given that DevUI offers an OpenAI Responses API, it internally maps messages and
187239
| `response.function_result.complete` | `FunctionResultContent` | DevUI |
188240
| `response.function_approval.requested` | `FunctionApprovalRequestContent` | DevUI |
189241
| `response.function_approval.responded` | `FunctionApprovalResponseContent` | DevUI |
242+
| `response.output_item.added` (ResponseOutputImage) | `DataContent` (images) | DevUI |
243+
| `response.output_item.added` (ResponseOutputFile) | `DataContent` (files) | DevUI |
244+
| `response.output_item.added` (ResponseOutputData) | `DataContent` (other) | DevUI |
245+
| `response.output_item.added` (ResponseOutputImage/File) | `UriContent` (images/files) | DevUI |
190246
| `error` | `ErrorContent` | OpenAI |
191247
| Final `Response.usage` field (not streamed) | `UsageContent` | OpenAI |
192248
| | **Workflow Events** | |
193249
| `response.output_item.added` (ExecutorActionItem)* | `ExecutorInvokedEvent` | OpenAI |
194250
| `response.output_item.done` (ExecutorActionItem)* | `ExecutorCompletedEvent` | OpenAI |
195251
| `response.output_item.done` (ExecutorActionItem with error)* | `ExecutorFailedEvent` | OpenAI |
252+
| `response.output_item.added` (ResponseOutputMessage) | `WorkflowOutputEvent` | OpenAI |
196253
| `response.workflow_event.complete` | `WorkflowEvent` (other) | DevUI |
197254
| `response.trace.complete` | `WorkflowStatusEvent` | DevUI |
198255
| `response.trace.complete` | `WorkflowWarningEvent` | DevUI |
199256
| | **Trace Content** | |
200-
| `response.trace.complete` | `DataContent` | DevUI |
201-
| `response.trace.complete` | `UriContent` | DevUI |
257+
| `response.trace.complete` | `DataContent` (no data/errors) | DevUI |
258+
| `response.trace.complete` | `UriContent` (unsupported MIME) | DevUI |
202259
| `response.trace.complete` | `HostedFileContent` | DevUI |
203260
| `response.trace.complete` | `HostedVectorStoreContent` | DevUI |
204261
@@ -213,15 +270,19 @@ DevUI follows the OpenAI Responses API specification for maximum compatibility:
213270
214271
**OpenAI Standard Event Types Used:**
215272
216-
- `ResponseOutputItemAddedEvent` - Output item notifications (function calls and results)
273+
- `ResponseOutputItemAddedEvent` - Output item notifications (function calls, images, files, data)
217274
- `ResponseOutputItemDoneEvent` - Output item completion notifications
218275
- `Response.usage` - Token usage (in final response, not streamed)
219-
- All standard text, reasoning, and function call events
220276
221277
**Custom DevUI Extensions:**
222278
279+
- `response.output_item.added` with custom item types:
280+
- `ResponseOutputImage` - Agent-generated images (inline display)
281+
- `ResponseOutputFile` - Agent-generated files (inline display)
282+
- `ResponseOutputData` - Agent-generated structured data (inline display)
223283
- `response.function_approval.requested` - Function approval requests (for interactive approval workflows)
224284
- `response.function_approval.responded` - Function approval responses (user approval/rejection)
285+
- `response.function_result.complete` - Server-side function execution results
225286
- `response.workflow_event.complete` - Agent Framework workflow events
226287
- `response.trace.complete` - Execution traces and internal content (DataContent, UriContent, hosted files/stores)
227288
@@ -254,18 +315,28 @@ These custom extensions are clearly namespaced and can be safely ignored by stan
254315
255316
## Security
256317
257-
DevUI is designed as a **sample application for local development** and should not be exposed to untrusted networks or used in production environments.
318+
DevUI is designed as a **sample application for local development** and should not be exposed to untrusted networks without proper authentication.
319+
320+
**For production deployments:**
321+
322+
```bash
323+
# User mode with authentication (recommended)
324+
devui ./agents --mode user --auth --host 0.0.0.0
325+
```
326+
327+
This restricts developer APIs (reload, deployment, entity details) and requires Bearer token authentication.
258328
259329
**Security features:**
260330
331+
- User mode restricts developer-facing APIs
332+
- Optional Bearer token authentication via `--auth`
261333
- Only loads entities from local directories or in-memory registration
262334
- No remote code execution capabilities
263335
- Binds to localhost (127.0.0.1) by default
264-
- All samples must be manually downloaded and reviewed before running
265336
266337
**Best practices:**
267338
268-
- Never expose DevUI to the internet
339+
- Use `--mode user --auth` for any deployment exposed to end users
269340
- Review all agent/workflow code before running
270341
- Only load entities from trusted sources
271342
- Use `.env` files for sensitive credentials (never commit them)

python/packages/devui/agent_framework_devui/__init__.py

Lines changed: 127 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,20 +5,87 @@
55
import importlib.metadata
66
import logging
77
import webbrowser
8+
from collections.abc import Callable
89
from typing import Any
910

11+
from ._conversations import CheckpointConversationManager
1012
from ._server import DevServer
1113
from .models import AgentFrameworkRequest, OpenAIError, OpenAIResponse, ResponseStreamEvent
1214
from .models._discovery_models import DiscoveryResponse, EntityInfo, EnvVarRequirement
1315

1416
logger = logging.getLogger(__name__)
1517

18+
# Module-level cleanup registry (before serve() is called)
19+
_cleanup_registry: dict[int, list[Callable[[], Any]]] = {}
20+
1621
try:
1722
__version__ = importlib.metadata.version(__name__)
1823
except importlib.metadata.PackageNotFoundError:
1924
__version__ = "0.0.0" # Fallback for development mode
2025

2126

27+
def register_cleanup(entity: Any, *hooks: Callable[[], Any]) -> None:
28+
"""Register cleanup hook(s) for an entity.
29+
30+
Cleanup hooks execute during DevUI server shutdown, before entity
31+
clients are closed. Supports both synchronous and asynchronous callables.
32+
33+
Args:
34+
entity: Agent, workflow, or other entity object
35+
*hooks: One or more cleanup callables (sync or async)
36+
37+
Raises:
38+
ValueError: If no hooks provided
39+
40+
Examples:
41+
Single cleanup hook:
42+
>>> from agent_framework.devui import serve, register_cleanup
43+
>>> credential = DefaultAzureCredential()
44+
>>> agent = ChatAgent(...)
45+
>>> register_cleanup(agent, credential.close)
46+
>>> serve(entities=[agent])
47+
48+
Multiple cleanup hooks:
49+
>>> register_cleanup(agent, credential.close, session.close, db_pool.close)
50+
51+
Works with file-based discovery:
52+
>>> # In agents/my_agent/agent.py
53+
>>> from agent_framework.devui import register_cleanup
54+
>>> credential = DefaultAzureCredential()
55+
>>> agent = ChatAgent(...)
56+
>>> register_cleanup(agent, credential.close)
57+
>>> # Run: devui ./agents
58+
"""
59+
if not hooks:
60+
raise ValueError("At least one cleanup hook required")
61+
62+
# Use id() to track entity identity (works across modules)
63+
entity_id = id(entity)
64+
65+
if entity_id not in _cleanup_registry:
66+
_cleanup_registry[entity_id] = []
67+
68+
_cleanup_registry[entity_id].extend(hooks)
69+
70+
logger.debug(
71+
f"Registered {len(hooks)} cleanup hook(s) for {type(entity).__name__} "
72+
f"(id: {entity_id}, total: {len(_cleanup_registry[entity_id])})"
73+
)
74+
75+
76+
def _get_registered_cleanup_hooks(entity: Any) -> list[Callable[[], Any]]:
77+
"""Get cleanup hooks registered for an entity (internal use).
78+
79+
Args:
80+
entity: Entity object to get hooks for
81+
82+
Returns:
83+
List of cleanup hooks registered for the entity
84+
"""
85+
entity_id = id(entity)
86+
return _cleanup_registry.get(entity_id, [])
87+
88+
2289
def serve(
2390
entities: list[Any] | None = None,
2491
entities_dir: str | None = None,
@@ -28,6 +95,9 @@ def serve(
2895
cors_origins: list[str] | None = None,
2996
ui_enabled: bool = True,
3097
tracing_enabled: bool = False,
98+
mode: str = "developer",
99+
auth_enabled: bool = False,
100+
auth_token: str | None = None,
31101
) -> None:
32102
"""Launch Agent Framework DevUI with simple API.
33103
@@ -40,6 +110,9 @@ def serve(
40110
cors_origins: List of allowed CORS origins
41111
ui_enabled: Whether to enable the UI
42112
tracing_enabled: Whether to enable OpenTelemetry tracing
113+
mode: Server mode - 'developer' (full access, verbose errors) or 'user' (restricted APIs, generic errors)
114+
auth_enabled: Whether to enable Bearer token authentication
115+
auth_token: Custom authentication token (auto-generated if not provided with auth_enabled=True)
43116
"""
44117
import re
45118

@@ -53,6 +126,52 @@ def serve(
53126
if not isinstance(port, int) or not (1 <= port <= 65535):
54127
raise ValueError(f"Invalid port: {port}. Must be integer between 1 and 65535")
55128

129+
# Security check: Warn if network-exposed without authentication
130+
if host not in ("127.0.0.1", "localhost") and not auth_enabled:
131+
logger.warning("⚠️ WARNING: Exposing DevUI to network without authentication!")
132+
logger.warning("⚠️ This is INSECURE - anyone on your network can access your agents")
133+
logger.warning("💡 For network exposure, add --auth flag: devui --host 0.0.0.0 --auth")
134+
135+
# Handle authentication configuration
136+
if auth_enabled:
137+
import os
138+
import secrets
139+
140+
# Check if token is in environment variable first
141+
if not auth_token:
142+
auth_token = os.environ.get("DEVUI_AUTH_TOKEN")
143+
144+
# Auto-generate token if STILL not provided
145+
if not auth_token:
146+
# Check if we're in a production-like environment
147+
is_production = (
148+
host not in ("127.0.0.1", "localhost") # Exposed to network
149+
or os.environ.get("CI") == "true" # Running in CI
150+
or os.environ.get("KUBERNETES_SERVICE_HOST") # Running in k8s
151+
)
152+
153+
if is_production:
154+
# REFUSE to start without explicit token
155+
logger.error("❌ Authentication enabled but no token provided")
156+
logger.error("❌ Auto-generated tokens are NOT secure for network-exposed deployments")
157+
logger.error("💡 Set token: export DEVUI_AUTH_TOKEN=<your-secure-token>")
158+
logger.error("💡 Or pass: serve(entities=[...], auth_token='your-token')")
159+
raise ValueError("DEVUI_AUTH_TOKEN required when host is not localhost")
160+
161+
# Development mode: auto-generate and show
162+
auth_token = secrets.token_urlsafe(32)
163+
logger.info("🔒 Authentication enabled with auto-generated token")
164+
logger.info("\n" + "=" * 70)
165+
logger.info("🔑 DEV TOKEN (localhost only, shown once):")
166+
logger.info(f" {auth_token}")
167+
logger.info("=" * 70 + "\n")
168+
else:
169+
logger.info("🔒 Authentication enabled with provided token")
170+
171+
# Set environment variable for server to use
172+
os.environ["AUTH_REQUIRED"] = "true"
173+
os.environ["DEVUI_AUTH_TOKEN"] = auth_token
174+
56175
# Configure tracing environment variables if enabled
57176
if tracing_enabled:
58177
import os
@@ -72,7 +191,12 @@ def serve(
72191

73192
# Create server with direct parameters
74193
server = DevServer(
75-
entities_dir=entities_dir, port=port, host=host, cors_origins=cors_origins, ui_enabled=ui_enabled
194+
entities_dir=entities_dir,
195+
port=port,
196+
host=host,
197+
cors_origins=cors_origins,
198+
ui_enabled=ui_enabled,
199+
mode=mode,
76200
)
77201

78202
# Register in-memory entities if provided
@@ -139,6 +263,7 @@ def main() -> None:
139263
# Export main public API
140264
__all__ = [
141265
"AgentFrameworkRequest",
266+
"CheckpointConversationManager",
142267
"DevServer",
143268
"DiscoveryResponse",
144269
"EntityInfo",
@@ -147,5 +272,6 @@ def main() -> None:
147272
"OpenAIResponse",
148273
"ResponseStreamEvent",
149274
"main",
275+
"register_cleanup",
150276
"serve",
151277
]

0 commit comments

Comments
 (0)