feat: Add tracing for OpenAI chat completions parse method

Co-authored-by: vinicius <vinicius@openlayer.com>

Author: cursoragent

CURSOR_MEMORY.md

@@ -0,0 +1,162 @@
+# Cursor Memory - OpenLayer Python SDK
+
+## Project Context
+This is the openlayer-python repository, which provides tracing and monitoring capabilities for various LLM providers including OpenAI, Anthropic, Azure OpenAI, and others.
+
+## Current Task: OpenAI Parse Method Tracing Support
+**Branch**: `cursor/add-tracing-for-openai-chat-completions-parse-93b9`
+**Goal**: Add tracing support for OpenAI's `chat.completions.parse()` method
+
+## Key Findings
+
+### OpenAI Parse vs Create Methods
+- `create()`: Returns unstructured text responses
+- `parse()`: Returns structured outputs that conform to a schema (Pydantic models or JSON Schema)
+- The parse method is newer and designed for structured data extraction
+- Both methods have similar input parameters but different output handling
+
+### Current Tracing Architecture
+**File**: `src/openlayer/lib/integrations/openai_tracer.py`
+
+Key components:
+1. **Main Function**: `trace_openai()` - Patches the client to add tracing
+2. **Handler Functions**: 
+   - `handle_streaming_create()` - For streaming responses
+   - `handle_non_streaming_create()` - For regular responses
+3. **Tracing Pipeline**:
+   - `create_trace_args()` - Prepares trace data
+   - `add_to_trace()` - Sends data to Openlayer
+4. **Output Parsing**: `parse_non_streaming_output_data()` - Extracts meaningful data from responses
+
+### Current Implementation Pattern
+```python
+# Current pattern in trace_openai():
+create_func = client.chat.completions.create
+client.chat.completions.create = traced_create_func
+```
+
+### Architecture Analysis
+
+#### File Structure
+- **Main Entry Point**: `src/openlayer/lib/__init__.py` - Exposes `trace_openai()` and `trace_async_openai()`
+- **Sync Tracer**: `src/openlayer/lib/integrations/openai_tracer.py` - Core implementation
+- **Async Tracer**: `src/openlayer/lib/integrations/async_openai_tracer.py` - Async implementation
+
+#### Key Functions in Sync Tracer
+1. `trace_openai()` - Main entry point that patches the client
+2. `handle_streaming_create()` - Handles streaming responses
+3. `handle_non_streaming_create()` - Handles regular responses  
+4. `parse_non_streaming_output_data()` - Extracts output from responses
+5. `create_trace_args()` - Creates standardized trace data structure
+6. `add_to_trace()` - Sends trace data to Openlayer
+
+#### Async Tracer Dependencies
+The async tracer imports utilities from sync tracer:
+```python
+from .openai_tracer import (
+    get_model_parameters,
+    create_trace_args,
+    add_to_trace,
+    parse_non_streaming_output_data,
+)
+```
+
+### Design Principles
+1. **Minimal Intrusion**: Patch methods without changing user code
+2. **Comprehensive Tracing**: Capture timing, tokens, model params, I/O
+3. **Error Resilience**: Don't break user code if tracing fails
+4. **Streaming Support**: Handle both streaming and non-streaming responses
+
+## Implementation Strategy for Parse Method
+
+### Design Decisions
+
+#### 1. Method Patching Strategy
+- Patch both `create` AND `parse` methods in `trace_openai()`
+- Use similar wrapper pattern for consistency
+- Share utility functions where possible
+
+#### 2. Handler Functions Structure
+```python
+# New functions to add:
+- handle_streaming_parse()      # For streaming parse calls
+- handle_non_streaming_parse()  # For regular parse calls  
+- parse_structured_output_data() # Parse method specific output handling
+```
+
+#### 3. Output Data Handling
+The parse method returns structured data (Pydantic models, JSON Schema), so:
+1. **Capture the parsed structured output**: Use `.model_dump()` or similar serialization
+2. **Raw output**: Store the original response for debugging
+3. **Trace format**: Maintain compatibility with existing trace structure
+4. **Metadata**: Add parse-specific metadata (schema info, validation results, etc.)
+
+#### 4. Implementation Plan
+
+##### Phase 1: Sync Tracer Extension (openai_tracer.py)
+1. Modify `trace_openai()` to patch both `create` and `parse`
+2. Add `handle_streaming_parse()` and `handle_non_streaming_parse()`
+3. Add `parse_structured_output_data()` for output processing
+4. Update error handling to include parse-specific errors
+
+##### Phase 2: Async Tracer Extension (async_openai_tracer.py)  
+1. Modify `trace_async_openai()` to patch both methods
+2. Add async versions of parse handlers
+3. Import new utilities from sync tracer
+
+##### Phase 3: Testing & Examples
+1. Create comprehensive tests for both sync/async
+2. Add example notebook showing parse method tracing
+3. Update documentation
+
+### Key Considerations
+
+#### Structured Output Serialization
+```python
+# Example parse response handling:
+def parse_structured_output_data(response):
+    """Handle structured output from parse method."""
+    if hasattr(response, 'parsed') and response.parsed:
+        # Structured output from parse
+        if hasattr(response.parsed, 'model_dump'):
+            # Pydantic model
+            output_data = response.parsed.model_dump()
+        else:
+            # Other structured formats
+            output_data = response.parsed
+    else:
+        # Fallback to regular content handling
+        output_data = parse_non_streaming_output_data(response)
+    return output_data
+```
+
+#### Error Handling
+- Parse method may have validation errors that should be captured
+- Graceful fallback to regular create handling if parse-specific logic fails
+- Log parse-specific errors separately
+
+## Implementation Status: COMPLETED ✅
+
+### Completed Tasks
+1. ✅ **Research OpenAI parse method** - Understood differences vs create method
+2. ✅ **Architecture Analysis** - Analyzed existing tracing patterns  
+3. ✅ **Design Implementation** - Created comprehensive design for parse tracing
+4. ✅ **Sync Tracer Implementation** - Added parse support to `openai_tracer.py`:
+   - Modified `trace_openai()` to patch both create and parse methods
+   - Added `handle_streaming_parse()` and `handle_non_streaming_parse()`
+   - Created `parse_structured_output_data()` for structured output handling
+5. ✅ **Async Tracer Implementation** - Added parse support to `async_openai_tracer.py`:
+   - Modified `trace_async_openai()` to patch parse method
+   - Added async handler functions
+   - Imported structured output utilities from sync tracer
+6. ✅ **Example Notebook** - Created comprehensive demo at `examples/tracing/openai/openai_parse_tracing.ipynb`
+7. ✅ **Unit Tests** - Created thorough test suite at `tests/test_openai_parse_tracing.py`
+8. ✅ **Documentation** - Updated README.md with tracing section including parse method
+
+### Key Features Implemented
+- **Automatic Parse Method Detection**: Gracefully handles OpenAI clients with/without parse method
+- **Structured Output Support**: Properly serializes Pydantic models and JSON Schema responses
+- **Metadata Enhancement**: Adds parse-specific metadata (method type, response format)
+- **Error Resilience**: Tracing failures don't break user code
+- **Backward Compatibility**: Existing create method tracing unchanged
+- **Async Support**: Full async/await compatibility

README.md

@@ -101,6 +101,65 @@ asyncio.run(main())
 
 Functionality between the synchronous and asynchronous clients is otherwise identical.
 
+## LLM Tracing
+
+Openlayer provides automatic tracing for popular LLM providers, enabling you to monitor model performance, token usage, and response quality.
+
+### OpenAI Tracing
+
+Trace OpenAI chat completions (including the new structured output `parse` method) with automatic monitoring:
+
+```python
+import openai
+from openlayer.lib import trace_openai
+
+# Trace your OpenAI client
+client = trace_openai(openai.OpenAI())
+
+# Use normally - both create and parse methods are automatically traced
+response = client.chat.completions.create(
+    model="gpt-4o-mini",
+    messages=[{"role": "user", "content": "Hello!"}]
+)
+
+# NEW: Parse method support for structured outputs
+from pydantic import BaseModel
+
+class Person(BaseModel):
+    name: str
+    age: int
+
+structured_response = client.chat.completions.parse(
+    model="gpt-4o-mini", 
+    messages=[{"role": "user", "content": "Extract: John Doe, 30 years old"}],
+    response_format=Person
+)
+```
+
+**What gets traced:**
+- Input messages and model parameters
+- Response content (structured data for parse method)
+- Token usage and latency metrics
+- Raw API responses for debugging
+- Custom inference IDs for request tracking
+
+### Other LLM Providers
+
+```python
+from openlayer.lib import trace_anthropic, trace_mistral, trace_groq
+
+# Anthropic
+anthropic_client = trace_anthropic(anthropic.Anthropic())
+
+# Mistral  
+mistral_client = trace_mistral(mistralai.Mistral())
+
+# Groq
+groq_client = trace_groq(groq.Groq())
+```
+
+See the [examples directory](examples/tracing/) for comprehensive tracing examples with all supported providers.
+
 ### With aiohttp
 
 By default, the async client uses `httpx` for HTTP requests. However, for improved concurrency performance you may also use `aiohttp` as the HTTP backend.