Skip to content

Commit f08ceef

Browse files
apaniukovCopilot
apaniukov
and
Copilot
authored
[Docs] Add Structured Output Docs (#2950)
## Description Add SO page to the docs <!--- Jira ticket number (e.g., 123). Delete if there's no ticket. Don't include full link or project name. --> Ticket: CVS-173810 ## Checklist: - [ ] Tests have been updated or added to cover the new code <!--- If the change isn't maintenance related, update the tests at https://github.com/openvinotoolkit/openvino.genai/tree/master/tests or explain in the description why the tests don't need an update. --> - [ ] This patch fully addresses the ticket. <!--- If follow-up pull requests are needed, specify in description. --> - [x] I have made corresponding changes to the documentation --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent d811cfd commit f08ceef

File tree

3 files changed

+134
-1
lines changed

3 files changed

+134
-1
lines changed

‎site/docs/guides/debug-logging.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
sidebar_position: 7
2+
sidebar_position: 8
33
---
44

55
# Debug Logging

‎site/docs/guides/structured-output.mdx‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
---
2+
sidebar_position: 7
3+
---
4+
5+
# Structured Output
6+
7+
Structured output generation forces a model to produce responses that follow a defined format - for example JSON, a regular expression, or an EBNF grammar. OpenVINO GenAI provides a flexible structured output enforcement so the model can generate well‑formed output.
8+
9+
## How It Works
10+
11+
![Example from XGrammar paper](/img/structured_output_work_example.png)
12+
13+
Structured output generation consists of several steps:
14+
1. Compile the structured-output configuration (JSON Schema, regex, or EBNF grammar) into an internal grammar representation.
15+
2. After the model computes the probability distribution for the next token, a mask is applied that prevents tokens incompatible with the current grammar state from being sampled.
16+
3. When a token is sampled, advance the grammar state and recompute the mask for the next sampling round.
17+
18+
Refer to [XGrammar paper](https://arxiv.org/pdf/2411.15100) for more details.
19+
20+
:::info
21+
Structured enforcement guarantees the syntactic format but not the semantic correctness of generated data. Always validate and sanitize outputs.
22+
:::
23+
24+
:::info
25+
Structured constraints can affect model behavior and may reduce accuracy on some downstream tasks.
26+
:::
27+
28+
## What OpenVINO GenAI Provides
29+
30+
- Base structured output formats: JSON, regex, and grammar (EBNF) based structured generation.
31+
- Structural tags:
32+
- Compound grammars: combine several grammars to handle more complex tasks or enforce different function calling styles.
33+
- Tags: blend regular free-form generation with structural tags during single generate call.
34+
35+
## Structured Output Configuration
36+
37+
Structured output is configured through a `StructuredOutputConfig` (exposed in the Python and C++ APIs). The main options are:
38+
39+
- `json_schema` - a JSON Schema that the generator will try to satisfy; useful when you need typed fields and nested structures.
40+
- `regex` - a regular expression that the output must match.
41+
- `grammar` - an EBNF grammar that describes structure of the output.
42+
- `structural_tags_config` and `compound_grammar` - advanced options to combine multiple grammars.
43+
44+
:::info
45+
You should set only one primary structured constraint (for example `json_schema`, `regex`, or `grammar`) on a `StructuredOutputConfig` instance. The library validates the configuration and will raise an error if conflicting constraints are provided (for example both `json_schema` and `regex`).
46+
:::
47+
48+
:::warning Deprecation Note
49+
The `StructuralTagsConfig` class and the `compound_grammar` field are deprecated. Use the `structural_tags_config` field on `StructuredOutputConfig` instead; it provides the same functionality and is the recommended API going forward.
50+
:::
51+
52+
## Using Structured Output
53+
54+
### Examples
55+
56+
<LanguageTabs>
57+
<TabItemPython>
58+
```python
59+
import json
60+
import openvino_genai as ov_genai
61+
62+
63+
pipe = ov_genai.LLMPipeline(model_path, "CPU")
64+
65+
# Structured output configuration: simple JSON schema
66+
so_config = ov_genai.StructuredOutputConfig(
67+
json_schema=json.dumps({
68+
"type": "object",
69+
"properties": {
70+
"name": {"type": "string"},
71+
"age": {"type": "integer"}
72+
},
73+
"required": ["name"]
74+
}),
75+
)
76+
gen_config = ov_genai.GenerationConfig(max_new_tokens=100, structured_output_config=so_config)
77+
78+
prompt = "Extract a person's name and age from the text and return only a JSON object.\nText: 'Alice is 29 years old.'"
79+
result = pipe.generate(prompt, generation_config=gen_config)
80+
print(json.loads(result))
81+
```
82+
</TabItemPython>
83+
<TabItemCpp>
84+
```cpp
85+
#include <openvino/genai/llm_pipeline.hpp>
86+
87+
using namespace ov::genai;
88+
89+
int main() {
90+
std::string models_path = "/path/to/model/dir";
91+
auto pipe = LLMPipeline(models_path, "CPU");
92+
93+
// Build a JSON schema object (represented using the SDK types) and configure structured output
94+
StructuredOutputConfig so_config;
95+
so_config.json_schema = R"({
96+
"type": "object",
97+
"properties": {
98+
"name": {"type": "string"},
99+
"age": {"type": "integer"}
100+
},
101+
"required": ["name"]
102+
})";
103+
104+
std::string prompt = "Extract a person's name and age and return only a JSON object.\nText: 'Bob is 34 years old.'";
105+
106+
// Attach structured output config to a GenerationConfig and call generate
107+
GenerationConfig gen_config;
108+
gen_config.max_new_tokens = 100;
109+
gen_config.structured_output_config = so_config;
110+
111+
auto result = pipe.generate(prompt, gen_config);
112+
113+
std::cout << "Model output: " << result.texts[0] << std::endl;
114+
return 0;
115+
}
116+
```
117+
</TabItemCpp>
118+
</LanguageTabs>
119+
120+
### Samples and Further Reading
121+
122+
You can find complete sample programs that demonstrate structured output generation here:
123+
124+
- [Python sample](https://github.com/openvinotoolkit/openvino.genai/blob/master/samples/python/text_generation/structured_output_generation.py)
125+
- [C++ sample](https://github.com/openvinotoolkit/openvino.genai/blob/master/samples/cpp/text_generation/structured_output_generation.cpp)
126+
- [Combining multiple grammars for structured output](https://github.com/openvinotoolkit/openvino.genai/blob/master/samples/python/text_generation/compound_grammar_generation.py)
127+
- [Trigger structured output during regular generation run](https://github.com/openvinotoolkit/openvino.genai/blob/master/samples/python/text_generation/structural_tags_generation.py)
128+
- [Test-driven examples](https://github.com/openvinotoolkit/openvino.genai/blob/master/tests/python_tests/test_structured_output.py) (comprehensive examples covering JSON, regex, EBNF and structural tags)
129+
130+
OpenVINO GenAI JavaScript bindings also support the Structured Output feature. Check [JS samples](/docs/samples/js/text_generation#7-structured-output-sample-structured_output_sample) for usage examples.

‎site/static/img/structured_output_work_example.png‎

Lines changed: 3 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)