Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 49 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ OpenVINO™ GenAI library provides very lightweight C++ and Python APIs to run t
- [Speech recognition using Whisper family models](#speech-to-text)
- [Text-to-speech generation using SpeechT5 TTS models](#text-to-speech)
- [Text embedding for Retrieval-Augmented Generation (RAG)](#text-embedding), for example, compute embeddings for documents and queries to enable efficient retrieval in RAG workflows.
- [Text reranking for Retrieval-Augmented Generation (RAG)](#text-rerank), for example, reorder candidate documents by semantic relevance to a query to improve retrieval quality in RAG workflows.

Library efficiently supports LoRA adapters for Text and Image generation scenarios:
- Load multiple adapters per model
Expand Down Expand Up @@ -513,7 +514,7 @@ int main(int argc, char* argv[]) {

```sh
# Download and convert the BAAI/bge-small-en-v1.5 model to OpenVINO format
optimum-cli export openvino --trust-remote-code --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5
optimum-cli export openvino --task feature-extraction --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5
```

### Compute embeddings using TextEmbeddingPipeline API in Python
Expand Down Expand Up @@ -567,6 +568,53 @@ int main(int argc, char* argv[]) {
```
</details>

<a id="text-rerank"></a>

## Text Reranking
<details>

### Converting and preparing a text reranking model from Hugging Face library

```sh
optimum-cli export openvino --task text-classification --model cross-encoder/ms-marco-MiniLM-L6-v2 cross-encoder/ms-marco-MiniLM-L6-v2
```

### Rerank documents using TextRerankPipeline API in Python

```python
import openvino_genai

pipeline = openvino_genai.TextRerankPipeline("./cross-encoder/ms-marco-MiniLM-L6-v2", "CPU", top_n=3)

rerank_result = pipeline.rerank(query, documents)

print("Reranked documents:")
for index, score in rerank_result:
print(f"Document {index} (score: {score:.4f}): {documents[index]}")
```

### Rerank documents using TextRerankPipeline API in C++

```cpp
#include "openvino/genai/rag/text_rerank_pipeline.hpp"
#include <iostream>

int main(int argc, char* argv[]) {
std::vector<std::string> documents(argv + 3, argv + argc);
std::string models_path = argv[1], query = argv[2];

ov::genai::TextRerankPipeline pipeline(models_path, "CPU", ov::genai::top_n(3));

auto rerank_result = pipeline.rerank(query, documents);

std::cout << "Reranked documents:\n";
for (const auto& [index, score] : rerank_result) {
std::cout << "Document " << index << " (score: " << score << "): " << documents[index] << '\n';
}
}
```
</details>

## Additional materials

- [List of supported models](https://openvinotoolkit.github.io/openvino.genai/docs/supported-models/)
Expand Down
4 changes: 2 additions & 2 deletions samples/cpp/rag/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ pip install --upgrade-strategy eager -r ../../export-requirements.txt
To export text embedding model run Optimum CLI command:

```sh
optimum-cli export openvino --trust-remote-code --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5
optimum-cli export openvino --task feature-extraction --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5
```

To export text reranking model run Optimum CLI command:

```sh
optimum-cli export openvino --trust-remote-code --model cross-encoder/ms-marco-MiniLM-L6-v2 cross-encoder/ms-marco-MiniLM-L6-v2
optimum-cli export openvino --task text-classification --model cross-encoder/ms-marco-MiniLM-L6-v2 cross-encoder/ms-marco-MiniLM-L6-v2
```


Expand Down
2 changes: 1 addition & 1 deletion samples/js/rag/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Install [../../export-requirements.txt](../../export-requirements.txt) to conver

```sh
pip install --upgrade-strategy eager -r ../../export-requirements.txt
optimum-cli export openvino --trust-remote-code --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5
optimum-cli export openvino --task feature-extraction --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5
```

## Sample Descriptions
Expand Down
6 changes: 3 additions & 3 deletions samples/python/rag/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ pip install --upgrade-strategy eager -r ../../export-requirements.txt
To export text embedding model run Optimum CLI command:

```sh
optimum-cli export openvino --trust-remote-code --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5
optimum-cli export openvino --task feature-extraction --model BAAI/bge-small-en-v1.5 BAAI/bge-small-en-v1.5
```

To export text reranking model run Optimum CLI command:

```sh
optimum-cli export openvino --trust-remote-code --model cross-encoder/ms-marco-MiniLM-L6-v2 cross-encoder/ms-marco-MiniLM-L6-v2
optimum-cli export openvino --task text-classification --model cross-encoder/ms-marco-MiniLM-L6-v2 cross-encoder/ms-marco-MiniLM-L6-v2
```

Alternatively, do it in Python code:
Expand All @@ -33,7 +33,7 @@ from transformers import AutoTokenizer

output_dir = "embedding_model"

model = OVModelForFeatureExtraction.from_pretrained("BAAI/bge-small-en-v1.5", export=True, trust_remote_code=True)
model = OVModelForFeatureExtraction.from_pretrained("BAAI/bge-small-en-v1.5", export=True)
model.save_pretrained(output_dir)

tokenizer = AutoTokenizer.from_pretrained("BAAI/bge-small-en-v1.5")
Expand Down
2 changes: 1 addition & 1 deletion site/docs/use-cases/text-embedding/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import UsageOptionsSection from './_sections/_usage_options/index.mdx';
<ConvertModelSection>
Download and convert a text embedding model (e.g. [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5)) to OpenVINO format from Hugging Face:

<OptimumCLI model='BAAI/bge-small-en-v1.5' outputDir='bge-small-en-v1_5_ov' trustRemoteCode />
<OptimumCLI model='BAAI/bge-small-en-v1.5' outputDir='bge-small-en-v1_5_ov' task='feature-extraction' />

See all supported [Text Embedding Models](/docs/supported-models/#text-embeddings-models).
</ConvertModelSection>
Expand Down
2 changes: 1 addition & 1 deletion site/docs/use-cases/text-rerank/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import RunModelSection from './_sections/_run_model/index.mdx';
<ConvertModelSection>
Download and convert a reranker model (e.g. [cross-encoder/ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2)) to OpenVINO format from Hugging Face:

<OptimumCLI model='cross-encoder/ms-marco-MiniLM-L6-v2' outputDir='cross-encoder/ms-marco-MiniLM-L6-v2' trustRemoteCode />
<OptimumCLI model='cross-encoder/ms-marco-MiniLM-L6-v2' outputDir='cross-encoder/ms-marco-MiniLM-L6-v2' task='text-classification' />

See all supported [Reranker Models](/docs/supported-models/#text-rerank-models).
</ConvertModelSection>
Expand Down
Loading