Skip to content

Commit d02b751

Browse files
PringledPringled
authored
docs: Add txtai integration docs (#130)
* Added txtai integration docs * Added txtai integration docs * Added txtai integration docs * Added txtai integration docs * Added txtai integration docs * Updates
1 parent 50c665e commit d02b751

File tree

1 file changed

+83
-67
lines changed

1 file changed

+83
-67
lines changed

‎README.md

Lines changed: 83 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -51,35 +51,35 @@ Model2Vec is a technique to turn any sentence transformer into a really small st
5151
- [Main Features](#main-features)
5252
- [What is Model2Vec?](#what-is-model2vec)
5353
- [Usage](#usage)
54-
- [Distillation](#distillation)
5554
- [Inference](#inference)
55+
- [Distillation](#distillation)
5656
- [Evaluation](#evaluation)
57-
- [Integrations](#integrations)
57+
- [Integrations](#integrations)
5858
- [Model List](#model-list)
5959
- [Results](#results)
6060
- [Related Work](#related-work)
6161
- [Citing](#citing)
6262

6363
## Quickstart
6464

65-
Install the package and all required extras with:
65+
Install the package with:
66+
6667
```bash
67-
pip install model2vec[distill]
68+
pip install model2vec
6869
```
6970

70-
If you want a light-weight version of the package which only requires `numpy`, omit the `distill` extra. This means you can't distill your own models, but you can use pre-trained models. This is useful for inference pipelines.
71+
This will install the base inference package, which only depends on `numpy` and a few other minor dependencies. If you want to distill your own models, you can install the distillation extras with:
7172

7273
```bash
73-
pip install model2vec
74+
pip install model2vec[distill]
7475
```
7576

76-
The easiest way to get started with Model2Vec is to download one of our [flagship models from the HuggingFace hub](https://huggingface.co/minishlab). These models are pre-trained and ready to use. The following code snippet shows how to load a model and make embeddings:
77+
The easiest way to get started with Model2Vec is to load one of our [flagship models from the HuggingFace hub](https://huggingface.co/minishlab). These models are pre-trained and ready to use. The following code snippet shows how to load a model and make embeddings:
7778
```python
7879
from model2vec import StaticModel
7980

8081
# Load a model from the HuggingFace hub (in this case the potion-base-8M model)
81-
model_name = "minishlab/potion-base-8M"
82-
model = StaticModel.from_pretrained(model_name)
82+
model = StaticModel.from_pretrained("minishlab/potion-base-8M")
8383

8484
# Make embeddings
8585
embeddings = model.encode(["It's dangerous to go alone!", "It's a secret to everybody."])
@@ -94,11 +94,8 @@ Instead of using one of our models, you can distill your own Model2Vec model fro
9494
```python
9595
from model2vec.distill import distill
9696

97-
# Choose a Sentence Transformer model
98-
model_name = "BAAI/bge-base-en-v1.5"
99-
100-
# Distill the model
101-
m2v_model = distill(model_name=model_name, pca_dims=256)
97+
# Distill a Sentence Transformer model, in this case the BAAI/bge-base-en-v1.5 model
98+
m2v_model = distill(model_name="BAAI/bge-base-en-v1.5", pca_dims=256)
10299

103100
# Save the model
104101
m2v_model.save_pretrained("m2v_model")
@@ -134,11 +131,12 @@ Model2Vec has the following features:
134131

135132
- **Small**: reduces the size of a Sentence Transformer model by a factor of 15, from 120M params, down to 7.5M (30 MB on disk, making it the smallest model on [MTEB](https://huggingface.co/spaces/mteb/leaderboard)!).
136133
- **Static, but better**: smaller than GLoVe and BPEmb, but [much more performant](results/README.md), even with the same vocabulary.
134+
- **Lightweight inference**: the base package's only major dependency is `numpy`.
137135
- **Fast distillation**: make your own model in 30 seconds.
138136
- **Fast inference**: up to 500 times faster on CPU than the original model. Go green or go home.
139137
- **No data needed**: Distillation happens directly on the token level, so no dataset is needed.
140138
- **Simple to use**: An easy to use interface for distilling and inferencing.
141-
- **Integrated into Sentence Transformers**: Model2Vec can be used directly in [Sentence Transformers](https://github.com/UKPLab/sentence-transformers).
139+
- **Integrated into Sentence Transformers and txtai**: Model2Vec can be used directly in [Sentence Transformers](https://github.com/UKPLab/sentence-transformers) and [txtai](https://github.com/neuml/txtai).
142140
- **Bring your own model**: Can be applied to any Sentence Transformer model.
143141
- **Bring your own vocabulary**: Can be applied to any vocabulary, allowing you to use your own domain-specific vocabulary. Need biomedical? Just get a medical dictionary, a biomedical model, and inference it.
144142
- **Multi-lingual**: Use any language. Need a French model? [Pick one](https://huggingface.co/models?library=sentence-transformers&language=fr&sort=trending). Need multilingual? [Here you go](https://huggingface.co/sentence-transformers/LaBSE).
@@ -166,6 +164,46 @@ Our flagship POTION models are pre-trained using [Tokenlearn](https://github.com
166164
## Usage
167165

168166

167+
### Inference
168+
169+
<details>
170+
<summary> Inference using pretrained model </summary>
171+
<br>
172+
173+
Inference works as follows. The example shows one of our own models, but you can also just load a local one, or another one from the hub.
174+
```python
175+
from model2vec import StaticModel
176+
177+
# Load a model from the Hub. You can optionally pass a token when loading a private model
178+
model = StaticModel.from_pretrained(model_name="minishlab/potion-base-8M", token=None)
179+
180+
# Make embeddings
181+
embeddings = model.encode(["It's dangerous to go alone!", "It's a secret to everybody."])
182+
183+
# Make sequences of token embeddings
184+
token_embeddings = model.encode_as_sequence(["It's dangerous to go alone!", "It's a secret to everybody."])
185+
```
186+
</details>
187+
188+
189+
<details>
190+
<summary> Inference using the Sentence Transformers library </summary>
191+
<br>
192+
193+
The following code snippet shows how to use a Model2Vec model in the [Sentence Transformers](https://github.com/UKPLab/sentence-transformers) library. This is useful if you want to use the model in a Sentence Transformers pipeline.
194+
195+
```python
196+
from sentence_transformers import SentenceTransformer
197+
from sentence_transformers.models import StaticEmbedding
198+
199+
# Initialize a StaticEmbedding module
200+
static_embedding = StaticEmbedding.from_model2vec("minishlab/potion-base-8M")
201+
model = SentenceTransformer(modules=[static_embedding])
202+
embeddings = model.encode(["It's dangerous to go alone!", "It's a secret to everybody."])
203+
```
204+
205+
</details>
206+
169207
### Distillation
170208

171209
<details>
@@ -176,11 +214,8 @@ The following code can be used to distill a model from a Sentence Transformer. A
176214
```python
177215
from model2vec.distill import distill
178216

179-
# Choose a Sentence Transformer model
180-
model_name = "BAAI/bge-base-en-v1.5"
181-
182-
# Distill the model
183-
m2v_model = distill(model_name=model_name, pca_dims=256)
217+
# Distill a Sentence Transformer model
218+
m2v_model = distill(model_name="BAAI/bge-base-en-v1.5", pca_dims=256)
184219

185220
# Save the model
186221
m2v_model.save_pretrained("m2v_model")
@@ -240,11 +275,9 @@ from model2vec.distill import distill
240275

241276
# Load a vocabulary as a list of strings
242277
vocabulary = ["word1", "word2", "word3"]
243-
# Choose a Sentence Transformer model
244-
model_name = "BAAI/bge-base-en-v1.5"
245278

246-
# Distill the model with the custom vocabulary
247-
m2v_model = distill(model_name=model_name, vocabulary=vocabulary)
279+
# Distill a Sentence Transformer model with the custom vocabulary
280+
m2v_model = distill(model_name="BAAI/bge-base-en-v1.5", vocabulary=vocabulary)
248281

249282
# Save the model
250283
m2v_model.save_pretrained("m2v_model")
@@ -263,48 +296,6 @@ m2v_model = distill(model_name=model_name, vocabulary=vocabulary, use_subword=Fa
263296

264297
</details>
265298

266-
### Inference
267-
268-
<details>
269-
<summary> Inference using pretrained model </summary>
270-
<br>
271-
272-
Inference works as follows. The example shows one of our own models, but you can also just load a local one, or another one from the hub.
273-
```python
274-
from model2vec import StaticModel
275-
276-
# Load a model from the HuggingFace hub, or a local one.
277-
model_name = "minishlab/potion-base-8M"
278-
# You can optionally pass a token if you're loading a private model
279-
model = StaticModel.from_pretrained(model_name, token=None)
280-
281-
# Make embeddings
282-
embeddings = model.encode(["It's dangerous to go alone!", "It's a secret to everybody."])
283-
284-
# Make sequences of token embeddings
285-
token_embeddings = model.encode_as_sequence(["It's dangerous to go alone!", "It's a secret to everybody."])
286-
```
287-
</details>
288-
289-
290-
<details>
291-
<summary> Inference using the Sentence Transformers library </summary>
292-
<br>
293-
294-
The following code snippet shows how to use a Model2Vec model in the [Sentence Transformers](https://github.com/UKPLab/sentence-transformers) library. This is useful if you want to use the model in a Sentence Transformers pipeline.
295-
296-
```python
297-
from sentence_transformers import SentenceTransformer
298-
from sentence_transformers.models import StaticEmbedding
299-
300-
# Initialize a StaticEmbedding module
301-
static_embedding = StaticEmbedding.from_model2vec("minishlab/potion-base-8M")
302-
model = SentenceTransformer(modules=[static_embedding])
303-
embeddings = model.encode(["It's dangerous to go alone!", "It's a secret to everybody."])
304-
```
305-
306-
</details>
307-
308299

309300
### Evaluation
310301

@@ -357,7 +348,7 @@ print(make_leaderboard(task_scores))
357348
```
358349
</details>
359350

360-
### Integrations
351+
## Integrations
361352
<details>
362353
<summary> Sentence Transformers </summary>
363354
<br>
@@ -389,6 +380,31 @@ embeddings = model.encode(["It's dangerous to go alone!", "It's a secret to ever
389380
</details>
390381

391382

383+
<details>
384+
<summary> Txtai </summary>
385+
<br>
386+
387+
Model2Vec can be used in [txtai](https://github.com/neuml/txtai) for text embeddings, nearest-neighbors search, and any of the other functionalities that txtai offers. The following code snippet shows how to use Model2Vec in txtai:
388+
389+
```python
390+
from txtai import Embeddings
391+
392+
# Load a model2vec model
393+
embeddings = Embeddings(path="minishlab/potion-base-8M", method="model2vec", backend="numpy")
394+
395+
# Create some example texts
396+
texts = ["Enduring Stew", "Hearty Elixir", "Mighty Mushroom Risotto", "Spicy Meat Skewer", "Chilly Fruit Salad"]
397+
398+
# Create embeddings for downstream tasks
399+
vectors = embeddings.batchtransform(texts)
400+
401+
# Or create a nearest-neighbors index and search it
402+
embeddings.index(texts)
403+
result = embeddings.search("Risotto", 1)
404+
```
405+
406+
</details>
407+
392408
<details>
393409
<summary> Transformers.js </summary>
394410

0 commit comments

Comments
 (0)