Skip to content

Commit 0517cd9

Browse files
authored
docs: Edit pass and update example model (#20198)
1 parent f7121a2 commit 0517cd9

File tree

1 file changed

+48
-39
lines changed
  • llama-index-integrations/embeddings/llama-index-embeddings-nvidia

1 file changed

+48
-39
lines changed
Lines changed: 48 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,74 +1,83 @@
1-
# NVIDIA NIMs
1+
# LlamaIndex Embeddings Integration: NVIDIA NIM Microservices
22

3-
The `llama-index-embeddings-nvidia` package contains LlamaIndex integrations building applications with models on
4-
NVIDIA NIM inference microservice. NIM supports models across domains like chat, embedding, and re-ranking models
5-
from the community as well as NVIDIA. These models are optimized by NVIDIA to deliver the best performance on NVIDIA
6-
accelerated infrastructure and deployed as a NIM, an easy-to-use, prebuilt containers that deploy anywhere using a single
7-
command on NVIDIA accelerated infrastructure.
3+
The `llama-index-embeddings-nvidia` package contains LlamaIndex integrations for building applications with [NVIDIA NIM microservices](https://developer.nvidia.com/nim).
4+
With the NVIDIA embeddings connector, you can connect to, and generate content from, compatible models.
85

9-
NVIDIA hosted deployments of NIMs are available to test on the [NVIDIA API catalog](https://build.nvidia.com/). After testing,
10-
NIMs can be exported from NVIDIA’s API catalog using the NVIDIA AI Enterprise license and run on-premises or in the cloud,
11-
giving enterprises ownership and full control of their IP and AI application.
6+
NVIDIA NIM supports models across domains like chat, embedding, and re-ranking, from the community as well as from NVIDIA.
7+
Each model is optimized by NVIDIA to deliver the best performance on NVIDIA-accelerated infrastructure and is packaged as a NIM,
8+
an easy-to-use, prebuilt container that deploys anywhere using a single command on NVIDIA accelerated infrastructure.
9+
At their core, NIM microservices are containers that provide interactive APIs for running inference on an AI Model.
1210

13-
NIMs are packaged as container images on a per model basis and are distributed as NGC container images through the NVIDIA NGC Catalog.
14-
At their core, NIMs provide easy, consistent, and familiar APIs for running inference on an AI model.
11+
NVIDIA-hosted deployments are available on the [NVIDIA API catalog](https://build.nvidia.com/) to test each NIM.
12+
After you explore, you can download NIM microservices from the API catalog, which is included with the NVIDIA AI Enterprise license.
13+
The ability to run models on-premises or in your own cloud gives your enterprise ownership of your customizations and full control of your IP and AI application.
1514

16-
# NVIDIA's Embeddings connector
15+
Use this documentation to learn how to install the `llama-index-embeddings-nvidia` package and use it to connect to a model.
16+
The following example connects to the NVIDIA Retrieval QA E5 Embedding Model.
1717

18-
With this connector, you'll be able to connect to and generate from compatible models available and hosted on [NVIDIA API Catalog](https://build.nvidia.com/), such as:
18+
<!-- Don't link to the model yet because until the reader signs in at a following step, the link might 404 -->
1919

20-
- NVIDIA's Retrieval QA Embedding Model [embed-qa-4](https://build.nvidia.com/nvidia/embed-qa-4)
20+
## Install the Package
2121

22-
## Installation
22+
To install the `llama-index-embeddings-nvidia` package, run the following code.
2323

2424
```bash
2525
pip install llama-index-embeddings-nvidia
2626
```
2727

28-
## Setup
28+
## Access the NVIDIA API Catalog
2929

30-
**To get started:**
30+
To get access to the NVIDIA API Catalog, do the following:
3131

32-
1. Create a free account with [NVIDIA](https://build.nvidia.com/), which hosts NVIDIA AI Foundation models.
32+
1. Create a free account on the [NVIDIA API Catalog](https://build.nvidia.com/) and log in.
33+
2. Click your profile icon, and then click **API Keys**. The **API Keys** page appears.
34+
3. Click **Generate API Key**. The **Generate API Key** window appears.
35+
4. Click **Generate Key**. You should see **API Key Granted**, and your key appears.
36+
5. Copy and save the key as `NVIDIA_API_KEY`.
37+
6. To verify your key, use the following code.
3338

34-
2. Select the `Retrieval` tab, then select your model of choice.
39+
```python
40+
import getpass
41+
import os
3542

36-
3. Under `Input` select the `Python` tab, and click `Get API Key`. Then click `Generate Key`.
43+
if os.environ.get("NVIDIA_API_KEY", "").startswith("nvapi-"):
44+
print("Valid NVIDIA_API_KEY already in environment. Delete to reset")
45+
else:
46+
nvapi_key = getpass.getpass("NVAPI Key (starts with nvapi-): ")
47+
assert nvapi_key.startswith(
48+
"nvapi-"
49+
), f"{nvapi_key[:5]}... is not a valid key"
50+
os.environ["NVIDIA_API_KEY"] = nvapi_key
51+
```
3752

38-
4. Copy and save the generated key as `NVIDIA_API_KEY`. From there, you should have access to the endpoints.
53+
You can now use your key to access endpoints on the NVIDIA API Catalog.
3954

40-
```python
41-
import getpass
42-
import os
43-
44-
if os.environ.get("NVIDIA_API_KEY", "").startswith("nvapi-"):
45-
print("Valid NVIDIA_API_KEY already in environment. Delete to reset")
46-
else:
47-
nvapi_key = getpass.getpass("NVAPI Key (starts with nvapi-): ")
48-
assert nvapi_key.startswith(
49-
"nvapi-"
50-
), f"{nvapi_key[:5]}... is not a valid key"
51-
os.environ["NVIDIA_API_KEY"] = nvapi_key
52-
```
55+
## Work with the API Catalog
5356

54-
## Working with API Catalog
57+
To submit a query to the [nv-embedqa-e5-v5](https://build.nvidia.com/nvidia/nv-embedqa-e5-v5/modelcard) model,
58+
run the following code.
5559

5660
```python
5761
from llama_index.embeddings.nvidia import NVIDIAEmbedding
5862

59-
embedder = NVIDIAEmbedding()
63+
embedder = NVIDIAEmbedding(model="nv-embedqa-e5-v5")
6064
embedder.get_query_embedding("What's the weather like in Komchatka?")
6165
```
6266

63-
## Working with NVIDIA NIMs
67+
## Self-host with NVIDIA NIM Microservices
6468

65-
When ready to deploy, you can self-host models with NVIDIA NIM—which is included with the NVIDIA AI Enterprise software license—and run them anywhere, giving you ownership of your customizations and full control of your intellectual property (IP) and AI applications.
69+
When you are ready to deploy your AI application, you can self-host models with NVIDIA NIM.
70+
For more information, refer to [NVIDIA AI Enterprise](https://www.nvidia.com/en-us/data-center/products/ai-enterprise/).
6671

67-
[Learn more about NIMs](https://developer.nvidia.com/blog/nvidia-nim-offers-optimized-inference-microservices-for-deploying-ai-models-at-scale/)
72+
The following example code connects to a locally-hosted NIM Microservice.
6873

6974
```python
7075
from llama_index.embeddings.nvidia import NVIDIAEmbedding
7176

7277
# connect to an embedding NIM running at localhost:8080
7378
embedder = NVIDIAEmbeddings(base_url="http://localhost:8080/v1")
7479
```
80+
81+
## Related Topics
82+
83+
- [Overview of NeMo Retriever Text Embedding NIM](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/overview.html)

0 commit comments

Comments
 (0)