langchain integration not compatible with GPT 3.5 or GPT 4

[mmoya91]

Hello, I'm trying to use Azure's OpenAI service with spacy-llm via langchain given that the Azure integration is not currently supported

explosion/spaCy#12653

However, I noticed that the langchain model is using prompting structure that's consistent with openAI GPT-3 and prior rather than GPT-3.5 or GPT-4 which uses langchain.chat_models
https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/chat_models/base.py#L82

rather than langchain.llms
https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/llms/base.py#L158

In particular, this line seems to be referencing langchain.llms

spacy-llm/spacy_llm/models/langchain/model.py

Line 16 in 8e9a11b

["langchain.llms.base.BaseLLM", Iterable[Any]],

as a result, I'm getting the following error when I try considering a pinned updated version of langchain and GPT-4 or GPT-3.5

openai.error.InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.chat_completion.ChatCompletion'>

[mmoya91]

should we consider registering a new component like azureopenai? If so, what steps would you recommend around that?

[rmitsch]

Hi @mmoya91, thanks for bringing that up. Native OpenAI Azure support is on our backlog, but we weren't aware that it didn't work via the langchain integration either.

We'll look into this and publish a fix - hopefully later this week. A new component isn't needed. We might add new model handles (e. g. spacy.GPT-3-5-Azure.v1) or add an config parameter determining whether to run already supported OpenAI models on Azure. In any case we'll fix how we integrate non-BaseLLM models from langchain.

[mmoya91]

thank you @rmitsch would greatly appreciate langchain support for

langchain.chat_models.base.BaseChatModel

for GPT-3.5 and GPT-4 in addition to

langchain.llms.base.BaseLLM
for models prior

[rmitsch]

1. Native support for Azure OpenAI will be available with the 0.6.0 release (out tomorrow or on Monday).
2. The current langchain integration already supports chat models. The typing isn't accurate, this will be fixed in spacy-llm 0.6.0, but this doesn't impact the outcome.

Example for running spacy-llm >= 0.5.x with langchain and Azure Open AI:

# This assumes you have your Azure OpenAI key in an env variable called  OPENAI_API_KEY.
_pipe_cfg = {
    "model": {
        "@llm_models": "langchain.AzureOpenAI.v1",
        "name": "gpt-35-turbo",  # Note: this is the MODEL name, this will always be one of the existing models.
        "config": {
            "deployment_name": "gpt-35-turbo",  # This is the deployment name and can be arbitrary. In this case our deployment name is identical to the model name.
            "openai_api_version": "2023-05-15",  # Adjust if necessary.
            "openai_api_base": "https://xxx.openai.azure.com/"  # Use your Azure OpenAI endpoint here.
        },
    },
    "task": {"@llm_tasks": "spacy.NoOp.v1"},
   }

nlp = spacy.blank("en")
nlp.add_pipe("llm", config=_pipe_cfg)
nlp("This is a test.")

[rmitsch]

openai.error.InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.chat_completion.ChatCompletion'>

This error is due to not providing a deployment_name. This is necessary in order for Azure OpenAI to work, which is why langchain fails without one.

I'll close this issue, as this is not an issue - feel free to open a discussion if you have further usage questions though!

[rmitsch]

Note: v0.6.0 is out and has native support for Azure OpenAI.

[skanda1005]

I am getting this error with AzureOpen AI

openai.error.InvalidRequestError: The completion operation does not work with the specified model, gpt-4. Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993.

this is because it doesn't support the completions endpoint but rather chat completions. how do I overcome this issue?

[rmitsch]

This is a bug that was fixed in #337. The fix will be included in spacy-llm 0.6.3, which will probably be released beginning of next week.

[skanda1005]

Is this 0.6.3 release yet to be released? When is it expected to be out?