[Question]: Can't use official retrieval plugin. Bug?

[veigamann]

Contact Details

No response

What is your question?

After extensively reading the documentation for both the retrieval plugin and LibreChat, I successfully deployed the retrieval plugin API. I then used the Swagger web UI to insert and query information into the API's vector database, which worked as expected.

Following the LibreChat documentation, I added the plugin's manifest file to /api/app/clients/tools/.well-known and made sure that the filename matched the name_for_model property. The plugin now appears in the plugin store and can be installed without any problems. However, when I try to insert or query information, I consistently get Error: Request failed with status code 400.

Then, in an effort to resolve the issue, I updated the description_for_model property in the plugin manifest file to provide additional instructions for calling the API, but the error still persists.

While reviewing the retrieval API logs, I noticed that whenever the LLM attempts to use the retrieval plugin, it logs INFO: <my ip address>:55698 - "GET /openapi.yaml HTTP/1.1" 200 OK. I was expecting to see a "POST /query HTTP/1.1" 200 OK log entry, but this is not the case. It only logs the "GET /openapi.yaml" request.

For reference, I'm using gpt-3.5-turbo (with temperature 0) as the agent model and gpt-3.5-turbo-16k as the completion model.

I've taken care to follow all the necessary steps based on the documentation, but I must be missing or misunderstanding something. If there's any additional information or configuration I need to consider, I would appreciate any insights.

More Details

Librechat Logs

ask log
{
  text: 'when is my birthday? use the retrieval plugin',
  conversationId: null,
  endpointOption: {
    chatGptLabel: null,
    promptPrefix: null,
    tools: [ 'retrieval' ],
    modelOptions: {
      model: 'gpt-3.5-turbo-16k',
      temperature: 0.8,
      top_p: 1,
      presence_penalty: 0,
      frequency_penalty: 0
    },
    agentOptions: {
      agent: 'functions',
      skipCompletion: true,
      model: 'gpt-3.5-turbo',
      temperature: 0
    }
  }
}
Plugins sendMessage when is my birthday? use the retrieval plugin {
  getIds: [Function: getIds],
  user: '64d53ab7c22efbd7a631378b',
  parentMessageId: '00000000-0000-0000-0000-000000000000',
  conversationId: null,
  overrideParentMessageId: null,
  onAgentAction: [Function: onAgentAction],
  onChainEnd: [Function: onChainEnd],
  onStart: [Function: onStart],
  chatGptLabel: null,
  promptPrefix: null,
  tools: [ 'retrieval' ],
  modelOptions: {
    model: 'gpt-3.5-turbo-16k',
    temperature: 0.8,
    top_p: 1,
    presence_penalty: 0,
    frequency_penalty: 0
  },
  agentOptions: {
    agent: 'functions',
    skipCompletion: true,
    model: 'gpt-3.5-turbo',
    temperature: 0
  },
  onProgress: [Function: wrapper],
  abortController: AbortController { signal: AbortSignal { aborted: false } }
}
Loading history for conversation c9a73fb2-f91f-4b75-aa49-b3010080bb27 00000000-0000-0000-0000-000000000000
getTokenCountForMessage {
  role: 'user',
  content: 'when is my birthday? use the retrieval plugin'
}
freeAndResetAllEncoders: reached 25 encodings, resetting...
propertyTokenCounts [ 1, 9 ]
<---------------------------------DIFF--------------------------------->
Difference between payload (1) and orderedWithInstructions (1): 0
remainingContextTokens, this.maxContextTokens (1/2) 15985 15999
remainingContextTokens, this.maxContextTokens (2/2) 15985 15999
<-------------------------PAYLOAD/TOKEN COUNT MAP------------------------->
Payload: [
  {
    role: 'user',
    content: 'when is my birthday? use the retrieval plugin',
    tokenCount: 14
  }
]
Token Count Map: { '7ef0fbe8-5426-4e13-93da-3e1bb5840d91': 14 }
Prompt Tokens 14 15985 15999
{ '7ef0fbe8-5426-4e13-93da-3e1bb5840d91': 14, instructions: undefined }
userMessage.tokenCount 14
[Meilisearch] Convo not found and will index c9a73fb2-f91f-4b75-aa49-b3010080bb27
createLLM: configOptions
{}
<-----Agent Model: gpt-3.5-turbo | Temp: 0 | Functions: true----->
files [
  'Ai_PDF.json',
  'Diagrams.json',
  'Dr_Thoths_Tarot.json',
  'DreamInterpreter.json',
  'VoxScript.json',
  'aitoolhunt.json',
  'askyourpdf.json',
  'drink_maestro.json',
  'earthImagesAndVisualizations.json',
  'image_prompt_enhancer.json',
  'qrCodes.json',
  'rephrase.json',
  'retrieval.json',
  'scholarai.json',
  'uberchord.json',
  'web_search.json'
]
{
  Ai_PDF: [AsyncFunction (anonymous)],
  Diagrams: [AsyncFunction (anonymous)],
  Dr_Thoths_Tarot: [AsyncFunction (anonymous)],
  DreamInterpreter: [AsyncFunction (anonymous)],
  VoxScript: [AsyncFunction (anonymous)],
  aitoolhunt: [AsyncFunction (anonymous)],
  askyourpdf: [AsyncFunction (anonymous)],
  drink_maestro: [AsyncFunction (anonymous)],
  earthImagesAndVisualizations: [AsyncFunction (anonymous)],
  image_prompt_enhancer: [AsyncFunction (anonymous)],
  qrCodes: [AsyncFunction (anonymous)],
  rephrase: [AsyncFunction (anonymous)],
  retrieval: [AsyncFunction (anonymous)],
  scholarai: [AsyncFunction (anonymous)],
  uberchord: [AsyncFunction (anonymous)],
  web_search: [AsyncFunction (anonymous)]
}
Requested Tools
[ 'retrieval' ]
Loaded Tools
[ 'retrieval' ]
Loaded agent.
Attempt 1 of 1
[chain/start] [1:chain:AgentExecutor] Entering Chain run with input: {
  "input": "when is my birthday? use the retrieval plugin",
  "signal": {},
  "chat_history": []
}
[llm/start] [1:chain:AgentExecutor > 2:llm:ChatOpenAI] Entering LLM run with input: {
  "messages": [
    [
      {
        "lc": 1,
        "type": "constructor",
        "id": [
          "langchain",
          "schema",
          "SystemMessage"
        ],
        "kwargs": {
          "content": "You are a helpful AI assistant.",
          "additional_kwargs": {}
        }
      },
      {
        "lc": 1,
        "type": "constructor",
        "id": [
          "langchain",
          "schema",
          "HumanMessage"
        ],
        "kwargs": {
          "content": "when is my birthday? use the retrieval plugin",
          "additional_kwargs": {}
        }
      }
    ]
  ]
}
[llm/end] [1:chain:AgentExecutor > 2:llm:ChatOpenAI] [1.11s] Exiting LLM run with output: {
  "generations": [
    [
      {
        "text": "",
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "AIMessage"
          ],
          "kwargs": {
            "content": "",
            "additional_kwargs": {
              "function_call": {
                "name": "retrieval",
                "arguments": "{\n  \"query\": \"When is my birthday?\"\n}"
              }
            }
          }
        }
      }
    ]
  ],
  "llmOutput": {
    "tokenUsage": {
      "completionTokens": 19,
      "promptTokens": 93,
      "totalTokens": 112
    }
  }
}
Latest Agent Action  {
  tool: 'retrieval',
  toolInput: { query: 'When is my birthday?' },
  log: ''
}
[agent/action] [1:chain:AgentExecutor] Agent selected action: {
  "tool": "retrieval",
  "toolInput": {
    "query": "When is my birthday?"
  },
  "log": ""
}
[tool/start] [1:chain:AgentExecutor > 3:tool:DynamicStructuredTool] Entering Tool run with input: "{"query":"When is my birthday?"}"
Attempting to load an OpenAPI 3.0.2 spec. This may result in degraded performance. Convert your OpenAPI spec to 3.1.0 for better support.
[chain/start] [1:chain:SequentialChain] Entering Chain run with input: {
  "query": "when is my birthday? use the retrieval plugin\n\n||>Instructions: Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.\nPrioritize using responses for subsequent requests to better fulfill the query."
}
[chain/start] [1:chain:SequentialChain > 2:chain:LLMChain] Entering Chain run with input: {
  "query": "when is my birthday? use the retrieval plugin\n\n||>Instructions: Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.\nPrioritize using responses for subsequent requests to better fulfill the query."
}
[llm/start] [1:chain:SequentialChain > 2:chain:LLMChain > 3:llm:ChatOpenAI] Entering LLM run with input: {
  "messages": [
    [
      {
        "lc": 1,
        "type": "constructor",
        "id": [
          "langchain",
          "schema",
          "HumanMessage"
        ],
        "kwargs": {
          "content": "Use the provided API's to respond to this user query:\n\nwhen is my birthday? use the retrieval plugin\n\n||>Instructions: Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.\nPrioritize using responses for subsequent requests to better fulfill the query.",
          "additional_kwargs": {}
        }
      }
    ]
  ]
}
[llm/error] [1:chain:SequentialChain > 2:chain:LLMChain > 3:llm:ChatOpenAI] [257ms] LLM run errored with error: "Request failed with status code 400"
[chain/error] [1:chain:SequentialChain > 2:chain:LLMChain] [260ms] Chain run errored with error: "Request failed with status code 400"
[chain/error] [1:chain:SequentialChain] [263ms] Chain run errored with error: "Request failed with status code 400"
[tool/error] [1:chain:AgentExecutor > 3:tool:DynamicStructuredTool] [422ms] Tool run errored with error: "Request failed with status code 400"
[chain/error] [1:chain:AgentExecutor] [1.55s] Chain run errored with error: "Request failed with status code 400"
Error: Request failed with status code 400
    at createError (/app/node_modules/openai/node_modules/axios/lib/core/createError.js:16:15)
    at settle (/app/node_modules/openai/node_modules/axios/lib/core/settle.js:17:12)
    at IncomingMessage.handleStreamEnd (/app/node_modules/openai/node_modules/axios/lib/adapters/http.js:322:11)
    at IncomingMessage.emit (node:events:525:35)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  config: {
    transitional: {
      silentJSONParsing: true,
      forcedJSONParsing: true,
      clarifyTimeoutError: false
    },
    adapter: [Function: httpAdapter],
    transformRequest: [ [Function: transformRequest] ],
    transformResponse: [ [Function: transformResponse] ],
    timeout: 0,
    xsrfCookieName: 'XSRF-TOKEN',
    xsrfHeaderName: 'X-XSRF-TOKEN',
    maxContentLength: -1,
    maxBodyLength: -1,
    validateStatus: [Function: validateStatus],
    headers: {
      Accept: 'application/json, text/plain, */*',
      'Content-Type': 'application/json',
      'User-Agent': 'OpenAI/NodeJS/3.3.0',
      Authorization: 'Bearer sk-DFHu00A8H<redacted>R1GG2NI39',
      'Content-Length': 1942
    },
    method: 'post',
    data: '{"model":"gpt-3.5-turbo","temperature":0,"top_p":1,"frequency_penalty":0,"presence_penalty":0,"n":1,"stream":false,"functions":[{"name":"upsert_upsert_post","description":"Save chat information. Accepts an array of documents with text (potential questions + conversation text), metadata (source 'chat' and timestamp, no ID as this will be generated). Confirm with the user before saving, ask for more details/context.","parameters":{"type":"object","properties":{"data":{"type":"object","properties":{"documents":{"type":"array"}},"required":[],"additionalProperties":{}}},"required":["data"]}},{"name":"query_query_post","description":"Accepts search query objects array each with query and optional filter. Break down complex questions into sub-questions. Refine results by criteria, e.g. time / source, don't do this often. Split queries if ResponseTooLargeError occurs.","parameters":{"type":"object","properties":{"data":{"type":"object","properties":{"queries":{"type":"array"}},"required":[],"additionalProperties":{}}},"required":["data"]}}],"messages":[{"role":"user","content":"Use the provided API's to respond to this user query:\\n\\nwhen is my birthday? use the retrieval plugin\\n\\n||>Instructions: Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.\\nPrioritize using responses for subsequent requests to better fulfill the query."}]}',
    url: 'https://api.openai.com/v1/chat/completions'
  },
  request: <ref *1> ClientRequest {
    _events: [Object: null prototype] {
      abort: [Function (anonymous)],
      aborted: [Function (anonymous)],
      connect: [Function (anonymous)],
      error: [Function (anonymous)],
      socket: [Function (anonymous)],
      timeout: [Function (anonymous)],
      finish: [Function: requestOnFinish]
    },
    _eventsCount: 7,
    _maxListeners: undefined,
    outputData: [],
    outputSize: 0,
    writable: true,
    destroyed: true,
    _last: false,
    chunkedEncoding: false,
    shouldKeepAlive: true,
    maxRequestsOnConnectionReached: false,
    _defaultKeepAlive: true,
    useChunkedEncodingByDefault: true,
    sendDate: false,
    _removedConnection: false,
    _removedContLen: false,
    _removedTE: false,
    strictContentLength: false,
    _contentLength: 1942,
    _hasBody: true,
    _trailer: '',
    finished: true,
    _headerSent: true,
    _closed: true,
    socket: TLSSocket {
      _tlsOptions: [Object],
      _secureEstablished: true,
      _securePending: false,
      _newSessionPending: false,
      _controlReleased: true,
      secureConnecting: false,
      _SNICallback: null,
      servername: 'api.openai.com',
      alpnProtocol: false,
      authorized: true,
      authorizationError: null,
      encrypted: true,
      _events: [Object: null prototype],
      _eventsCount: 9,
      connecting: false,
      _hadError: false,
      _parent: null,
      _host: 'api.openai.com',
      _closeAfterHandlingError: false,
      _readableState: [ReadableState],
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: false,
      _sockname: null,
      _pendingData: null,
      _pendingEncoding: '',
      server: undefined,
      _server: null,
      ssl: [TLSWrap],
      _requestCert: true,
      _rejectUnauthorized: true,
      timeout: 5000,
      parser: null,
      _httpMessage: null,
      [Symbol(res)]: [TLSWrap],
      [Symbol(verified)]: true,
      [Symbol(pendingSession)]: null,
      [Symbol(async_id_symbol)]: -1,
      [Symbol(kHandle)]: [TLSWrap],
      [Symbol(lastWriteQueueSize)]: 0,
      [Symbol(timeout)]: Timeout {
        _idleTimeout: 5000,
        _idlePrev: [TimersList],
        _idleNext: [Timeout],
        _idleStart: 2595312,
        _onTimeout: [Function: bound ],
        _timerArgs: undefined,
        _repeat: null,
        _destroyed: false,
        [Symbol(refed)]: false,
        [Symbol(kHasPrimitive)]: false,
        [Symbol(asyncId)]: 15965,
        [Symbol(triggerId)]: 15963
      },
      [Symbol(kBuffer)]: null,
      [Symbol(kBufferCb)]: null,
      [Symbol(kBufferGen)]: null,
      [Symbol(kCapture)]: false,
      [Symbol(kSetNoDelay)]: false,
      [Symbol(kSetKeepAlive)]: true,
      [Symbol(kSetKeepAliveInitialDelay)]: 1,
      [Symbol(kBytesRead)]: 0,
      [Symbol(kBytesWritten)]: 0,
      [Symbol(connect-options)]: [Object]
    },
    _header: 'POST /v1/chat/completions HTTP/1.1\r\n' +
      'Accept: application/json, text/plain, */*\r\n' +
      'Content-Type: application/json\r\n' +
      'User-Agent: OpenAI/NodeJS/3.3.0\r\n' +
      'Authorization: Bearer sk-DFHu00A8H<redacted>R1GG2NI39\r\n' +
      'Content-Length: 1942\r\n' +
      'Host: api.openai.com\r\n' +
      'Connection: keep-alive\r\n' +
      '\r\n',
    _keepAliveTimeout: 0,
    _onPendingData: [Function: nop],
    agent: Agent {
      _events: [Object: null prototype],
      _eventsCount: 2,
      _maxListeners: undefined,
      defaultPort: 443,
      protocol: 'https:',
      options: [Object: null prototype],
      requests: [Object: null prototype] {},
      sockets: [Object: null prototype] {},
      freeSockets: [Object: null prototype],
      keepAliveMsecs: 1000,
      keepAlive: true,
      maxSockets: Infinity,
      maxFreeSockets: 256,
      scheduling: 'lifo',
      maxTotalSockets: Infinity,
      totalSocketCount: 1,
      maxCachedSessions: 100,
      _sessionCache: [Object],
      [Symbol(kCapture)]: false
    },
    socketPath: undefined,
    method: 'POST',
    maxHeaderSize: undefined,
    insecureHTTPParser: undefined,
    joinDuplicateHeaders: undefined,
    path: '/v1/chat/completions',
    _ended: true,
    res: IncomingMessage {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 4,
      _maxListeners: undefined,
      socket: null,
      httpVersionMajor: 1,
      httpVersionMinor: 1,
      httpVersion: '1.1',
      complete: true,
      rawHeaders: [Array],
      rawTrailers: [],
      joinDuplicateHeaders: undefined,
      aborted: false,
      upgrade: false,
      url: '',
      method: null,
      statusCode: 400,
      statusMessage: 'Bad Request',
      client: [TLSSocket],
      _consuming: false,
      _dumped: false,
      req: [Circular *1],
      responseUrl: 'https://api.openai.com/v1/chat/completions',
      redirects: [],
      [Symbol(kCapture)]: false,
      [Symbol(kHeaders)]: [Object],
      [Symbol(kHeadersCount)]: 40,
      [Symbol(kTrailers)]: null,
      [Symbol(kTrailersCount)]: 0
    },
    aborted: false,
    timeoutCb: null,
    upgradeOrConnect: false,
    parser: null,
    maxHeadersCount: null,
    reusedSocket: true,
    host: 'api.openai.com',
    protocol: 'https:',
    _redirectable: Writable {
      _writableState: [WritableState],
      _events: [Object: null prototype],
      _eventsCount: 3,
      _maxListeners: undefined,
      _options: [Object],
      _ended: true,
      _ending: true,
      _redirectCount: 0,
      _redirects: [],
      _requestBodyLength: 1942,
      _requestBodyBuffers: [],
      _onNativeResponse: [Function (anonymous)],
      _currentRequest: [Circular *1],
      _currentUrl: 'https://api.openai.com/v1/chat/completions',
      [Symbol(kCapture)]: false
    },
    [Symbol(kCapture)]: false,
    [Symbol(kBytesWritten)]: 0,
    [Symbol(kNeedDrain)]: false,
    [Symbol(corked)]: 0,
    [Symbol(kOutHeaders)]: [Object: null prototype] {
      accept: [Array],
      'content-type': [Array],
      'user-agent': [Array],
      authorization: [Array],
      'content-length': [Array],
      host: [Array]
    },
    [Symbol(errored)]: null,
    [Symbol(kHighWaterMark)]: 16384,
    [Symbol(kUniqueHeaders)]: null
  },
  response: {
    status: 400,
    statusText: 'Bad Request',
    headers: {
      date: 'Tue, 15 Aug 2023 18:05:37 GMT',
      'content-type': 'application/json',
      'content-length': '265',
      connection: 'keep-alive',
      'access-control-allow-origin': '*',
      'openai-organization': 'user-hkbdfgzgevahxkblkp8csj57',
      'openai-processing-ms': '9',
      'openai-version': '2020-10-01',
      'strict-transport-security': 'max-age=15724800; includeSubDomains',
      'x-ratelimit-limit-requests': '3500',
      'x-ratelimit-limit-tokens': '90000',
      'x-ratelimit-remaining-requests': '3499',
      'x-ratelimit-remaining-tokens': '89770',
      'x-ratelimit-reset-requests': '17ms',
      'x-ratelimit-reset-tokens': '152ms',
      'x-request-id': 'f59379fad3a69758c81896e15d145ab3',
      'cf-cache-status': 'DYNAMIC',
      server: 'cloudflare',
      'cf-ray': '7f735de699efa6bb-GRU',
      'alt-svc': 'h3=":443"; ma=86400'
    },
    config: {
      transitional: [Object],
      adapter: [Function: httpAdapter],
      transformRequest: [Array],
      transformResponse: [Array],
      timeout: 0,
      xsrfCookieName: 'XSRF-TOKEN',
      xsrfHeaderName: 'X-XSRF-TOKEN',
      maxContentLength: -1,
      maxBodyLength: -1,
      validateStatus: [Function: validateStatus],
      headers: [Object],
      method: 'post',
      data: '{"model":"gpt-3.5-turbo","temperature":0,"top_p":1,"frequency_penalty":0,"presence_penalty":0,"n":1,"stream":false,"functions":[{"name":"upsert_upsert_post","description":"Save chat information. Accepts an array of documents with text (potential questions + conversation text), metadata (source 'chat' and timestamp, no ID as this will be generated). Confirm with the user before saving, ask for more details/context.","parameters":{"type":"object","properties":{"data":{"type":"object","properties":{"documents":{"type":"array"}},"required":[],"additionalProperties":{}}},"required":["data"]}},{"name":"query_query_post","description":"Accepts search query objects array each with query and optional filter. Break down complex questions into sub-questions. Refine results by criteria, e.g. time / source, don't do this often. Split queries if ResponseTooLargeError occurs.","parameters":{"type":"object","properties":{"data":{"type":"object","properties":{"queries":{"type":"array"}},"required":[],"additionalProperties":{}}},"required":["data"]}}],"messages":[{"role":"user","content":"Use the provided API's to respond to this user query:\\n\\nwhen is my birthday? use the retrieval plugin\\n\\n||>Instructions: Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.\\nPrioritize using responses for subsequent requests to better fulfill the query."}]}',
      url: 'https://api.openai.com/v1/chat/completions'
    },
    request: <ref *1> ClientRequest {
      _events: [Object: null prototype],
      _eventsCount: 7,
      _maxListeners: undefined,
      outputData: [],
      outputSize: 0,
      writable: true,
      destroyed: true,
      _last: false,
      chunkedEncoding: false,
      shouldKeepAlive: true,
      maxRequestsOnConnectionReached: false,
      _defaultKeepAlive: true,
      useChunkedEncodingByDefault: true,
      sendDate: false,
      _removedConnection: false,
      _removedContLen: false,
      _removedTE: false,
      strictContentLength: false,
      _contentLength: 1942,
      _hasBody: true,
      _trailer: '',
      finished: true,
      _headerSent: true,
      _closed: true,
      socket: [TLSSocket],
      _header: 'POST /v1/chat/completions HTTP/1.1\r\n' +
        'Accept: application/json, text/plain, */*\r\n' +
        'Content-Type: application/json\r\n' +
        'User-Agent: OpenAI/NodeJS/3.3.0\r\n' +
        'Authorization: Bearer sk-DFHu00A8H<redacted>R1GG2NI39\r\n' +
        'Content-Length: 1942\r\n' +
        'Host: api.openai.com\r\n' +
        'Connection: keep-alive\r\n' +
        '\r\n',
      _keepAliveTimeout: 0,
      _onPendingData: [Function: nop],
      agent: [Agent],
      socketPath: undefined,
      method: 'POST',
      maxHeaderSize: undefined,
      insecureHTTPParser: undefined,
      joinDuplicateHeaders: undefined,
      path: '/v1/chat/completions',
      _ended: true,
      res: [IncomingMessage],
      aborted: false,
      timeoutCb: null,
      upgradeOrConnect: false,
      parser: null,
      maxHeadersCount: null,
      reusedSocket: true,
      host: 'api.openai.com',
      protocol: 'https:',
      _redirectable: [Writable],
      [Symbol(kCapture)]: false,
      [Symbol(kBytesWritten)]: 0,
      [Symbol(kNeedDrain)]: false,
      [Symbol(corked)]: 0,
      [Symbol(kOutHeaders)]: [Object: null prototype],
      [Symbol(errored)]: null,
      [Symbol(kHighWaterMark)]: 16384,
      [Symbol(kUniqueHeaders)]: null
    },
    data: { error: [Object] }
  },
  isAxiosError: true,
  toJSON: [Function: toJSON],
  attemptNumber: 1,
  retriesLeft: 6
}
Failed to parse JSON: SyntaxError: Unexpected token 'w', "when is my"... is not valid JSON
    at JSON.parse (<anonymous>)
    at findMessageContent (/app/api/utils/findMessageContent.js:22:30)
    at PluginsClient.executorCall (/app/api/app/clients/PluginsClient.js:297:25)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async PluginsClient.sendMessage (/app/api/app/clients/PluginsClient.js:414:5)
    at async ask (/app/api/server/routes/ask/gptPlugins.js:222:20)
    at async /app/api/server/routes/ask/gptPlugins.js:67:10
Stream ended
getTokenCountForMessage {
  role: 'assistant',
  content: 'Encountered an error while attempting to respond. Error: Request failed with status code 400'
}
propertyTokenCounts [ 1, 19 ]
[Meilisearch] Convo not found and will index c9a73fb2-f91f-4b75-aa49-b3010080bb27
CLIENT RESPONSE
{
  messageId: 'c8fc40c2-f228-440a-ae30-c845d1b8c8d6',
  conversationId: 'c9a73fb2-f91f-4b75-aa49-b3010080bb27',
  parentMessageId: '7ef0fbe8-5426-4e13-93da-3e1bb5840d91',
  isCreatedByUser: false,
  model: 'gpt-3.5-turbo-16k',
  sender: 'Assistant',
  promptTokens: 14,
  text: 'Encountered an error while attempting to respond. Error: Request failed with status code 400',
  completionTokens: 24,
  output: 'Encountered an error while attempting to respond. Error: Request failed with status code 400',
  intermediateSteps: [
    {
      tool: 'retrieval',
      toolInput: { query: 'When is my birthday?' },
      log: ''
    }
  ],
  errorMessage: 'Request failed with status code 400'
}
CONVERSATION TITLE Error Encountered While Retrieving Birthday
[Meilisearch] Convo not found and will index c9a73fb2-f91f-4b75-aa49-b3010080bb27

What is the main subject of your question?

Other

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

It's fetching the openapi spec to generate the functions it will use.
"GET /openapi.yaml HTTP/1.1" 200 OK

It would be helpful to see the logs from the LibreChat side when it fails, but I suspect it's having trouble either interpreting the spec or there is a server url mismatch.

do you also mind sharing what your .well-known/openapi.yaml looks like?

See this part for more detail: https://docs.librechat.ai/features/plugins/chatgpt_plugins_openapi.html#custom-openapi-spec-files

[veigamann]

Sure! Here:

openapi.yaml

openapi: 3.0.2
info:
  title: Retrieval Plugin API
  description: A retrieval API for querying and filtering documents based on natural language queries and metadata
  version: 1.0.0
  servers:
    - url: https://retrieval-plugin.mydomain.tld
paths:
  /upsert:
    post:
      summary: Upsert
      description: Save chat information. Accepts an array of documents with text (potential questions + conversation text), metadata (source 'chat' and timestamp, no ID as this will be generated). Confirm with the user before saving, ask for more details/context.
      operationId: upsert_upsert_post
      requestBody:
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/UpsertRequest"
        required: true
      responses:
        "200":
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/UpsertResponse"
        "422":
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/HTTPValidationError"
      security:
        - HTTPBearer: []
  /query:
    post:
      summary: Query
      description: Accepts search query objects array each with query and optional filter. Break down complex questions into sub-questions. Refine results by criteria, e.g. time / source, don't do this often. Split queries if ResponseTooLargeError occurs.
      operationId: query_query_post
      requestBody:
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/QueryRequest"
        required: true
      responses:
        "200":
          description: Successful Response
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/QueryResponse"
        "422":
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/HTTPValidationError"
      security:
        - HTTPBearer: []
components:
  schemas:
    Document:
      title: Document
      required:
        - text
      type: object
      properties:
        id:
          title: Id
          type: string
        text:
          title: Text
          type: string
        metadata:
          $ref: "#/components/schemas/DocumentMetadata"
    DocumentChunkMetadata:
      title: DocumentChunkMetadata
      type: object
      properties:
        source:
          $ref: "#/components/schemas/Source"
        source_id:
          title: Source Id
          type: string
        url:
          title: Url
          type: string
        created_at:
          title: Created At
          type: string
        author:
          title: Author
          type: string
        document_id:
          title: Document Id
          type: string
    DocumentChunkWithScore:
      title: DocumentChunkWithScore
      required:
        - text
        - metadata
        - score
      type: object
      properties:
        id:
          title: Id
          type: string
        text:
          title: Text
          type: string
        metadata:
          $ref: "#/components/schemas/DocumentChunkMetadata"
        embedding:
          title: Embedding
          type: array
          items:
            type: number
        score:
          title: Score
          type: number
    DocumentMetadata:
      title: DocumentMetadata
      type: object
      properties:
        source:
          $ref: "#/components/schemas/Source"
        source_id:
          title: Source Id
          type: string
        url:
          title: Url
          type: string
        created_at:
          title: Created At
          type: string
        author:
          title: Author
          type: string
    DocumentMetadataFilter:
      title: DocumentMetadataFilter
      type: object
      properties:
        document_id:
          title: Document Id
          type: string
        source:
          $ref: "#/components/schemas/Source"
        source_id:
          title: Source Id
          type: string
        author:
          title: Author
          type: string
        start_date:
          title: Start Date
          type: string
        end_date:
          title: End Date
          type: string
    HTTPValidationError:
      title: HTTPValidationError
      type: object
      properties:
        detail:
          title: Detail
          type: array
          items:
            $ref: "#/components/schemas/ValidationError"
    Query:
      title: Query
      required:
        - query
      type: object
      properties:
        query:
          title: Query
          type: string
        filter:
          $ref: "#/components/schemas/DocumentMetadataFilter"
        top_k:
          title: Top K
          type: integer
          default: 3
    QueryRequest:
      title: QueryRequest
      required:
        - queries
      type: object
      properties:
        queries:
          title: Queries
          type: array
          items:
            $ref: "#/components/schemas/Query"
    QueryResponse:
      title: QueryResponse
      required:
        - results
      type: object
      properties:
        results:
          title: Results
          type: array
          items:
            $ref: "#/components/schemas/QueryResult"
    QueryResult:
      title: QueryResult
      required:
        - query
        - results
      type: object
      properties:
        query:
          title: Query
          type: string
        results:
          title: Results
          type: array
          items:
            $ref: "#/components/schemas/DocumentChunkWithScore"
    Source:
      title: Source
      enum:
        - email
        - file
        - chat
      type: string
      description: An enumeration.
    UpsertRequest:
      title: UpsertRequest
      required:
        - documents
      type: object
      properties:
        documents:
          title: Documents
          type: array
          items:
            $ref: "#/components/schemas/Document"
    UpsertResponse:
      title: UpsertResponse
      required:
        - ids
      type: object
      properties:
        ids:
          title: Ids
          type: array
          items:
            type: string
    ValidationError:
      title: ValidationError
      required:
        - loc
        - msg
        - type
      type: object
      properties:
        loc:
          title: Location
          type: array
          items:
            anyOf:
              - type: string
              - type: integer
        msg:
          title: Message
          type: string
        type:
          title: Error Type
          type: string
  securitySchemes:
    HTTPBearer:
      type: http
      scheme: bearer

Also the manifest file in case you need it:

ai-plugin.json (/api/app/clients/tools/.well-known/retrieval.json)

{
    "schema_version": "v1",
    "name_for_model": "retrieval",
    "name_for_human": "Retrieval Plugin",
    "description_for_model": "Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.",
    "description_for_human": "Search through your documents.",
    "auth": {
      "type": "user_http",
      "authorization_type": "bearer"
    },
    "api": {
      "type": "openapi",
      "url": "https://retrieval-plugin.mydomain.tld/.well-known/openapi.yaml",
      "has_user_authentication": false
    },
    "logo_url": "https://retrieval-plugin.mydomain.tld/.well-known/logo.png",
    "contact_email": "hello@contact.com",
    "legal_info_url": "hello@legal.com"
  }

Note: Figured I'd change the actual API URL from the spec/manifest file before posting here for security reasons, even though it's for testing purposes only.

[danny-avila]

"auth": {
"type": "user_http",
"authorization_type": "bearer"
},

I can see there potentially being an issue with authorization and no verification token

I noticed the retrieval plugin requires a BEARER_TOKEN, it shows in retrieval docs it's required.

I'm only handling the specific setup shown here atm, because it's how a lot of chatgpt plugins authenticate. You can mirror the setup from the librechat docs.

it would also help to see the logs from librechat when you experience this:

[danny-avila]

my bad, i didn't see you already posted the logs. will look at them now

[danny-avila]

Ok so from what I can tell OpenAI is rejecting the function call even tho it might be correct because the description may be too long, in particular for query_query_post.

you can try including the openapi schema manually and try making that shorter. OpenAI has no documentation on this but I've discovered in the past a hard limit on the function description and it's pretty short.

Part of this will need reworking on my end as I will have to mess with langchain/the openAI api in a more granular fashion and I hope to get to it soon. it should make the chatgpt plugins more reliable when I get to it

[veigamann]

Apologies for not highlighting the logs more clearly in the issue description, my bad. I'll make sure they're more visible next time.

Regarding the BEARER_TOKEN, I followed the librechat docs and tried that approach initially. It didn't seem to work or change anything, so I reversed it. I'll revisit this and try again, ensuring my setup matches the documentation.

As for manually including the OpenAPI schema, if by that you mean adding a /api/app/clients/tools/.well-known/openapi/retrieval.yaml file with the spec, and adjusting the manifest as follows:

"api": {
      "type": "openapi",
      "url": "retrieval.yaml",
      "has_user_authentication": false
    },

I did attempt that, but it didn't work either, unfortunately. However, I'll try shortening the query_query_post and let you know if that helps.

[veigamann]

Made the following changes:

• Implemented the BEARER_TOKEN as outlined in the LibreChat docs.
• Simplified the descriptions for the /upsert and /query endpoints in the OpenAPI spec, and renamed query_query_post to query_post, likewise for upsert.
• Added the spec to /api/app/clients/tools/.well-known/openapi/retrieval.yaml and modified the manifest accordingly.

However, still not working. Resulting in the same error: Request failed with status code 400.

Here's the updated spec and manifest:

Updated OpenAPI spec (abbreviated)

paths:
  /upsert:
    post:
      summary: Upsert
      # Shortened the description
      description: Save chat documents with text, metadata like source and timestamp. Confirm with user before saving.
      operationId: upsert_post # previously was 'upsert_upsert_post'
      # Omitted for brevity
  /query:
    post:
      summary: Query
      # Shortened the description
      description: Accepts search queries with text and filters. Break into sub-queries if needed. Refine by criteria like time and source. Handle large responses.
      operationId: query_post # previously was 'query_query_post'
      # Omitted for brevity

Updated manifest file (abbreviated)

{
    // Ommited for brevity
    "auth": {
      "type": "service_http",
      "authorization_type": "bearer",
      "verification_tokens": {
        "openai": "eyJhb<token>DQI"
      }
    },
    "api": {
      "type": "openapi",
      "url": "retrieval.yaml",
      "has_user_authentication": false
    }
    // Omitted for brevity
  }

[veigamann]

I've been looking into the LibreChat logs, and it seems that createOpenAPIChain might be used internally for API calls. I'm not familiar with the codebase, and I know there's a lot more prompt injection, it's not that simple, so I apologize if I'm mistaken. I decided to test it out in a repl:

import { createOpenAPIChain } from "langchain/chains";
import { ChatOpenAI } from "langchain/chat_models/openai";

const apiKey = "sk-..."
const apiSpec = "https://retrieval-plugin.mydomain.tld/.well-known/openapi.yaml";

const chatModel = new ChatOpenAI({ modelName: "gpt-3.5-turbo-0613", temperature: 0, openAIApiKey: apiKey });

const chain = await createOpenAPIChain(apiSpec, {
  verbose: true,
  llm: chatModel,
  headers: {
    authorization: "Bearer eyJhb<token>DQI",
  }
});

try {
  const result = await chain.run(`Save this for later: my birthday is on august 4th, 2001`);
  console.log(JSON.stringify(result, null, 2));
} catch (e) {
  console.dir(e.response.data);
}

Got this error:

{
  error: {
    message: [
      "Invalid schema for function 'upsert_post': In context=('properties', 'data', 'properties', 'documents'), array schema missing items"
    ],
    type: 'invalid_request_error',
    param: null,
    code: null
  }
}

I'm not sure if this is relevant at all, but it may be why we're seeing the bad request error? I've been googling for a while, couldn't find anything.

[danny-avila]

I've been looking into the LibreChat logs, and it seems that createOpenAPIChain might be used internally for API calls. I'm not familiar with the codebase, and I know there's a lot more prompt injection, it's not that simple, so I apologize if I'm mistaken. I decided to test it out in a repl:

import { createOpenAPIChain } from "langchain/chains";
import { ChatOpenAI } from "langchain/chat_models/openai";

const apiKey = "sk-..."
const apiSpec = "https://retrieval-plugin.mydomain.tld/.well-known/openapi.yaml";

const chatModel = new ChatOpenAI({ modelName: "gpt-3.5-turbo-0613", temperature: 0, openAIApiKey: apiKey });

const chain = await createOpenAPIChain(apiSpec, {
  verbose: true,
  llm: chatModel,
  headers: {
    authorization: "Bearer eyJhb<token>DQI",
  }
});

try {
  const result = await chain.run(`Save this for later: my birthday is on august 4th, 2001`);
  console.log(JSON.stringify(result, null, 2));
} catch (e) {
  console.dir(e.response.data);
}

Got this error:

{
  error: {
    message: [
      "Invalid schema for function 'upsert_post': In context=('properties', 'data', 'properties', 'documents'), array schema missing items"
    ],
    type: 'invalid_request_error',
    param: null,
    code: null
  }
}

I'm not sure if this is relevant at all, but it may be why we're seeing the bad request error? I've been googling for a while, couldn't find anything.

in your example, the LLM is generating a payload that doesn't conform to the schema spec, and langchain is strict in aligning the two, so it will throw a schema error (createOpenAPIChain).

In the original error, (status code 400), that is a problem with OpenAI API and the "function" payload. The plugin is converted to a function as understood by the OpenAI API. Note how the API reference says nothing about function description length, but there is a hard limit I've encountered on top of other issues with function formation that are not documented by OpenAI.

Since this is an "official" OpenAI chatgpt plugin, I think it will be worth it to convert this to a langchain tool for reliability

Also other than this, my work with the chatgpt plugins is a first iteration. I've been making notes on how to improve it, including with issues like this where the problem originates with a bad API call to OpenAI for generation, and also focusing on better prompt injecting as you mentioned. I'm wrapping up other features, but will tackle this soon since I already started on it, too.

[danny-avila]

@veigamann I made a lot of general improvements to the handling of plugins in #845, which is now merged

Let me know how the retrieval plugin fares as I haven't had time to set it up.

Note that file support/vectorization is a next major step I'm taking with the project.

[veigamann]

Sorry for the delay, I will try it out later today or tomorrow in the afternoon/evening and get back to you.

[danny-avila]

Sorry for the delay, I will try it out later today or tomorrow in the afternoon/evening and get back to you.

No worries, thanks for doing that!