Structured Output fails with specific combination of types

[nicolablago]

Package

langchain-google-vertexai

Checked other resources

• I added a descriptive title to this issue
• I searched the LangChain documentation with the integrated search
• I used the GitHub search to find a similar issue and didn't find it
• I am sure this is a bug and not a question or request for help

Example Code

from langchain_core.prompts import ChatPromptTemplate
from langchain_google_vertexai import ChatVertexAI
from pydantic import BaseModel, Field


class Dress(BaseModel):
    size: str | list[str] | None = Field(description="The size of the dress")


model = ChatVertexAI(
    model="gemini-2.5-flash",
    temperature=0,
    project="...",
    location="europe-west4",
    verbose=True,
    thinking_budget=0
)

prompt = ChatPromptTemplate.from_template(
    "Given the context below, provide the size of the dress.\n\nContext: {context}\n\n"
)
structured_llm = model.with_structured_output(Dress)
chain = prompt | structured_llm
result = chain.invoke(
    {
        "context": "The dress is red and available in sizes S, M, and L.",
    }
)
print(result)

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 954, in _ConvertScalarFieldValue
    number = int(value)
             ^^^^^^^^^^
ValueError: invalid literal for int() with base 10: 'NULL'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 959, in _ConvertScalarFieldValue
    raise EnumStringValueParseError(
google.protobuf.json_format.EnumStringValueParseError: Invalid enum value NULL for enum type google.cloud.aiplatform.v1beta1.Type

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 709, in _ConvertFieldValuePair
    self._ConvertAndSetScalar(
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 872, in _ConvertAndSetScalar
    value = _ConvertScalarFieldValue(js_value, field, path)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 975, in _ConvertScalarFieldValue
    raise EnumStringValueParseError('{0} at {1}'.format(e, path)) from e
google.protobuf.json_format.EnumStringValueParseError: Invalid enum value NULL for enum type google.cloud.aiplatform.v1beta1.Type at Schema.properties[size].anyOf[2].type

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 688, in _ConvertFieldValuePair
    self.ConvertMessage(
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 546, in ConvertMessage
    self._ConvertFieldValuePair(value, message, path)
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 714, in _ConvertFieldValuePair
    raise ParseError(
google.protobuf.json_format.ParseError: Failed to parse type field: Invalid enum value NULL for enum type google.cloud.aiplatform.v1beta1.Type at Schema.properties[size].anyOf[2].type.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 661, in _ConvertFieldValuePair
    self._ConvertMapFieldValue(
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 855, in _ConvertMapFieldValue
    self.ConvertMessage(
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 546, in ConvertMessage
    self._ConvertFieldValuePair(value, message, path)
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 714, in _ConvertFieldValuePair
    raise ParseError(
google.protobuf.json_format.ParseError: Failed to parse anyOf field: Failed to parse type field: Invalid enum value NULL for enum type google.cloud.aiplatform.v1beta1.Type at Schema.properties[size].anyOf[2].type..

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/.../scratch3.py", line 25, in <module>
    structured_llm = model.with_structured_output(Dress)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.venv/lib/python3.12/site-packages/langchain_google_vertexai/chat_models.py", line 2590, in with_structured_output
    tool_name = _get_tool_name(schema)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.venv/lib/python3.12/site-packages/langchain_google_vertexai/chat_models.py", line 2783, in _get_tool_name
    vertexai_tool = _format_to_gapic_tool([tool])
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.venv/lib/python3.12/site-packages/langchain_google_vertexai/functions_utils.py", line 366, in _format_to_gapic_tool
    fd = _format_to_gapic_function_declaration(tool)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.venv/lib/python3.12/site-packages/langchain_google_vertexai/functions_utils.py", line 275, in _format_to_gapic_function_declaration
    return _format_pydantic_to_function_declaration(tool)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.venv/lib/python3.12/site-packages/langchain_google_vertexai/functions_utils.py", line 223, in _format_pydantic_to_function_declaration
    parameters=_dict_to_gapic_schema(schema, pydantic_version=pydantic_version),
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.venv/lib/python3.12/site-packages/langchain_google_vertexai/functions_utils.py", line 174, in _dict_to_gapic_schema
    return gapic.Schema.from_json(json_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../.venv/lib/python3.12/site-packages/proto/message.py", line 549, in from_json
    Parse(payload, instance._pb, ignore_unknown_fields=ignore_unknown_fields)
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 470, in Parse
    raise e
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 466, in Parse
    return ParseDict(
           ^^^^^^^^^^
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 500, in ParseDict
    parser.ConvertMessage(js_dict, message, '')
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 546, in ConvertMessage
    self._ConvertFieldValuePair(value, message, path)
  File "/home/.../.venv/lib/python3.12/site-packages/google/protobuf/json_format.py", line 714, in _ConvertFieldValuePair
    raise ParseError(
google.protobuf.json_format.ParseError: Failed to parse properties field: Failed to parse anyOf field: Failed to parse type field: Invalid enum value NULL for enum type google.cloud.aiplatform.v1beta1.Type at Schema.properties[size].anyOf[2].type...

Description

When using a pydantic model with specific types, the request for structured output fails:

• str ✅
• str | None ✅
• list[str] ✅
• list[str] | None ✅
• str | list[str] ✅
• str | list[str] | None ❌

[alipqb]

Hi,
I'm facing the same problem when I use VertexAI with the langchain deepagent.
This is the error I get:

Failed to parse properties field: Failed to parse anyOf field: Failed to parse type field: Invalid enum value NULL for enum type google.cloud.aiplatform.v1beta1.Type at Schema.properties[fields].anyOf[2].type..

My output structured schema is a very simple Pydantic schema:

class OutputSchema(BaseModel):
    output: str

I think I get the error for tool calls.

[mdrxy]

Investigating