HuggingFaceEndpoint: max_new_tokens not applied to the model in practice

[RobGhys]

Checked other resources

• This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/).
• I added a clear and descriptive title that summarizes this issue.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• I read what a minimal reproducible example is (https://stackoverflow.com/help/minimal-reproducible-example).
• I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Example Code

import os
import getpass
from langchain_huggingface import ChatHuggingFace, HuggingFaceEndpoint

def main():

    os.environ["HUGGINGFACEHUB_API_TOKEN"] = getpass.getpass(
        "Enter your Hugging Face API key: "
    )

    llm = HuggingFaceEndpoint(
        repo_id="deepseek-ai/DeepSeek-R1-Distill-Qwen-7B",
        task="text-generation",
        max_new_tokens=100,
        streaming=True,
        do_sample=False,
        temperature=0.9,
        repetition_penalty=1.03,
        provider="auto", 
        server_kwargs={"bill_to": "KandoCare"}
    )

    chat_model = ChatHuggingFace(llm=llm)

    from langchain_core.messages import (
        HumanMessage,
        SystemMessage,
    )

    messages = [
        SystemMessage(content="You're a helpful assistant"),
        HumanMessage(
            content="What happens when an unstoppable force meets an immovable object?"
        ),
    ]

    for chunk in chat_model.stream(messages):
        print(chunk.content, end="", flush=True)

    print('\n***done***')

if __name__ == '__main__':
    main()

Error Message and Stack Trace (if applicable)

(.venv) ~/github/demo:[master]
python huggingface_langgraph.py
Okay, so I came across this question about what happens when an unstoppable force meets an immovable object. At first glance, it seems like a classic physics paradox. I remember hearing about the philosophical riddle before, right? Like the idea of a bullet hitting a block that can't move. But now I'm trying to break it down logically. Let me think through it step by step.

So, if there's a bullet fired at a wall that's unshakable, how does that even happen? The bullet has mass and velocity, so it should hit the wall. If the wall is truly immovable, does that mean it doesn't apply any force back? Or does it somehow get deformed? But wait, if the wall can't move, maybe the energy is absorbed somehow. But realistically, isn't the bullet still transferring its momentum to the wall? Maybe the wall absorbs it, causing some kind of wave or something. But if the wall can't move, maybe the energy disperses into the surrounding area instead. Hmm, but in reality, wouldn't that cause some kind of pressure wave until... Wait, but if the wall can't move, where does the energy go?

Maybe the bullet just stops because the wall can't provide the necessary reaction. So, the bullet would just stop where it hits. But then, what happens after that? Does it dissipate the kinetic energy without the wall moving? Maybe the energy is lost as heat or sound? Or maybe the bullet bounces back with less energy. But if the wall is unbreakable, maybe it just stops. I'm not sure. I guess in reality, the bullet might just decelerate and dissipate its energy elsewhere. But in the scenario, since the wall can't move, maybe it just stops and dissipates energy as heat or something else. Or perhaps the energy is transferred as waves radiating outwards. But I'm not sure. Alternatively, maybe it keeps going on infinitely, but that doesn't make sense. Maybe the bullet would just stop. I'm getting confused here. I think the bullet would just embed itself inside the wall, but the wall can't move, so maybe the energy spreads out. I'm not entirely certain. I need to consider forces and momentum conservation. Momentum transfer requires both objects interacting. If the wall can't move, the bullet can't push back, so maybe it just stops. So the bullet just stops and maybe the energy spreads as waves. Or maybe the energy is dispersed. Alternatively, maybe the system just becomes zero, but how?

Wait, in physics, when two objects collide, they exchange momentum. If one can't move, the force from the bullet would be absorbed, but since the wall can't react, maybe it just stops. So the bullet just stops and dissipates energy as heat or spreads. I'm not entirely sure, but I'm not certain. Maybe the bullet just stops and the system's energy is dispersed. I'm getting stuck in circles. Maybe it's like a static situation. Maybe the bullet just stops, and energy is lost as heat or something else. I'm not entirely sure. I think the bullet would just stop and dissipate energy as waves. Or maybe the energy is spread outwards. I'm getting confused. I'm not sure. Maybe it's more about conservation of momentum. If the wall can't move, the bullet just stops, but then what about the energy? I'm not sure. Maybe the bullet just stops and dissipates energy. I'm getting stuck. I think the bullet just stops and energy spreads. Or perhaps the bullet just stops and the energy is lost as sound. Not sure. I'm confused. I'm not certain. Maybe the bullet just stops, but the energy is dissipated. I'm not sure. I'm not sure. Maybe the bullet just stops, and the energy is dispersed. Or maybe the bullet just stops and the energy is radiated. I'm getting stuck. Maybe the bullet just stops, and energy is spread outwards. Hmm, not sure. I'm getting tangled up. I think the bullet stops and energy is dissipated. I'm not sure. Maybe the bullet just stops, and energy is spread. Or perhaps the bullet just stops, and energy is spread. I'm confused. I'm not certain. Maybe the bullet stops and energy is lost as sound. I'm not sure. I'm getting stuck. Maybe the bullet just stops, and energy is dissipated. I'm not certain. Maybe the bullet just stops and the energy is dispersed. I'm not sure. Maybe the bullet just stops and energy is spread. I'm getting tangled. I'm confused. I think the bullet stops and energy is dissipated. I'm not sure. I'm getting stuck. Maybe the bullet just stops and energy is spread. Or perhaps the bullet just stops and energy is lost. I'm not certain. I'm not sure. I'm getting confused. Maybe the bullet just stops, and energy is dissipated. I'm not sure. Maybe the bullet stops and energy is spread. I'm not certain. I'm stuck. Maybe the bullet just stops and the energy is dissipated. I'm not sure. I'm getting tangled. I think the bullet stops and energy is spread. Or perhaps the bullet stops and energy is lost. I'm confused. Maybe the bullet just stops, and energy is dissipated. I'm not certain. I'm not sure. I'm getting stuck. I think the bullet just stops and energy is dissipated. Or perhaps the bullet stops and energy is spread. I'm not certain. I'm getting confused. I'm not sure. Maybe the bullet stops and energy is dissipated. I'm not certain. I'm stuck. I think the bullet just stops and energy is spread. Or perhaps the bullet stops and energy is lost. I'm confused. Maybe the bullet stops and energy is dissipated. I'm not sure. I'm getting tangled. I think the bullet stops and energy is dissipated. I'm not certain. Maybe the bullet stops and energy is spread. Or perhaps the bullet stops and energy is lost. I'm not sure. I'm confused. I'm getting stuck. I think the bullet just stops and energy is dissipated. I'm not certain. I'm getting tangled. Maybe the bullet stops and energy is spread. Or maybe the bullet stops and energy is lost. I'm not sure. I'm getting stuck. I think the bullet just stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. Maybe the bullet just stops and energy is dissipated. I'm not sure. I'm confused. I'm getting tangled. I think the bullet stops and energy is spread. Or perhaps the bullet stops and energy is lost. I'm not certain. I'm confused. Maybe the bullet stops and energy is dissipated. I'm not sure. I'm getting stuck. I think the bullet stops and energy is spread. Or maybe the bullet stops and energy is lost. I'm not sure. I'm getting confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm not certain. I'm confused. I'm getting tangled. I think the bullet stops and energy is dissipated. I'm not sure. Maybe the bullet stops and energy is spread. Or perhaps the bullet stops and energy is lost. I'm not certain. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not sure. I'm getting tangled. I'm not certain. I'm confused. I think the bullet stops and energy is dissipated. I'm getting stuck. Maybe the bullet stops and energy is spread. Or perhaps the bullet stops and energy is lost. I'm not sure. I'm confused. I'm getting tangled. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not certain. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm not sure. I'm getting confused. I'm getting tangled. I'm not certain. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not sure. I'm confused. I'm getting stuck. I'm getting tangled. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm not certain. I'm confused. I'm getting confused. I'm stuck. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not sure. I'm getting stuck. I'm confused. I'm getting tangled. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm not certain. I'm confused. I'm getting stuck. I'm confused. I'm getting tangled. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not sure. I'm confused. I'm getting stuck. I'm getting confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting tangled. I'm not sure. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not certain. I'm confused. I'm getting stuck. I'm getting tangled. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm not sure. I'm confused. I'm getting confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not certain. I'm confused. I'm getting tangled. I'm confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm not sure. I'm confused. I'm getting stuck. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not certain. I'm confused. I'm getting stuck. I'm getting tangled. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm not sure. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm not sure. I'm confused. I'm getting stuck. I'm getting tangled. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm not certain. I'm confused. I'm getting confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm not sure. I'm confused. I'm getting tangled. I'm confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not certain. I'm getting confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm getting confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not sure. I'm confused. I'm getting confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm not certain. I'm confused. I'm getting stuck. I'm getting confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm getting confused. I'm confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm getting confused. I'm confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm getting confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting confused. I'm confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm getting confused. I'm confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting confused. I'm confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm getting confused. I'm confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm getting stuck. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm getting confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting confused. I'm getting stuck. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm getting confused. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting confused. I'm getting stuck. I'm confused. I'm confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting confused. I'm getting stuck. I'm confused. I think the bullet stops and energy is dissipated. Or perhaps the bullet stops and energy is lost. I'm confused. I'm getting stuck. I'm confused. I'm getting confused. I think the bullet stops and energy is dissipated. Or maybe the bullet stops and energy is lost. I'm getting confused. I'm confused. I'm getting stuck. I'm confused. I think the bullet stops and^CTraceback (most recent call last):

Description

• I am trying to use HuggingFaceEndpoint and ChatHuggingFace to generate a simple message.
• Applying a max_new_tokens value does not seem to work. Indeed, the output from the model continues for much more than the "100" new_tokens that I have asked for.

System Info

(.venv) ~/github/gemma git:[master]
python -m langchain_core.sys_info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 24.5.0: Tue Apr 22 19:53:27 PDT 2025; root:xnu-11417.121.6~2/RELEASE_ARM64_T6041
Python Version: 3.12.4 (main, Jan 28 2025, 14:24:59) [Clang 16.0.0 (clang-1600.0.26.6)]

Package Information

langchain_core: 0.3.72
langsmith: 0.4.8
langchain_huggingface: 0.3.1

Optional packages not installed

langserve

Other Dependencies

httpx: 0.28.1
huggingface-hub>=0.33.4: Installed. No version info available.
jsonpatch<2.0,>=1.33: Installed. No version info available.
langchain-core<1.0.0,>=0.3.70: Installed. No version info available.
langsmith-pyo3: Installed. No version info available.
langsmith>=0.3.45: Installed. No version info available.
openai-agents: Installed. No version info available.
opentelemetry-api: Installed. No version info available.
opentelemetry-exporter-otlp-proto-http: Installed. No version info available.
opentelemetry-sdk: Installed. No version info available.
orjson: 3.11.0
packaging: 25.0
packaging>=23.2: Installed. No version info available.
pydantic: 2.11.7
pydantic>=2.7.4: Installed. No version info available.
pytest: Installed. No version info available.
PyYAML>=5.3: Installed. No version info available.
requests: 2.32.4
requests-toolbelt: 1.0.0
rich: Installed. No version info available.
sentence-transformers>=2.6.0;: Installed. No version info available.
tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
tokenizers>=0.19.1: Installed. No version info available.
transformers>=4.39.0;: Installed. No version info available.
typing-extensions>=4.7: Installed. No version info available.
zstandard: 0.23.0

[azibek]

Seems like the params defined in HuggingFaceEndpoint are not passed on to ChatHuggingFace.stream() method.

Explicitly adding max_tokens to stream() does limit the output tokens.

for chunk in chat_model.stream(messages, max_tokens=100):
        print(chunk.content, end="", flush=True)

I can take a deeper dive and do a proper fix.