add support for tool calls and responses (gemini)

[GustavoWidman]

initial work, had to hotwire the call_id to the fn_name because gemini doesn't work based on call_id. currently passing tests, but i had to change the question to "What is the temperature in C, in Paris, France" instead of "What is the temperature in C, in Paris" because it wasn't using the tool and asking what the country was.

[jeremychone]

This is great work. I was planning to do it today, so this will be my base.

Yes, Flash is fast and relatively good, but it likes to ask questions. I guess there are multiple Parises, to be technically correct.

[GustavoWidman]

yea lol, i'd finished implementing the tool and the test wasn't passing but it wasn't because the model couldn't call the tool, but because it was asking what country "Paris" was in before using the tool 🤦

Yes, Flash is fast and relatively good, but it likes to ask questions. I guess there are multiple Parises, to be technically correct.

[jeremychone]

@GustavoWidman That's interesting. I have been using Flash 2 quite a bit, and it does like to ask questions, and sometimes be a smarty pants if you ask the same thing twice (and it has it in context, obviously). o3-mini is a coding beast, but Flash is so fast. But for tool calls, Flash is probably a great option for many use cases.

Anyway, I am planning to work on this today.

[jeremychone]

@GustavoWidman Very good PR, by the way. Very clean and focused. I am going to merge it.

Question: if Gemini does not have a call_id, do you know how they "relate" the tool call and tool response?

[jeremychone]

Ok, @GustavoWidman merge. Thanks a lot!!!

[GustavoWidman]

@GustavoWidman Very good PR, by the way. Very clean and focused. I am going to merge it.

thanks

Question: if Gemini does not have a call_id, do you know how they "relate" the tool call and tool response?

from what i've managed to understand, gemini makes sure there is an equal amount of toolcalls and toolresponses, and matches toolcall with toolresponse based on how close they are to each other (literally proximity inside the messages vector) and if they are of the same tool

single turn tool call request and response example ("simple"): https://ai.google.dev/gemini-api/docs/function-calling#expandable-3 and https://ai.google.dev/gemini-api/docs/function-calling#expandable-2
muli-turn tool call request and response example ("full flow"): https://ai.google.dev/gemini-api/docs/function-calling#expandable-7 and https://ai.google.dev/gemini-api/docs/function-calling#expandable-8

there is still room to implement the tool_config options (force the model to use a tool, etc etc), see https://ai.google.dev/gemini-api/docs/function-calling#single-turn-any-mode

[jeremychone]

@GustavoWidman Btw, I think you are right about the potential "proximity tactic."

In fact, I have been questioning if that was what the other providers were doing with their call_ids scheme before sending it to the LLM. I am not an LLM expert, but it seems to be a waste of "tuning capital" to fine-tune a model to go and fish for tool call_ids. But there might be other benefits, perhaps.