Skip to content

handle parallel function calls from gemini #406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 9, 2024
Merged

handle parallel function calls from gemini #406

merged 2 commits into from
May 9, 2024

Conversation

vandyxiaowei
Copy link
Contributor

@vandyxiaowei vandyxiaowei commented May 3, 2024 •
edited by ShishirPatil
Loading

Handle parallel function calls for Gemini handler for the Berkeley Function Calling Leaderboard.

This PR does NOT change values in BFCL.

HuanzhiMao
HuanzhiMao approved these changes May 7, 2024
View reviewed changes
Copy link
Collaborator

@HuanzhiMao HuanzhiMao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing @vandyxiaowei! LGTM

@ShishirPatil ShishirPatil merged commit 42f9d28 into ShishirPatil:main May 9, 2024
@HuanzhiMao HuanzhiMao mentioned this pull request May 9, 2024
Merged
ShishirPatil pushed a commit that referenced this pull request May 15, 2024
@HuanzhiMao
[BFCL] Patch Gemini Handler (#421)
40b07a5 
Build on the work of @vandyxiaowei #406, this PR is a bug fix for the
generation pipeline for Gemini models ([Gemini-1.5-Pro
(FC)](https://deepmind.google/technologies/gemini/#introduction) and
[Gemini-1.0-Pro
(FC)](https://deepmind.google/technologies/gemini/#introduction)) to
make it more robust. The Gemini model output `result["candidates"][0]`
does not always have the key `"content"`, which would cause the current
generation pipeline to error out.

This PR **DOES NOT** change the leaderboard score.
@HuanzhiMao HuanzhiMao mentioned this pull request May 15, 2024
Merged
ShishirPatil pushed a commit that referenced this pull request May 15, 2024
@HuanzhiMao
Leaderboard Update, in sync with PR#406 (handle parallel function cal…
e04e9ed 
…ls from gemini) (#427)

This PR updates the leaderboard to reflect the change in score due to
the added support for parallel function calls for gemini series in #406.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ShishirPatil ShishirPatil ShishirPatil approved these changes

@HuanzhiMao HuanzhiMao HuanzhiMao approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vandyxiaowei @ShishirPatil @HuanzhiMao