Skip to content

⚡ feat: Gemini-1.5 Flash, gpt-4o imports, modelSpec greeting fix #2729

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 15, 2024
Merged

⚡ feat: Gemini-1.5 Flash, gpt-4o imports, modelSpec greeting fix #2729

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c0cb34c
fix: Gemini Flash stream fix
danny-avila May 15, 2024
45ab376
fix: correct `sender` field for gpt-4o imports from ChatGPT
danny-avila May 15, 2024
39d4193
add flash model examples and fix vertex streaming
danny-avila May 15, 2024
3c80e58
style: modelSpec greeting fix
danny-avila May 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,10 +116,10 @@ GOOGLE_KEY=user_provided
# GOOGLE_REVERSE_PROXY=

# Gemini API
# GOOGLE_MODELS=gemini-1.0-pro,gemini-1.0-pro-001,gemini-1.0-pro-latest,gemini-1.0-pro-vision-latest,gemini-1.5-pro-latest,gemini-pro,gemini-pro-vision
# GOOGLE_MODELS=gemini-1.5-flash-latest,gemini-1.0-pro,gemini-1.0-pro-001,gemini-1.0-pro-latest,gemini-1.0-pro-vision-latest,gemini-1.5-pro-latest,gemini-pro,gemini-pro-vision

# Vertex AI
# GOOGLE_MODELS=gemini-1.5-pro-preview-0409,gemini-1.0-pro-vision-001,gemini-pro,gemini-pro-vision,chat-bison,chat-bison-32k,codechat-bison,codechat-bison-32k,text-bison,text-bison-32k,text-unicorn,code-gecko,code-bison,code-bison-32k
# GOOGLE_MODELS=gemini-1.5-flash-preview-0514,gemini-1.5-pro-preview-0409,gemini-1.0-pro-vision-001,gemini-pro,gemini-pro-vision,chat-bison,chat-bison-32k,codechat-bison,codechat-bison-32k,text-bison,text-bison-32k,text-unicorn,code-gecko,code-bison,code-bison-32k

# Google Gemini Safety Settings
# NOTE (Vertex AI): You do not have access to the BLOCK_NONE setting by default.
Expand Down
13 changes: 9 additions & 4 deletions api/app/clients/GoogleClient.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -683,11 +683,12 @@ class GoogleClient extends BaseClient {
const safetySettings = _payload.safetySettings;
requestOptions.safetySettings = safetySettings;

const delay = modelName.includes('flash') ? 8 : 14;
const result = await client.generateContentStream(requestOptions);
for await (const chunk of result.stream) {
const chunkText = chunk.text();
this.generateTextStream(chunkText, onProgress, {
delay: 12,
await this.generateTextStream(chunkText, onProgress, {
delay,
});
reply += chunkText;
}
Expand All @@ -701,10 +702,14 @@ class GoogleClient extends BaseClient {
safetySettings: safetySettings,
});

let delay = this.isGenerativeModel ? 12 : 8;
if (modelName.includes('flash')) {
delay = 5;
}
for await (const chunk of stream) {
const chunkText = chunk?.content ?? chunk;
this.generateTextStream(chunkText, onProgress, {
delay: this.isGenerativeModel ? 12 : 8,
await this.generateTextStream(chunkText, onProgress, {
delay,
});
reply += chunkText;
}
Expand Down
2 changes: 1 addition & 1 deletion api/server/utils/import/importers.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ function processConversation(conv, importBatchBuilder, requestUserId) {
const isCreatedByUser = role === 'user';
let sender = isCreatedByUser ? 'user' : 'GPT-3.5';
const model = mapping.message.metadata.model_slug || openAISettings.model.default;
if (model === 'gpt-4') {
if (model.includes('gpt-4')) {
sender = 'GPT-4';
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Technically", it's GPT-4 and GPT-4o 😅 (ofc I haven't looked up what model looks like)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this works for when the model is vanilla "gpt-4" as well as "gpt-4o"

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you mean about differentiating the sender, I'll think about that soon, as that would mean a few other changes app-wide

Copy link

@stdedos stdedos May 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, tbh, I would'be just slapped all gpt-4-1106-preview strings into a huge lookup table (and hide the actual model name e.g. under a hover on the username).

At least for the OpenAI models, that go like

  • gpt-4o-2024-05-13
  • gpt-4-1106-preview
  • gpt-3.5-turbo

I'd try e.g.

function extractUsername(modelName) {
    // Check for specific model name patterns and return corresponding usernames
    if (modelName.startsWith('gpt-4o-')) {
        return 'GPT-4o';
    } else if (modelName.startsWith('gpt-4-')) {
        return 'GPT-4';
    } else if (modelName.startsWith('gpt-3.5-turbo')) {
        return 'GPT-3';
    }

    ...
}

and/or something "smarter" (which ofc you can support with tests)

Ofc, up to you, the project, and your time 🙏 (and things I haven't seen, since I haven't really dug the source)

}

Expand Down
2 changes: 1 addition & 1 deletion client/src/components/Chat/Landing.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ export default function Landing({ Header }: { Header?: ReactNode }) {
</div> */}
</div>
) : (
<div className="mb-5 text-2xl font-medium dark:text-white">
<div className="mb-5 max-w-[75vh] px-12 text-center text-lg font-medium dark:text-white md:px-0 md:text-2xl">
{endpoint === EModelEndpoint.assistants
? conversation?.greeting ?? localize('com_nav_welcome_assistant')
: conversation?.greeting ?? localize('com_nav_welcome_message')}
Expand Down