Skip to content

Conversation

drernie
Copy link
Member

@drernie drernie commented Aug 11, 2025

Description

Dynamically generate list of available models to select from in Qurator Dev Tools

Core Infrastructure Changes:

  • Added AWS Bedrock SDK dependencies (@aws-sdk/client-bedrock and @aws-sdk/client-bedrock-runtime) to support dynamic model querying
  • Created a new ListModels.ts utility that discovers available Bedrock models by region, filtering for TEXT input/output capable models and testing access permissions

UI/UX Enhancements:

  • Expanded the DevTools component with a model selection dialog that allows developers to browse and select from discovered models

@drernie drernie changed the base branch from master to qurator-more-devtools August 11, 2025 23:27
@drernie drernie requested a review from Copilot August 11, 2025 23:27
Copy link

codecov bot commented Aug 11, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.74%. Comparing base (58be9f2) to head (0e956ba).

Additional details and impacted files
@@                    Coverage Diff                     @@
##           qurator-more-devtools    #4469       +/-   ##
==========================================================
+ Coverage                  37.79%   91.74%   +53.95%     
==========================================================
  Files                        822      108      -714     
  Lines                      36311     9873    -26438     
  Branches                    5933        0     -5933     
==========================================================
- Hits                       13722     9058     -4664     
+ Misses                     21362      815    -20547     
+ Partials                    1227        0     -1227     
Flag Coverage Δ
api-python 91.41% <ø> (ø)
catalog ?
lambda 92.60% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

greptile-apps[bot]

This comment was marked as duplicate.

Copilot

This comment was marked as duplicate.

@drernie drernie self-assigned this Aug 11, 2025
@drernie drernie requested a review from nl0 August 11, 2025 23:41
@drernie
Copy link
Member Author

drernie commented Aug 11, 2025

Would you be open to using a popup of available models, rather than requiring users to enter them manually?

@nl0
Copy link
Member

nl0 commented Aug 12, 2025

Would you be open to using a popup of available models, rather than requiring users to enter them manually?

i'm not convinced this adds enough value.
this was designed as a rudimentary dev tool to test other models, and i don't feel like we should spend more effort evolving it, adding all this unnecessary complexity (new dependencies, calls, filtering, ui state).
also, the code in the PR does not even list the inference profile we use by default, since it only list the models, not the inference profiles.
is it that bad of a DX to just copypaste a model id from the docs?

@nl0
Copy link
Member

nl0 commented Aug 12, 2025

but if you do believe we should spend more effort on this, we should:

  1. add inference profiles to the list
  2. filter modalities in the request to avoid doing that in the response
  3. remove the strange hybrid CLI mode
  4. fix client instantiation (preferably use the same client we use in the rest of the code)
  5. clean up the PR from all the vibe-coding artifacts

@drernie
Copy link
Member Author

drernie commented Aug 14, 2025

is it that bad of a DX to just copypaste a model id from the docs?

It is not that bad if you:

  • have AWS Console Access (most users do not)
  • know where to look and find the Bedrock models
  • understand which are enabled and how to copy the ID

But if we can enable ordinary users to trivially experiment with any model their AWS Admin has enabled, that is a game changer.

@nl0 nl0 force-pushed the qurator-more-devtools branch from 87189d8 to d3c1c93 Compare August 22, 2025 13:04
Base automatically changed from qurator-more-devtools to master August 25, 2025 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants