Skip to content

Conversation

michaeldwan
Copy link
Member

@michaeldwan michaeldwan commented Sep 8, 2025

Turns out I can't remove cog-runtime support entirely. We're still moving towards porting cog-runtime back into cog as the default, but that's not ready yet. So what we'll do until then is:

  1. honor the cog_runtime setting in cog.yaml so models that have opted in can continue to use it
  2. pin coglet (the python model library in cog-runtime) to a version that predates the concurrency and isolation work in cog-runtime (the go server/runner portion.) That stuff is unfortunately necessary to complete, but we're not comfy exposing existing models to it until we port coglet back to cog and its large test suite. This applies to both cog_runtime: true and fast: true models!
  3. stop showing "major upgrade" warnings or opt-in prompts to ensure we don't encourage more usage until we're confident everything works and have integration tests verifying it keeps working

@michaeldwan michaeldwan force-pushed the restore-coglet-with-pinning branch from 99ba328 to ff7d16d Compare September 8, 2025 20:03
@michaeldwan michaeldwan marked this pull request as ready for review September 8, 2025 21:02
Copy link
Member

@tempusfrangit tempusfrangit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will also want to ensure we are adhering to the right version max in monobase.

@michaeldwan
Copy link
Member Author

@tempusfrangit good catch! I think I got it set in the only place that coglet is referenced. Based on this #2349 it doesn't look like anything is using R8_COGLET_RUNTIME, and based on this https://github.com/replicate/monobase/blob/5f8525f38ca916cfef0f55eb9270ab874a1c388b/README.md?plain=1#L88 it seems like setting R8_COG_RUNTIME to a url should work correctly.

@michaeldwan michaeldwan merged commit 08cef1a into main Sep 9, 2025
26 checks passed
@michaeldwan michaeldwan deleted the restore-coglet-with-pinning branch September 9, 2025 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants