Skip to content

Add code to log completions as OpenPipe request logs #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 26, 2025

Conversation

arcticfly
Copy link
Contributor

@arcticfly arcticfly commented Mar 26, 2025

This PR adds an example of logging completions to OpenPipe.

Code:

    requested_at = int(time.time() * 1000)
    chat_completion = await client.chat.completions.create(
        messages=messages, model=model.name
    )

    puzzle_key = generate_puzzle_key(puzzle)
    print(f"puzzle_key: {puzzle_key}")
    op_client.report(
        requested_at=requested_at,
        received_at=int(time.time() * 1000),
        req_payload={
            "model": model.name,
            "messages": messages,
            "metadata": {
                "notebook-id": "temporal-clue",
                "iteration": str(iteration),
                "validation": str(is_validation),
                "puzzle_key": puzzle_key,
            },
        },
        resp_payload=chat_completion,
        status_code=200,
    )

Logs in OpenPipe:
Screenshot 2025-03-26 at 2 31 47 AM

"unsloth ; sys_platform == 'linux'",
"unsloth-zoo ; sys_platform == 'linux'",
"vllm>=0.7.0",
"wandb>=0.19.8",
"peft>=0.14.0",
"openpipe>=4.49.0",
"torchtune==0.5.0",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bradhilton Not sure if it matters whether we use 0.5.0 or 0.6.0 which was released today, but thought it was best to specify to be safe.

@@ -1,35 +1,23 @@
version = 1
revision = 1
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bradhilton Which version of uv are you using? I'm on 0.6.10, but I can standardize to yours to avoid rewriting the lockfile on each installation!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just upgraded my uv install, we should be good

"\n",
" puzzle_key = generate_puzzle_key(puzzle)\n",
" print(f\"puzzle_key: {puzzle_key}\")\n",
" op_client.report(\n",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic for reporting completions to OpenPipe

@arcticfly arcticfly changed the title [WIP] Add code to log completions as OpenPipe request logs Add code to log completions as OpenPipe request logs Mar 26, 2025
@arcticfly arcticfly requested a review from bradhilton March 26, 2025 09:31
@bradhilton bradhilton merged commit 3d60472 into main Mar 26, 2025
@bradhilton bradhilton deleted the add-openpipe branch March 26, 2025 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants