Skip to content

Add comprehensive OpenAI TTS models tracking support #2836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Sahelisaha04
Copy link

Added comprehensive tracking and cost calculation for OpenAI TTS models (tts-1, tts-1-hd).

What Changed

  • Character-based billing: TTS models charge per character, not tokens - now we track that correctly
  • Automatic detection: Works with existing track_openai() - no code changes needed
  • Real costs: Shows actual OpenAI pricing ($0.000015/char for tts-1, $0.000030/char for tts-1-hd)
  • Full metadata capture: Tracks model, voice, response format, speed, and all API parameters

API Coverage

  • client.audio.speech.create() - Regular TTS calls
  • client.audio.speech.with_streaming_response.create() - Streaming TTS calls
  • All existing OpenAI tracking setups continue working

Technical Implementation

Backend Changes:

  • Extended ModelPrice with inputCharacterPrice field
  • Added character-based cost calculation in SpanCostCalculator
  • Auto-detection of billing type (token vs character) in CostService
  • Support for input_characters usage tracking

Frontend Updates:

  • Extended UsageData interface for character usage
  • Updated trace helpers to display character metrics
  • Backward compatible with existing token displays

Python SDK:

  • New OpenaiAudioSpeechTrackDecorator for TTS tracking
  • OpenaiAudioSpeechUsage class for character-based usage
  • Updated OpikUsageFactory to support TTS models
  • Seamless integration with existing OpenAI tracking

Usage Example

import openai
from opik.integrations.openai import track_openai

client = track_openai(openai.OpenAI(), project_name="tts-demo")

# Automatically tracked with character usage and costs
response = client.audio.speech.create(
    model="tts-1",
    voice="alloy",
    input="This text will be tracked with accurate character-based billing."
)

Testing & Validation

22 comprehensive tests added

  • 9 integration tests (end-to-end TTS tracking)
  • 13 unit tests (character calculation, edge cases)

Manual testing completed

  • Real OpenAI API call validation
  • Cost calculation accuracy verified
  • Both regular and streaming API tested

Zero breaking changes

  • Fully backward compatible
  • Existing projects work unchanged
  • Optional character usage fields

Performance & Security

  • No performance impact: Leverages existing infrastructure
  • Minimal overhead: Simple character counting
  • Secure: Uses existing API tracking patterns
  • No new dependencies: Built on current architecture

/claim #2202
fixes #2202

- Implement character-based usage tracking for OpenAI TTS models (tts-1, tts-1-hd)
- Add OpenaiAudioSpeechTrackDecorator for audio.speech.create() and with_streaming_response.create()
- Create OpenAIAudioSpeechUsage class for character-based billing calculations
- Update backend ModelPrice and CostService to support input_cost_per_character pricing
- Add characterBasedCost calculation method to SpanCostCalculator
- Enhance frontend to display input_characters usage alongside token metrics
- Include comprehensive test coverage (9 integration + 13 unit tests)
- Add demo script showing TTS tracking capabilities
- Support both regular and streaming TTS API calls
- Track metadata: voice, response_format, speed, audio_size
- Map character count to prompt_tokens for UI consistency
- Integrate with existing cost estimation and display systems

Resolves comet-ml#2202 - Support OpenAI TTS models tracking
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FR]: Support Openai TTS models tracking
1 participant