[router] support Openai router conversation API CRUD

[key4ng]

Motivation

Basic support for conversation API CRUD

Modifications

Tests

## CREATE CONVERSATION

ubuntu@moirai-a10-exp:~/sglang$  curl http://localhost:30000/v1/conversations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "metadata": {"topic": "demo"}
  }'
{"id":"conv_ad518b1f7eee8ce468023661f1529422cd1292da1f51fb45","object":"conversation","created_at":1759852825,"metadata":{"topic":"demo"}}

## GET CONVERSATION
ubuntu@moirai-a10-exp:~/sglang$   curl http://localhost  curl http://localhost:30000/v1/conversations/conv_ad518b1f7eee8ce468023661f1529422cd1292da1f51fb45 \
  -H "Authorization: Bearer $OPENAI_API_KEY"
{"id":"conv_ad518b1f7eee8ce468023661f1529422cd1292da1f51fb45","object":"conversation","created_at":1759852825,"metadata":{"topic":"demo"}}

## UPDATE CONVERSATION -1: add and update metadata
ubuntu@moirai-a10-exp:~/sglang$ curl  http://localhost:30000/v1/conveubuntu@moirai-a10-exp:~/sglang$ curl  http://localhost:30000/v1/conversations/conv_ad518b1f7eee8ce468023661f1529422cd1292da1f51fb45 \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "metadata": {"topic": "demo_updated", "topic2": "yyds"}
  }'

## UPDATE CONVERSATION - 2: remove metadata
{"id":"conv_ad518b1f7eee8ce468023661f1529422cd1292da1f51fb45","object":"conversation","created_at":1759852825,"metadata":{"topic":"demo_updated","topic2":"yyds"}ubuntu@moirai-a10-exp:~/sglang$ curl  http://localhost:30000/v1/conversations/conv_ad518b1f7eee8ce468023661f1529422cd1292da1f51fb45 \\
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "metadata": {"topic2": null}
  }'
{"id":"conv_ad518b1f7eee8ce468023661f1529422cd1292da1f51fb45","object":"conversation","created_at":1759852825,"metadata":{"topic":"demo_updated"}}

## DELETE CONVERSATION
ubuntu@moirai-a10-exp:~/sglang$   curl -X DELET  curl -X DELETE http://localhost:30000/v1/conversations/conv_ad518b1f7eee8ce468023661f1529422cd1292da1f51fb45 \
  -H "Authorization: Bearer $OPENAI_API_KEY"
{"id":"conv_ad518b1f7eee8ce468023661f1529422cd1292da1f51fb45","object":"conversation.deleted","deleted":true}

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[key4ng]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces CRUD functionality for conversations, including a ConversationStorage trait and several implementations (in-memory, no-op, and Oracle). It also exposes these operations via new API endpoints. The implementation is well-structured, particularly the use of spawn_blocking for synchronous database calls and the clear separation of concerns in the storage modules. I've provided a few suggestions to improve performance in the Oracle store, adhere more closely to Rust idioms, and enhance code readability in the server routing.

[key4ng]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces comprehensive CRUD (Create, Read, Update, Delete) functionality for conversations. It adds new storage backends, including in-memory, no-op, and a detailed Oracle implementation, along with the necessary data structures and API endpoints. The changes are well-structured. My review focuses on improving performance by reducing unnecessary data cloning in the Oracle store, fixing a compilation error in the conversation ID generation logic, and suggesting a more efficient and idiomatic approach to JSON serialization in the server handlers. Overall, this is a solid addition of a major feature.

[slin1237]

should conversation handlers happen in router?
instead of server?

[key4ng]

yeah. my bad, resolved

[slin1237]

please add a todo
these spec validation should be elsewhere

[key4ng]

sure added

[key4ng]

^^ @slin1237 ci completed, ready to review