[exporter][queuebatch] Support merging and splitting context with batching

[sfc-gh-sili]

When batching, we merge multiple requests into one, so we need a solution to merge the corresponding contexts as well.

Jul 22, 2025: see updated proposals in #13320 (comment).

---

API proposal:
Different use cases require different strategies for merging these context fields in the resulting batched request. To address this need, the proposed API allows for individual control over each field:
Supported policies
* keepfirst: use the context value from the first request in the batch
* keeplast: use the context value from the last request in the batch
* keepall: always convert the field into an array and concat all value
* keepmax: choose the greatest. This is only supported for numeric types. Behavior is undefined if not numeric type
* keepmin: choose the least value. This is only supported for numeric types. Behavior is undefined if not numeric type

If no policy is specified for a field, the keepfirst strategy is used by default.

Example. Let assume that the incoming request comes in with a context carry a timestamp

req1 = [item1, item2, item3] ctx1 = {timestamp:1751501111}
req2 = [item4, item5, item6] ctx2 = {timestamp:1751502222}
req3 = [item7, item8, item9] ctx3 = {timestamp:1751503333}

Merge policy 1: Keep first

Config:

batch:
    context:
        timestamp: keepfirst

Merged requests:

mergedRequest1 = [item1, item2, item3, item4, item5] mergedCtx1 = {timestamp:1751501111}
mergedRequest2 = [item6, item7, item8, item9] mergedCtx2 = {timestamp:1751502222}

Merge policy 2: Keep last

Config:

batch:
    context:
        timestamp: keeplast

Merged requests:

mergedRequest1 = [item1, item2, item3, item4, item5] mergedCtx1 = {timestamp:1751502222}
mergedRequest2 = [item6, item7, item8, item9] mergedCtx2 = {timestamp:1751503333}

Merge policy 3: Keep all

Config:

batch:
    context:
        timestamp: keepall

Merged requests:

mergedRequest1 = [item1, item2, item3, item4, item5] mergedCtx1 = {timestamp:[1751501111, 1751502222]}
mergedRequest2 = [item6, item7, item8, item9] mergedCtx2 = {timestamp:[1751502222, 1751503333]}

Merge policy 4: Keep max / min

Config:

batch:
    context:
        timestamp: keepmax

Merged requests:

mergedRequest1 = [item1, item2, item3, item4, item5] mergedCtx1 = {timestamp:1751502222}
mergedRequest2 = [item6, item7, item8, item9] mergedCtx2 = {timestamp:1751503333}

Error handling: not-number fields will be dropped

---

Out of scope: preserving context across persistent storage is not yet supported. If persistent queue is used with batching, context will be lost.

[sfc-gh-sili]

cc: @sfc-gh-bnandibhatla

[dmitryax]

Can you please provide more details about the mentioned use cases? I don't understand what is meant by the context's value and timestamp in this issue.

[sfc-gh-sili]

@dmitryax We are looking to read metadata from extensions (e.g. an extension that intercepts http requests and modifies the headers), so we'd like this metadata to be put in context rather than resourceAttributes.

For instance, suppose we want to

• extract databaseID from context to add to the HTTP request header
• and record when the request was processed (timestamp) for tracing

type ContextKey string
const (
    dataBaseIDStr = "databaseID"
    minTimestampStr = "minTimestamp"
)

ctx = context.WithValue(ctx, ContextKey(dataBaseIDStr), databaseId)
ctx = context.WithValue(ctx, ContextKey(minTimestampStr), time.Now())

If the intercepted HTTP request (httpRequest, ctx) is aggregated from two exporter requests

Request 1:

req1 = {item1, item2, item3}
ctx1 = {databaseID: "this-is-database-id", minTimestamp: 1751501111}

Request 2:

req2 = {item4, item5, item6}
ctx2 = {databaseID: "this-is-database-id", minTimestamp: 1751502222}

We'd want the ctx to

• contain either of ctx1.databaseID or ctx2.databaseID, and the first one encountered would just be fine
• and the less of ctx1.minTimestamp and ctx2.minTimestamp

---

Currently, our context merging implementation is more focused on tracing purpose and does not retain the original contents of the incoming context.

I am thinking that preserving the first value by default while providing a knob to control the merge policy for individual context field could be a good approach. Let me know your thoughts

[dmitryax]

I am thinking that preserving the first value by default while providing a knob to control the merge policy for individual context field could be a good approach. Let me know your thoughts

Right. Preserving the first value by default makes sense — we can implement and document that behavior now.

However, customized merging logic can vary case by case. A single policy config option won’t be flexible enough. I think we should provide a way to define context merging logic using OTTL instead.

[sfc-gh-sili]

After exploring a few options, I think the most solid and practical path forward is:

✅ @jade-guiton-dd's approach described in this comment It prides users an programmatical API to specify context merging, granting complete control over how each key is handled. I am currently working on a draft that follows this direction and will share it shortly. Many thanks to Jade for contributing this idea!

✅ I also liked @lahsivjar proposal in this comment. While not as flexible as the above, it allows us to move forward without having to define what "context merging" means. The downside is that users don't get access to values that can not be parsed from partition key (an example I can think of would be timestamp)

Other options considered:

❌ Treating context as a dictionary—just iterating over all the keys to merge them—initially seemed like an easy fix. The problem is Golang's context.Context doesn't have an API to enumerate all key-value pairs (at least, not without using unsafe reflection, which is strongly discouraged). Moreover, Context keys are usually private constants to a library instead of a primitive type that can be specified from yaml config. The following code snippet would demonstrate this

type contextKey1 string
type contextKey2 string

// The following keys are different
val = ctx.Value(contextKey1("key"))
val = ctx.Value(contextKey2("key"))

❓ Another idea was to base the merged context on the current context instead of context.Background (in contextWithMergedLinks). This approach can break assumptions about how span links are managed. (experimented in #13416)

[lahsivjar]

Thanks @sfc-gh-sili for tackling this topic. It is a critical topic for us to start using the batcher with partitioner support. One point we can establish is that doing arbitrary merge operations or choosing a random context is not the way we want to pursue because of all the points mentioned in this and this comment.

Between the two approaches, I am more inclined towards @jade-guiton-dd 's proposal in this comment as it is more flexible and robust. While context merging is still a hard problem for some scenarios, the user has the flexibility to avoid this and get to the same point that I proposed with the encoding of keys in the partition key approach by defining the Merger function appropriately.