[Initiative]: Observability Query Language Standardization Specification

[manolama]

Name

Observability Query Language Standardization Specification

Short description

Define a set of models, querying semantics and operators to analyze observability telemetry data.

Responsible group

TAG Operational Resilience

Does the initiative belong to a subproject?

No

Subproject name

No response

Primary contact

Chris Larsen

Additional contacts

@alolita
@halcyondude
@vjsamuel

Initiative description

TAG OBS Query Standardization WG Charter

Summary

With the success of CNCF’s OpenTelemetry, observability instrumentation and collection has been standardized across the industry. Adopters are able instrument metrics, logs, traces and profiles in their systems one time and experiment with various observability backends or vendors by simply reconfiguring OTel collectors.

Unfortunately, every backend and vendor provides a different way to query and process the data. A plethora of domain specific languages (DSLs) exists with different telemetry models, semantic definitions and analytical capabilities. While pointing OTel at a new vendor is simple, rewriting dashboards, alerts and queries is not.

The goal of this working group is to recommend a set of standard models and query semantics for observability data. Such an open standard would unlock simple migrations using automated tooling, vendor solutions on top of customer provided data lakes, integration with business data, lower operational burden for observability engineers and reduce the cognitive load of observability data consumers.

Completed Work

1. Completed a set of DSL designer interviews, documented in GitHub. Languages include PromQL, TraceQL, DataDog QL, NRQL, KX’s Q, PPL.
2. Recorded and published presentations from DSL designers including PPL, KX Q, PromQL, NRQL, UQL, TraceQL, TScript, etc.
3. Documented a set of telemetry models in GitHub.
4. Documented a small set of observability use cases in GitHub.
5. Query language features rubric.
6. Draft semantic specification.
7. Draft context document surrounding the semantic specification.

Milestones (target H2 `26)

Work that is currently underway via biweekly meetings with end users and representatives from hyperscalers and observability vendors reviewing the standards. (videos):

1. Complete reviewing and commenting on the draft semantic specification.
2. Define and review standard observability specific operators

Follow Up/Concurrent Initiatives

The following are initiatives building on the deliverable of this work:

1. Reference implementation of an observability query federation and proxy project
2. Standard syntax recommendation
3. Blog posts about the spec
4. Maintenance of the standard

Resources

• CNCF Slack Channel: #obs-query-standard-wg
• CNCF Tag Observability Github Repo
• CNCF Observability Query Standard WG Meeting Notes
• TAG OBS Query Standardization WG Charter
• CNCF Observability Query Language Feature Rubric
• [CNCF DRAFT QLSWG Semantics Specification Context](https://docs.google.com/document/d/1A_r7tpLVELIhF1g6xnkmF3RNkX5_XWrSkEU0gKXKIK0/edit?tab=t.0
• CNCF DRAFT QLSWG Semantics Specification

Deliverable(s) or exit criteria

1. The final deliverable is a published observability query language semantic specification for modeling and querying and analyzing data.
2. A Github repo with examples, context and interview results.