(logs): Query strings may multiple stats commands in a single query

[moltar]

Describe the bug

Docs: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax-Stats.html#CWL_QuerySyntax-stats-multi

And the props only support one:

aws-cdk/packages/aws-cdk-lib/aws-logs/lib/query-definition.ts

Lines 57 to 62 in d8e6c09

	/**
	* Uses log field values to calculate aggregate statistics.
	*
	* @default - no stats in QueryString
	*/
	readonly stats?: string;

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Library Version

latest

Expected Behavior

Support 2 stats.

You can use as many as two stats commands in a single query. This enables you to perform an additional aggregation on the output of the first aggregation.

Current Behavior

Supports 1 stats.

Reproduction Steps

Look at the code.

Possible Solution

Add support for 2 stats.

Additional Information/Context

No response

AWS CDK Library version (aws-cdk-lib)

N/A

AWS CDK CLI version

N/A

Node.js Version

N/A

OS

N/A

Language

TypeScript

Language Version

N/A

Other information

No response

[ykethan]

Hey @moltar, Thank you for reporting this issue! I've confirmed this is a valid limitation in the AWS CDK implementation of CloudWatch Logs Insights QueryDefinition. According to the CloudWatch Logs Insights documentation, you can use up to two stats commands in a single query to perform an additional aggregation on the output of the first aggregation:

aws-cdk/packages/aws-cdk-lib/aws-logs/lib/query-definition.ts

Lines 57 to 62 in d8e6c09

	/**
	* Uses log field values to calculate aggregate statistics.
	*
	* @default - no stats in QueryString
	*/
	readonly stats?: string;

Fix suggestion:

We could try and update the QueryStringProps interface and QueryString class to support multiple stats statements, similar to how parse and filter are already handled with parseStatements and filterStatements arrays:

// In QueryStringProps interface
/**
 * A single statement for using log field values to calculate aggregate statistics.
 *
 * @deprecated Use `statsStatements` instead
 * @default - no stats in QueryString
 */
readonly stats?: string;

/**
 * An array of one or more statements for calculating aggregate statistics. 
 * CloudWatch Logs Insights supports up to two stats commands in a single query.
 * Each provided statement generates a separate stats line in the query string.
 *
 * Note: If provided, this property overrides any value provided for the `stats` property.
 *
 * @default - no stats in QueryString
 */
readonly statsStatements?: string[];

Then, the implementation in the QueryString class would need to be updated to handle multiple stats statements:

// In QueryString class constructor
if (props.statsStatements) {
  this.stats = props.statsStatements;
} else if (props.stats) {
  this.stats = [props.stats];
} else {
  this.stats = [];
}

// Change stats property from string to string[]
private readonly stats: string[];

// Update toString() method to use buildQueryLines instead of buildQueryLine for stats
...buildQueryLines('stats', this.stats),

Workaround

Until this feature is implemented, you can work around this limitation by directly including both stats commands in a single string:

new QueryString({
  // Other properties...
  stats: "sum(message_length)/1024/1024 as logs_mb BY bin(5m) | stats max(logs_mb) AS peak_ingest_mb, min(logs_mb) AS min_ingest_mb, avg(logs_mb) AS avg_ingest_mb"
})

Marking this as P2 bug.
We greatly appreciate community contributions! If you are interested in implementing this, do review our contribution guidelines for additional information on opening a pull request. Meanwhile, do add 👍 to help us prioritize.

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.