Skip to content

Feat: add new alerting rule fields #2120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/resources/rule_group.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,9 @@ Optional:
- `exec_err_state` (String) Describes what state to enter when the rule's query is invalid and the rule cannot be executed. Options are OK, Error, KeepLast, and Alerting. Defaults to Alerting if not set.
- `for` (String) The amount of time for which the rule must be breached for the rule to be considered to be Firing. Before this time has elapsed, the rule is only considered to be Pending. Defaults to `0`.
- `is_paused` (Boolean) Sets whether the alert should be paused or not. Defaults to `false`.
- `keep_firing_for` (String) The amount of time for which the rule will considered to be Recovering after initially Firing. Before this time has elapsed, the rule will continue to fire once it's been triggered.
- `labels` (Map of String) Key-value pairs to attach to the alert rule that can be used in matching, grouping, and routing. Defaults to `map[]`.
- `missing_series_evals_to_resolve` (Number) The number of missing series evaluations that must occur before the rule is considered to be resolved.
- `no_data_state` (String) Describes what state to enter when the rule's query returns No Data. Options are OK, NoData, KeepLast, and Alerting. Defaults to NoData if not set.
- `notification_settings` (Block List, Max: 1) Notification settings for the rule. If specified, it overrides the notification policies. Available since Grafana 10.4, requires feature flag 'alertingSimplifiedRouting' to be enabled. (see [below for nested schema](#nestedblock--rule--notification_settings))
- `record` (Block List, Max: 1) Settings for a recording rule. Available since Grafana 11.2, requires feature flag 'grafanaManagedRecordingRules' to be enabled. (see [below for nested schema](#nestedblock--rule--record))
Expand Down
8 changes: 4 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ require (
github.com/grafana/fleet-management-api v1.0.0
github.com/grafana/grafana-app-sdk v0.35.2-0.20250408075831-c2a87bde0849
github.com/grafana/grafana-com-public-clients/go/gcom v0.0.0-20250214150112-a52892176c26
github.com/grafana/grafana-openapi-client-go v0.0.0-20241113095943-9cb2bbfeb8a3
github.com/grafana/grafana-openapi-client-go v0.0.0-20250422190251-124d73258c08
github.com/grafana/grafana/apps/dashboard v0.0.0-20250314125419-399df82f0b25
github.com/grafana/grafana/apps/playlist v0.0.0-20250314125419-399df82f0b25
github.com/grafana/grafana/pkg/apimachinery v0.0.0-20250314125419-399df82f0b25
Expand Down Expand Up @@ -83,12 +83,12 @@ require (
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/analysis v0.23.0 // indirect
github.com/go-openapi/errors v0.22.0 // indirect
github.com/go-openapi/errors v0.22.1 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/loads v0.22.0 // indirect
github.com/go-openapi/spec v0.21.0 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/go-openapi/swag v0.23.1 // indirect
github.com/go-openapi/validate v0.24.0 // indirect
github.com/goccy/go-json v0.10.4 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
Expand Down Expand Up @@ -126,7 +126,7 @@ require (
github.com/klauspost/compress v1.17.11 // indirect
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
github.com/magefile/mage v1.15.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mailru/easyjson v0.9.0 // indirect
github.com/mattetti/filebuffer v1.0.1 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
Expand Down
12 changes: 12 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ github.com/go-openapi/analysis v0.23.0 h1:aGday7OWupfMs+LbmLZG4k0MYXIANxcuBTYUC0
github.com/go-openapi/analysis v0.23.0/go.mod h1:9mz9ZWaSlV8TvjQHLl2mUW2PbZtemkE8yA5v22ohupo=
github.com/go-openapi/errors v0.22.0 h1:c4xY/OLxUBSTiepAg3j/MHuAv5mJhnf53LLMWFB+u/w=
github.com/go-openapi/errors v0.22.0/go.mod h1:J3DmZScxCDufmIMsdOuDHxJbdOGC0xtUynjIx092vXE=
github.com/go-openapi/errors v0.22.1 h1:kslMRRnK7NCb/CvR1q1VWuEQCEIsBGn5GgKD9e+HYhU=
github.com/go-openapi/errors v0.22.1/go.mod h1:+n/5UdIqdVnLIJ6Q9Se8HNGUXYaY6CN8ImWzfi/Gzp0=
github.com/go-openapi/jsonpointer v0.21.0 h1:YgdVicSA9vH5RiHs9TZW5oyafXZFc6+2Vc1rr/O9oNQ=
github.com/go-openapi/jsonpointer v0.21.0/go.mod h1:IUyH9l/+uyhIYQ/PXVA41Rexl+kOkAPDdXEYns6fzUY=
github.com/go-openapi/jsonreference v0.21.0 h1:Rs+Y7hSXT83Jacb7kFyjn4ijOuVGSvOdF2+tg1TRrwQ=
Expand All @@ -113,6 +115,8 @@ github.com/go-openapi/strfmt v0.23.0 h1:nlUS6BCqcnAk0pyhi9Y+kdDVZdZMHfEKQiS4HaMg
github.com/go-openapi/strfmt v0.23.0/go.mod h1:NrtIpfKtWIygRkKVsxh7XQMDQW5HKQl6S5ik2elW+K4=
github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE=
github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=
github.com/go-openapi/swag v0.23.1 h1:lpsStH0n2ittzTnbaSloVZLuB5+fvSY/+hnagBjSNZU=
github.com/go-openapi/swag v0.23.1/go.mod h1:STZs8TbRvEQQKUA+JZNAm3EWlgaOBGpyFDqQnDHMef0=
github.com/go-openapi/validate v0.24.0 h1:LdfDKwNbpB6Vn40xhTdNZAnfLECL81w+VX3BumrGD58=
github.com/go-openapi/validate v0.24.0/go.mod h1:iyeX1sEufmv3nPbBdX3ieNviWnOZaJ1+zquzJEf2BAQ=
github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
Expand Down Expand Up @@ -171,6 +175,12 @@ github.com/grafana/grafana-com-public-clients/go/gcom v0.0.0-20250214150112-a528
github.com/grafana/grafana-com-public-clients/go/gcom v0.0.0-20250214150112-a52892176c26/go.mod h1:sYWkB3NhyirQJfy3wtNQ29UYjoHbRlJlYhqN1jNsC5g=
github.com/grafana/grafana-openapi-client-go v0.0.0-20241113095943-9cb2bbfeb8a3 h1:poKxGlUaEYVp2DMofC/I2GHw/vvtHAZ20c48I8rFB6M=
github.com/grafana/grafana-openapi-client-go v0.0.0-20241113095943-9cb2bbfeb8a3/go.mod h1:hiZnMmXc9KXNUlvkV2BKFsiWuIFF/fF4wGgYWEjBitI=
github.com/grafana/grafana-openapi-client-go v0.0.0-20250108132429-8d7e1f158f65 h1:AnfwjPE8TXJO8CX0Q5PvtzGta9Ls3iRASWVV4jHl4KA=
github.com/grafana/grafana-openapi-client-go v0.0.0-20250108132429-8d7e1f158f65/go.mod h1:hiZnMmXc9KXNUlvkV2BKFsiWuIFF/fF4wGgYWEjBitI=
github.com/grafana/grafana-openapi-client-go v0.0.0-20250416180334-fd971649195e h1:XST++B/RBwahjIFM76H6b+rvgXKLnUgoxKq0lSCfNWo=
github.com/grafana/grafana-openapi-client-go v0.0.0-20250416180334-fd971649195e/go.mod h1:AOzHLStinAJHJmcih1eEbIRImxpT6enYUsZLnnOvhbo=
github.com/grafana/grafana-openapi-client-go v0.0.0-20250422190251-124d73258c08 h1:zaZHmWQvq3oSJMgNUxOomsEIuWS7EkEMjodgBQ522S4=
github.com/grafana/grafana-openapi-client-go v0.0.0-20250422190251-124d73258c08/go.mod h1:hiZnMmXc9KXNUlvkV2BKFsiWuIFF/fF4wGgYWEjBitI=
github.com/grafana/grafana-plugin-sdk-go v0.272.0 h1:TmPIG+6e3lYGzkyfUfCHuaMaaiwDbkCacTZ7V/JaSeg=
github.com/grafana/grafana-plugin-sdk-go v0.272.0/go.mod h1:i/9KH9y/6m5hkRnG3H6aR2nOMPbJUmvo4XNrHjI15cU=
github.com/grafana/grafana/apps/dashboard v0.0.0-20250314125419-399df82f0b25 h1:a/G2DPvR2dDemU4kAouzvWijvy22FsMr9etOA/0KSoI=
Expand Down Expand Up @@ -294,6 +304,8 @@ github.com/magefile/mage v1.15.0 h1:BvGheCMAsG3bWUDbZ8AyXXpCNwU9u5CB6sM+HNb9HYg=
github.com/magefile/mage v1.15.0/go.mod h1:z5UZb/iS3GoOSn0JgWuiw7dxlurVYTu+/jHXqQg881A=
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/mailru/easyjson v0.9.0 h1:PrnmzHw7262yW8sTBwxi1PdJA3Iw/EKBa8psRf7d9a4=
github.com/mailru/easyjson v0.9.0/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU=
github.com/mattetti/filebuffer v1.0.1 h1:gG7pyfnSIZCxdoKq+cPa8T0hhYtD9NxCdI4D7PTjRLM=
github.com/mattetti/filebuffer v1.0.1/go.mod h1:YdMURNDOttIiruleeVr6f56OrMc+MydEnTcXwtkxNVs=
github.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
Expand Down
81 changes: 66 additions & 15 deletions internal/resources/grafana/resource_alerting_rule_group.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (
goapi "github.com/grafana/grafana-openapi-client-go/client"
"github.com/grafana/grafana-openapi-client-go/client/provisioning"
"github.com/grafana/grafana-openapi-client-go/models"
"github.com/hashicorp/go-cty/cty"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
Expand Down Expand Up @@ -104,6 +105,28 @@ This resource requires Grafana 9.1.0 or later.
return oldDuration == newDuration
},
},
"keep_firing_for": {
Type: schema.TypeString,
Optional: true,
Description: "The amount of time for which the rule will considered to be Recovering after initially Firing. Before this time has elapsed, the rule will continue to fire once it's been triggered.",
ValidateDiagFunc: common.ValidateDurationWithDays,
DiffSuppressFunc: func(k, oldValue, newValue string, d *schema.ResourceData) bool {
oldDuration, _ := strfmt.ParseDuration(oldValue)
newDuration, _ := strfmt.ParseDuration(newValue)
return oldDuration == newDuration
},
},
"missing_series_evals_to_resolve": {
Type: schema.TypeInt,
Optional: true,
Description: "The number of missing series evaluations that must occur before the rule is considered to be resolved.",
ValidateDiagFunc: func(i any, path cty.Path) (diags diag.Diagnostics) {
if i != nil && i.(int) < 1 {
return diag.Errorf("missing_series_evals_to_resolve must be greater than or equal to 1")
}
return nil
},
},
"no_data_state": {
Type: schema.TypeString,
Optional: true,
Expand Down Expand Up @@ -519,6 +542,15 @@ func packAlertRule(r *models.ProvisionedAlertRule) (interface{}, error) {
json["record"] = record
}

// FIXME: open api needs to be a reference to the duration
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexander-akhmetov I forgot to about this discovery. The For in the spec is a *Duration and the KeepFiringFor is a Duration. I think effectively they're the same thing but not 100% sure, can you have another look at this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why it's not a *Duration as well, they are similarly defined. I'll check.

Copy link
Contributor

@alexander-akhmetov alexander-akhmetov Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably because the For field is required, it's a pointer: go-swagger/go-swagger#1386 (comment), and KeepFiringFor is not required.

if r.KeepFiringFor != 0 {
json["keep_firing_for"] = r.KeepFiringFor.String()
}

if r.MissingSeriesEvalsToResolve > 1 {
json["missing_series_evals_to_resolve"] = r.MissingSeriesEvalsToResolve
}

return json, nil
}

Expand All @@ -538,11 +570,22 @@ func unpackAlertRule(raw interface{}, groupName string, folderUID string, orgID
return nil, err
}

keepFiringForStr := json["keep_firing_for"].(string)
if keepFiringForStr == "" {
keepFiringForStr = "0"
}
keepFiringForDuration, err := strfmt.ParseDuration(keepFiringForStr)
if err != nil {
return nil, err
}

ns, err := unpackNotificationSettings(json["notification_settings"])
if err != nil {
return nil, err
}

missingSeriesEvalsToResolve := int64(json["missing_series_evals_to_resolve"].(int))

// Check for conflicting fields before unpacking the rest of the rule.
// This is a workaround due to the lack of support for ConflictsWith in Lists in the SDK.
errState := json["exec_err_state"].(string)
Expand All @@ -555,6 +598,12 @@ func unpackAlertRule(raw interface{}, groupName string, folderUID string, orgID
if forDuration != 0 {
return nil, fmt.Errorf(incompatFieldMsgFmt, "for")
}
if keepFiringForDuration != 0 {
return nil, fmt.Errorf(incompatFieldMsgFmt, "keep_firing_for")
}
if missingSeriesEvalsToResolve != 0 {
return nil, fmt.Errorf(incompatFieldMsgFmt, "missing_series_evals_to_resolve")
}
if noDataState != "" {
return nil, fmt.Errorf(incompatFieldMsgFmt, "no_data_state")
}
Expand All @@ -580,21 +629,23 @@ func unpackAlertRule(raw interface{}, groupName string, folderUID string, orgID
}

rule := models.ProvisionedAlertRule{
UID: json["uid"].(string),
Title: common.Ref(json["name"].(string)),
FolderUID: common.Ref(folderUID),
RuleGroup: common.Ref(groupName),
OrgID: common.Ref(orgID),
ExecErrState: common.Ref(errState),
NoDataState: common.Ref(noDataState),
For: common.Ref(strfmt.Duration(forDuration)),
Data: data,
Condition: common.Ref(condition),
Labels: unpackMap(json["labels"]),
Annotations: unpackMap(json["annotations"]),
IsPaused: json["is_paused"].(bool),
NotificationSettings: ns,
Record: unpackRecord(json["record"]),
UID: json["uid"].(string),
Title: common.Ref(json["name"].(string)),
FolderUID: common.Ref(folderUID),
RuleGroup: common.Ref(groupName),
OrgID: common.Ref(orgID),
ExecErrState: common.Ref(errState),
NoDataState: common.Ref(noDataState),
For: common.Ref(strfmt.Duration(forDuration)),
KeepFiringFor: strfmt.Duration(keepFiringForDuration),
Data: data,
Condition: common.Ref(condition),
Labels: unpackMap(json["labels"]),
Annotations: unpackMap(json["annotations"]),
IsPaused: json["is_paused"].(bool),
NotificationSettings: ns,
Record: unpackRecord(json["record"]),
MissingSeriesEvalsToResolve: missingSeriesEvalsToResolve,
}

return &rule, nil
Expand Down
2 changes: 1 addition & 1 deletion internal/resources/grafana/resource_team.go
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,7 @@ func memberChanges(stateMembers, configMembers map[string]TeamMember) []MemberCh
func addMemberIdsToChanges(client *goapi.GrafanaHTTPAPI, changes []MemberChange) ([]MemberChange, error) {
gUserMap := make(map[string]int64)

resp, err := client.Org.GetOrgUsersForCurrentOrg()
resp, err := client.Org.GetOrgUsersForCurrentOrg(nil)
if err != nil {
return nil, err
}
Expand Down
2 changes: 1 addition & 1 deletion internal/resources/grafana/resources.go
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ var Resources = addValidationToResources(
resourceFolderPermission(),
resourceLibraryPanel(),
resourceMessageTemplate(),
resourceMuteTiming(),
// resourceMuteTiming(),
resourceNotificationPolicy(),
resourceOrganization(),
resourceOrganizationPreferences(),
Expand Down
Loading