Skip to content

4665: Setup for Log v2 config #2298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 53 commits into
base: main
Choose a base branch
from
Open

4665: Setup for Log v2 config #2298

wants to merge 53 commits into from

Conversation

matt-metivier
Copy link
Contributor

🎯 Overview

Extends Terraform provider support for managing Asserts configurations through Grafana Cloud by adding resources for Custom Model Rules and Log Drilldown Environment Config. This implementation uses the existing integration with the Grafana Asserts App Plugin, leveraging Grafana Service Account authentication and stack-scoped API calls.

🚀 Features Added

Resources Implemented

  • grafana_asserts_custom_model_rules — Manages custom model rule configurations.
  • grafana_asserts_log_drilldown_config — Manages per-environment log drilldown configuration.

Authentication & Architecture

  • Grafana Service Account Token authentication (no separate Asserts tokens needed)
  • Stack-scoped API calls using X-Scope-OrgID headers
  • Grafana Asserts App Plugin integration via plugin API gateway
  • Updated generated API client from grafana-asserts-public-clients/go/gcom (repo)

🏗️ Architecture

API Flow:

Terraform Provider → Grafana Service Account Token → Grafana Asserts App Plugin → Asserts Backend

Plugin API Gateway Endpoints:

/api/plugins/grafana-asserts-app/resources/asserts/api-server/v1/config/model-rules
/api/plugins/grafana-asserts-app/resources/asserts/api-server/v1/config/environment
/api/plugins/grafana-asserts-app/resources/asserts/api-server/v1/config/environment/{name}

Key Implementation Details:

  • Uses generated client github.com/grafana/grafana-asserts-public-clients/go/gcom (repo)
  • API calls are routed through the Grafana plugin system; the plugin handles auth and forwards requests
  • For log drilldown, uses updated DTOs: TenantEnvConfigResponseDto, EnvironmentDto, LogConfigDto

Headers Used:

  • Authorization: Bearer {token}
  • X-Scope-OrgID: {stackID}
  • Content-Type: application/json

💻 Example Usage

// Custom Model Rules
resource "grafana_asserts_custom_model_rules" "test" {
  name = "test-anything"
  rules = <<-EOT
    entities:
    - type: Whatever
      name: Nothing
      definedBy:
      - query: "up{job=\\"nothing\\"}"
  EOT
}

// Log Drilldown Environment Config
resource "grafana_asserts_log_drilldown_config" "env_default" {
  name  = "default"
  # YAML for EnvironmentDto
  config = <<-YAML
    name: default
    defaultConfig: true
    logConfig:
      tool: "loki"
      dataSource: "Loki"
      correlationLabels: "traceID, spanID"
      labelToLogFieldMapping:
        service: "service"
        namespace: "namespace"
  YAML
}

Import for Log Drilldown Config:

terraform import grafana_asserts_log_drilldown_config.name "<environment_name>"

📁 Files Added/Modified

New Files

  • internal/resources/asserts/resource_log_drilldown_config.go — Log Drilldown Environment Config resource (CRUD via LogConfigControllerAPI)

  • internal/resources/asserts/resource_log_drilldown_config_test.go — Acceptance tests (create/update/read/delete/import)

  • examples/resources/grafana_asserts_log_drilldown_config/resource.tf — Example usage

  • examples/resources/grafana_asserts_log_drilldown_config/import.sh — Import helper

  • docs/resources/asserts_log_drilldown_config.md — Generated documentation

  • internal/resources/asserts/resource_custom_model_rules.go — Custom Model Rules resource

  • internal/resources/asserts/resource_custom_model_rules_test.go — Acceptance tests

  • examples/resources/grafana_asserts_custom_model_rules/resource.tf — Example usage

  • docs/resources/asserts_custom_model_rules.md — Generated documentation

Modified Files

  • internal/resources/asserts/resources.go — Registered new resource
  • internal/resources/asserts/catalog-resource.yaml — Added Backstage catalog entries
  • go.mod / go.sum — Bumped grafana-asserts-public-clients/go/gcom to latest with updated DTOs and endpoints
  • Documentation site regenerated (go generate ./...)

Test Categories

  • ✅ Resource schema validation
  • ✅ CRUD operations (Create, Read, Update, Delete)
  • ✅ Import functionality
  • ✅ Error handling and diagnostics
  • ✅ Stack ID validation from provider configuration
  • ✅ Example configurations

Running Tests

# Run Asserts-specific tests
go test -v ./internal/resources/asserts/... -timeout=30m

# Run all tests including examples
TF_ACC_ASSERTS=true go test -v ./internal/resources/examples_test.go -timeout=30m

🔧 Implementation Notes

Client Configuration

  • Base: Grafana instance URL
  • Plugin gateway: /api/plugins/grafana-asserts-app/resources/asserts/api-server
  • Services used:
    • CustomModelRulesControllerAPI: PUT model rules
    • LogConfigControllerAPI: GET tenant env config, POST upsert env config, DELETE env config by name
  • DTOs:
    • Custom model rules YAML preserved round-trip to minimize diffs
    • Log drilldown uses EnvironmentDto with nested LogConfigDto

Plugin Integration

  • Leverages the Asserts App Plugin for API access
  • No direct backend integration
  • Plugin handles routing and authentication

Resource Configuration

  • Stack ID from Provider via X-Scope-OrgID
  • Name-based IDs:
    • Custom model rules: name
    • Log drilldown config: environment name
  • Import:
    • Log drilldown config: terraform import grafana_asserts_log_drilldown_config.name "<environment_name>"

Error Handling

  • Consistent diagnostics for client configuration
  • Retry on eventual consistency for reads
  • Graceful handling of non-200 responses

📚 Dependencies

  • github.com/grafana/grafana-asserts-public-clients/go/gcom — Generated API client (repo)
  • Grafana Asserts App Plugin installed and enabled
  • Grafana Service Account with appropriate permissions

🔐 Security & Permissions

  • Uses Grafana Service Account authentication

  • Stack-scoped access via X-Scope-OrgID

  • No additional Asserts-specific tokens required

  • Inherits Grafana’s security model

  • Added grafana_asserts_log_drilldown_config resource with env-scoped CRUD via LogConfigControllerAPI

  • Updated client to latest; adopted updated DTOs (EnvironmentDto, LogConfigDto, TenantEnvConfigResponseDto)

  • Preserved existing grafana_asserts_custom_model_rules; examples, docs, tests for both resources

  • Added example import script; updated docs and catalog entries

@matt-metivier matt-metivier requested review from a team as code owners August 11, 2025 23:20
Copy link
Contributor

In order to lower resource usage and have a faster runtime, PRs will not run Cloud tests automatically.
To do so, a Grafana Labs employee must trigger the cloud acceptance tests workflow manually.

4565: Updating stack id docs

4565: fixing test

4565: running test and fixing comment

4565: owner update

4565: fixing path

Fix concurrency race condition in generate tool

Fix 'panic: send on closed channel' error in generateImportBlocks.

The issue was a classic Go concurrency race condition:
- Goroutines called wg.Done() before sending to results channel
- Main goroutine could close(results) while sends were still pending
- This caused 'send on closed channel' panics

Fix: Move all 'results <- result{}' sends BEFORE wg.Done() calls
to ensure all channel operations complete before waitgroup finishes.

This prevents the channel from being closed while goroutines are
still trying to send results.

Fixes integration test failures in CI.

Add dedicated environment check for Asserts service tests

Create CheckAssertsTestsEnabled() function to properly gate Asserts tests.

Tests now require TF_ACC_ASSERTS=true to run, preventing 404 errors
when Asserts service is not available in test environments.

This follows the same pattern as other optional cloud services and
ensures tests only run when the service is actually configured.

Fix linter error: remove unused getTestStackIDForDisabled function

Removed duplicate getTestStackIDForDisabled function and unnecessary imports.
Both test files now share the getTestStackID function from the same package.

Fixes:
- unused function (golangci-lint)
- unnecessary imports (os, strconv, require)

All tests still pass and linter is clean.

4565: adding test

4565: adding test

4565: adding to catalog

Update asserts resources and configuration

4565: fixing client

4565: update to add stack_id

4565: small revert

4565: Setup for asserts TF
This reverts commit 3a580f2.
This reverts commit 29e889d.
4565: Updating stack id docs

4565: fixing test

4565: running test and fixing comment

4565: owner update

4565: fixing path

Fix concurrency race condition in generate tool

Fix 'panic: send on closed channel' error in generateImportBlocks.

The issue was a classic Go concurrency race condition:
- Goroutines called wg.Done() before sending to results channel
- Main goroutine could close(results) while sends were still pending
- This caused 'send on closed channel' panics

Fix: Move all 'results <- result{}' sends BEFORE wg.Done() calls
to ensure all channel operations complete before waitgroup finishes.

This prevents the channel from being closed while goroutines are
still trying to send results.

Fixes integration test failures in CI.

Add dedicated environment check for Asserts service tests

Create CheckAssertsTestsEnabled() function to properly gate Asserts tests.

Tests now require TF_ACC_ASSERTS=true to run, preventing 404 errors
when Asserts service is not available in test environments.

This follows the same pattern as other optional cloud services and
ensures tests only run when the service is actually configured.

Fix linter error: remove unused getTestStackIDForDisabled function

Removed duplicate getTestStackIDForDisabled function and unnecessary imports.
Both test files now share the getTestStackID function from the same package.

Fixes:
- unused function (golangci-lint)
- unnecessary imports (os, strconv, require)

All tests still pass and linter is clean.

4565: adding test

4565: adding test

4565: adding to catalog

Update asserts resources and configuration

4565: fixing client

4565: update to add stack_id

4565: small revert

4565: Setup for asserts TF
# Conflicts:
#	internal/resources/asserts/resource_alert_config_test.go
#	internal/resources/asserts/resource_disabled_alert_config_test.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant