Add GitLab webhook support #56

vigo · 2026-01-03T16:13:32Z

Summary

Add complete GitLab webhook processing pipeline with handler, storage, and consumer
Support all GitLab event types including premium/ultimate events (user_add_to_group, project_create, subgroup_create)
Add database migrations for gitlab and gitlab_user tables with partial indexes
Add GitHub Actions workflows for build and staging deployment
Update pre-commit config to use TekWizely/pre-commit-golang

Test plan

All existing tests pass
New gitlabwebhookhandler tests pass (21 tests)
New gitlabstorage tests pass (17 tests)
golangci-lint passes with 0 issues
Manual test with GitLab webhook payloads
Verify database migrations apply correctly

🤖 Generated with Claude Code

Implement full GitLab webhook processing pipeline: - Add database migrations for gitlab and gitlab_user tables - Create gitlabwebhookhandler with plain text token validation - Create gitlabstorage with nullable field handling for varying payloads - Add gitlabconsumer service for Kafka message processing - Update webhookserver with /v1/webhook/gitlab endpoint - Add Dockerfile and docker-compose service for gitlab-consumer - Add GitHub Actions workflows for gitlab-consumer (build + staging) - Add rake task for running gitlab consumer locally - Update pre-commit config to use TekWizely/pre-commit-golang Supports all GitLab webhook event types including premium/ultimate events (user_add_to_group, project_create, subgroup_create) with fallback handling for object_kind vs event_name and nested vs flat user/project fields. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

chatgpt-codex-connector · 2026-01-03T16:13:36Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

codecov · 2026-01-03T16:14:02Z

Codecov Report

❌ Patch coverage is 98.59649% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/storage/gitlabstorage/gitlabstorage.go	98.55%	1 Missing and 1 partial ⚠️
.../http/gitlabwebhookhandler/gitlabwebhookhandler.go	98.63%	2 Missing ⚠️

Files with missing lines	Coverage Δ
internal/storage/gitlabstorage/gitlabstorage.go	`98.55% <98.55%> (ø)`
.../http/gitlabwebhookhandler/gitlabwebhookhandler.go	`98.63% <98.63%> (ø)`

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Add TestMessageQueue_Full for handler queue full scenario - Add TestStore_Success_SkipsZeroProjectID for zero project_id - Add TestStore_Success_SkipsZeroUserID for zero user_id Coverage improved: - gitlabwebhookhandler: 97.7% → 98.9% - gitlabstorage: 97.6% → 98.8% 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

Copilot

Pull request overview

This PR adds comprehensive GitLab webhook support to the cauldron system, implementing a complete processing pipeline parallel to the existing GitHub functionality. The implementation includes webhook handling, Kafka-based message queuing, persistent storage, and consumer processing for all GitLab event types including premium/ultimate tier events.

Adds GitLab webhook HTTP handler with token-based authentication
Implements database schema migrations for gitlab and gitlab_user tables with optimized partial indexes
Creates complete GitLab consumer with storage layer for processing webhook events

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
migrations/000007_gitlab.up.sql	Creates gitlab table with partial indexes for nullable fields and GIN index for payload JSONB
migrations/000007_gitlab.down.sql	Rollback script for gitlab table
migrations/000008_gitlab_user.up.sql	Creates gitlab_user table with foreign key to app_user and indexes
migrations/000008_gitlab_user.down.sql	Rollback script for gitlab_user table
internal/transport/http/gitlabwebhookhandler/gitlabwebhookhandler.go	HTTP handler for GitLab webhooks with token validation and Kafka message queuing
internal/transport/http/gitlabwebhookhandler/gitlabwebhookhandler_test.go	Comprehensive test suite (21 tests) for GitLab webhook handler
internal/storage/gitlabstorage/gitlabstorage.go	Storage layer for persisting GitLab webhook data to PostgreSQL
internal/storage/gitlabstorage/gitlabstorage_test.go	Test suite (17 tests) for GitLab storage with mocked database
cmd/gitlabconsumer/main.go	Kafka consumer application for processing GitLab webhook messages
cmd/webhookserver/main.go	Updated to include GitLab webhook endpoint and worker pool
docker-compose.infra.yml	Adds gitlab-consumer service configuration
Dockerfile.gitlab-consumer	Multi-stage Docker build for GitLab consumer
.pre-commit-config.yaml	Updates to TekWizely/pre-commit-golang for better Go tooling
README.md	Updates badges and adds supported Git providers section
scripts/local/rake/run.rake	Adds rake task for running GitLab consumer locally
.github/workflows/*.yml	New and updated workflows for building and deploying GitLab consumer

Comments suppressed due to low confidence (1)

internal/transport/http/gitlabwebhookhandler/gitlabwebhookhandler_test.go:503

This test has a race condition. The message is read from the channel twice: once inside the goroutine (line 497) and once at the assertion (line 503). This will cause the test to hang or fail intermittently as the second read will block indefinitely. Remove either the goroutine or the final assertion.


	// Wait a bit for goroutine to execute default case (queue full)
	time.Sleep(50 * time.Millisecond)

	assert.Equal(t, fasthttp.StatusAccepted, ctx.Response.StatusCode())
}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-03T16:18:41Z

docker-compose.infra.yml

      - "8000:8000"
    environment:
      GITHUB_HMAC_SECRET: "${GITHUB_HMAC_SECRET}"
+      GITLAB_HMAC_SECRET: "${GITLAB_HMAC_SECRET}"


The environment variable name is inconsistent with the actual secret mechanism used. GitLab uses a plain token authentication (X-Gitlab-Token header), not HMAC-based authentication. This variable should be named GITLAB_WEBHOOK_SECRET or GITLAB_TOKEN instead of GITLAB_HMAC_SECRET to accurately reflect the authentication mechanism.

Suggested change

GITLAB_HMAC_SECRET: "${GITLAB_HMAC_SECRET}"

GITLAB_WEBHOOK_SECRET: "${GITLAB_WEBHOOK_SECRET}"

Copilot · 2026-01-03T16:18:42Z

cmd/webhookserver/main.go


 	var wg sync.WaitGroup

+	// GitHub message workers


The comment is misleading as these workers were already processing GitHub messages. This comment suggests a change in behavior when it's actually just clarification. Consider rewording to "// Process GitHub webhook messages" to be clearer about the existing behavior.

Suggested change

// GitHub message workers

// Process GitHub webhook messages

Copilot · 2026-01-03T16:18:42Z

cmd/webhookserver/main.go

 		}()
 	}

+	// GitLab message workers


The comment is misleading as these are new workers added in this PR. Consider rewording to "// Process GitLab webhook messages" to match the style of the GitHub workers comment and be more descriptive about what messages are being processed.

Suggested change

// GitLab message workers

// Process GitLab webhook messages

Copilot · 2026-01-03T16:18:43Z

internal/transport/http/gitlabwebhookhandler/gitlabwebhookhandler.go

+	go func() {
+		select {
+		case h.MessageQueue <- message:
+			h.Logger.Info(
+				"kafka message queued for processing",
+				"key", string(gitlabEventUUID),
+				"topic", h.Topic,
+			)
+		case <-ctx.Done():
+			h.Logger.Info("received context Done")
+		default:
+			h.Logger.Error(
+				"kafka message queue full, dropping message",
+				"key", string(gitlabEventUUID),
+				"topic", h.Topic,
+			)
+		}
+	}()


The goroutine for message queue handling has a potential issue. The select statement's default case will immediately drop messages if the queue is full, which may lead to silent data loss. Consider either blocking until the message can be sent, or returning an error status to the webhook caller when the queue is full so they can retry. The current implementation responds with StatusAccepted even if the message is dropped.

Suggested change

go func() {

select {

case h.MessageQueue <- message:

h.Logger.Info(

"kafka message queued for processing",

"key", string(gitlabEventUUID),

"topic", h.Topic,

)

case <-ctx.Done():

h.Logger.Info("received context Done")

default:

h.Logger.Error(

"kafka message queue full, dropping message",

"key", string(gitlabEventUUID),

"topic", h.Topic,

)

}

}()

select {

case h.MessageQueue <- message:

h.Logger.Info(

"kafka message queued for processing",

"key", string(gitlabEventUUID),

"topic", h.Topic,

)

case <-ctx.Done():

h.Logger.Info(

"received context Done before queuing kafka message",

"key", string(gitlabEventUUID),

"topic", h.Topic,

)

ctx.SetStatusCode(fasthttp.StatusServiceUnavailable)

return

}

Copilot · 2026-01-03T16:18:43Z

internal/storage/gitlabstorage/gitlabstorage_test.go

+			{Key: []byte("project-id"), Value: []byte(``)}, // empty, should skip
+		},
+	}


This test has duplicate headers with the same key "project-id" but different values (empty string and "0"). The test intent is unclear - in real Kafka messages, the last header value would typically take precedence. Consider separating these into distinct test cases or clarifying the expected behavior when both conditions should be tested.

Copilot · 2026-01-03T16:18:43Z

cmd/gitlabconsumer/main.go

+	}
+}
+
+// Run runs kafa gitlab consumer.


Typo in comment: "kafa" should be "kafka".

Suggested change

// Run runs kafa gitlab consumer.

// Run runs kafka gitlab consumer.

Copilot · 2026-01-03T16:18:43Z

cmd/webhookserver/main.go

 	serverIdleTimeout := getenv.Duration("SERVER_IDLE_TIMEOUT", webhookserver.ServerDefaultIdleTimeout)

 	githubHMACSecret := getenv.String("GITHUB_HMAC_SECRET", "")
+	gitlabHMACSecret := getenv.String("GITLAB_HMAC_SECRET", "")


The environment variable name is inconsistent with the actual secret mechanism used. GitLab uses a plain token authentication (X-Gitlab-Token header), not HMAC-based authentication. This variable should be named GITLAB_WEBHOOK_SECRET or GITLAB_TOKEN instead of GITLAB_HMAC_SECRET to accurately reflect the authentication mechanism.

Suggested change

gitlabHMACSecret := getenv.String("GITLAB_HMAC_SECRET", "")

gitlabWebhookSecret := getenv.String("GITLAB_WEBHOOK_SECRET", "")

gitlabHMACSecret := getenv.String("GITLAB_HMAC_SECRET", *gitlabWebhookSecret)

Copilot · 2026-01-03T16:18:44Z

Dockerfile.gitlab-consumer

+FROM alpine:latest AS certs
+RUN apk add --update --no-cache ca-certificates
+
+FROM busybox:latest


This Dockerfile uses unpinned base images alpine:latest and busybox:latest, which are mutable tags and introduce a supply-chain risk if the upstream images are compromised. Because the GitLab consumer container will be built and run with access to Kafka and database credentials, a malicious update to those tags could silently run attacker-controlled code inside your deployment. Pin these images to immutable digests or at least specific version tags to ensure you only run trusted image content.

Main entry point files don't have unit tests as they're integration points. Ignore them from patch coverage calculation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

Copilot AI review requested due to automatic review settings January 3, 2026 16:13

Copilot started reviewing on behalf of vigo January 3, 2026 16:13 View session

Copilot AI reviewed Jan 3, 2026

View reviewed changes

vigo and others added 2 commits January 3, 2026 19:21

fix typo in comments

e5909b8

Add codecov config to ignore main.go files

5695762

Main entry point files don't have unit tests as they're integration points. Ignore them from patch coverage calculation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

vigo merged commit 2de5846 into main Jan 3, 2026
4 checks passed

vigo deleted the gitlab-works branch January 3, 2026 16:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add GitLab webhook support #56

Add GitLab webhook support #56

Uh oh!

vigo commented Jan 3, 2026

Uh oh!

chatgpt-codex-connector bot commented Jan 3, 2026

Uh oh!

codecov bot commented Jan 3, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 3, 2026

Uh oh!

Copilot AI Jan 3, 2026

Uh oh!

Copilot AI Jan 3, 2026

Uh oh!

Copilot AI Jan 3, 2026

Uh oh!

Copilot AI Jan 3, 2026

Uh oh!

Copilot AI Jan 3, 2026

Uh oh!

Copilot AI Jan 3, 2026

Uh oh!

Copilot AI Jan 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	GITLAB_HMAC_SECRET: "${GITLAB_HMAC_SECRET}"
	GITLAB_WEBHOOK_SECRET: "${GITLAB_WEBHOOK_SECRET}"

	// Run runs kafa gitlab consumer.
	// Run runs kafka gitlab consumer.

	gitlabHMACSecret := getenv.String("GITLAB_HMAC_SECRET", "")
	gitlabWebhookSecret := getenv.String("GITLAB_WEBHOOK_SECRET", "")
	gitlabHMACSecret := getenv.String("GITLAB_HMAC_SECRET", *gitlabWebhookSecret)

Add GitLab webhook support #56

Add GitLab webhook support #56

Uh oh!

Conversation

vigo commented Jan 3, 2026

Summary

Test plan

Uh oh!

chatgpt-codex-connector bot commented Jan 3, 2026

Uh oh!

codecov bot commented Jan 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Jan 3, 2026 •

edited

Loading