Skip to content

Conversation

eshiferax
Copy link
Contributor

@eshiferax eshiferax commented Jun 6, 2025

For S3 tecton data sources (i.e. FileConfig BatchSource) in the same account as the dataplane, users currently need to manually edit their tecton-rift-compute role in order to give the jobs access to read those buckets

(if the source bucket is in a different account, they can get access with an s3 bucket policy + existing policy statement that already allows reads for buckets in separate accounts)

We should provide an option for this (additional read permissions for tecton-rift-compute) to be added via terraform to avoid drift / risk of removing their changes if they need to pull in updates or re-apply.

Adding new variable additional_s3_read_access_buckets (list of bucket names) here to support this.

Testing

  1. Tested with an internal cluster without setting this variable. No changes.
  2. Tested with an internal cluster setting additional_s3_read_access_buckets == [tst-tecton-access] Resulted in:
  # module.rift.aws_iam_policy.additional_s3_read_access[0] will be created
  + resource "aws_iam_policy" "additional_s3_read_access" {
      + arn         = (known after apply)
      + id          = (known after apply)
      + name        = "tecton-additional-s3-read-access"
      + name_prefix = (known after apply)
      + path        = "/"
      + policy      = jsonencode(
            {
              + Statement = [
                  + {
                      + Action   = [
                          + "s3:ListBucket"
                        ]
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:s3:::tst-tecton-access",
                        ]
                    },
                  + {
                      + Action   = [
                          + "s3:GetObject",
                          + "s3:GetObjectversion",
                        ]
                      + Effect   = "Allow"
                      + Resource = [
                          + "arn:aws:s3:::tst-tecton-access/*",
                        ]
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
      + policy_id   = (known after apply)
      + tags_all    = (known after apply)
    }

  # module.rift.aws_iam_role_policy_attachment.rift_compute_policies["additional_s3_read_access"] will be created
  + resource "aws_iam_role_policy_attachment" "rift_compute_policies" {
      + id         = (known after apply)
      + policy_arn = (known after apply)
      + role       = "tecton-rift-compute"
    }

Plan: 2 to add, 0 to change, 0 to destroy.

@eshiferax eshiferax force-pushed the support-additional-read-buckets branch from 21f24be to 92879c4 Compare June 8, 2025 21:03
@eshiferax eshiferax changed the title feat: Add option to extend allowed buckets (for read) for rift_Compute, for data sources. feat: Add option to extend allowed buckets (for read) for Rift Compute role Jun 8, 2025
@eshiferax eshiferax force-pushed the support-additional-read-buckets branch from 92879c4 to 8fe5c01 Compare June 8, 2025 21:11
@eshiferax eshiferax marked this pull request as ready for review June 9, 2025 14:01
@eshiferax eshiferax requested a review from a team June 9, 2025 14:23
@eshiferax eshiferax force-pushed the support-additional-read-buckets branch from 8fe5c01 to ed4b79c Compare June 9, 2025 17:48
…read source buckets)

Use bucket names instead of ARNS for 'additional_s3_read'
@eshiferax eshiferax force-pushed the support-additional-read-buckets branch from ed4b79c to 1b34e8b Compare June 9, 2025 17:52
@eshiferax eshiferax merged commit dd675e4 into master Jun 10, 2025
1 check passed
@eshiferax eshiferax deleted the support-additional-read-buckets branch June 10, 2025 13:27
eshiferax pushed a commit that referenced this pull request Jun 12, 2025
🤖 I have created a release *beep* *boop*
---


##
[1.5.0](v1.4.0...v1.5.0)
(2025-06-10)


### Features

* Add option to extend allowed buckets (for read) for Rift Compute role
([#219](#219))
([dd675e4](dd675e4))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants