feat: Add support for offline and online store KMS keys (#252)

Add the variables `offline_store_kms_key_arn` and
`online_store_kms_key_arn` to provide per-store keys to the Rift module.

---------

Co-authored-by: Taylor Daugherty <[email protected]>

Author: taylord-tecton

.pre-commit-config.yaml

@@ -3,17 +3,22 @@ repos:
     rev: "v0.20.0"
     hooks:
       - id: terraform-docs-go
+        language_version: "1.24.6"
         name: terraform-docs-modules
         args: ["-c", ".terraform-docs.yml", "./modules/"]
       - id: terraform-docs-go
+        language_version: "1.24.6"
         name: terraform-docs-deployment
         args: ["-c", "deployment/.terraform-docs.yml", "./deployment/"]
       - id: terraform-docs-go
+        language_version: "1.24.6"
         name: terraform-docs-emr
         args: ["-c", ".terraform-docs.yml", "./emr/"]
       - id: terraform-docs-go
+        language_version: "1.24.6"
         name: terraform-docs-rift_compute
         args: ["-c", "rift_compute/.terraform-docs.yml", "./rift_compute/"]
       - id: terraform-docs-go
+        language_version: "1.24.6"
         name: terraform-docs-privatelink
         args: ["-c", ".terraform-docs.yml", "./privatelink/"]

CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+#  [1.9.0](https://github.com/tecton-ai/tecton-terraform-setup/compare/v1.8.1...v1.9.0) (2025-09-15)
+
+### Features
+
+* Add new variable: deployment_role_permissions_boundary_arnARN
+    + ARN of the policy that is used to set the permissions boundary for the deployment role
+* Add new variable: rift_role_permissions_boundary_arn
+
+ARN of the policy that is used to set the permissions boundary for the rift compute roles
+
 ## [1.8.1](https://github.com/tecton-ai/tecton-terraform-setup/compare/v1.8.0...v1.8.1) (2025-08-18)
 
 

deployment/README.md

@@ -20,6 +20,7 @@
 | <a name="input_cross_account_role_allow_sts_metadata"></a> [cross\_account\_role\_allow\_sts\_metadata](#input\_cross\_account\_role\_allow\_sts\_metadata) | Enable sts:SetSourceIdentity and sts:TagSession permissions on the cross-role account. | `bool` | `false` | no |
 | <a name="input_databricks_spark_role_name"></a> [databricks\_spark\_role\_name](#input\_databricks\_spark\_role\_name) | n/a | `string` | `null` | no |
 | <a name="input_deployment_name"></a> [deployment\_name](#input\_deployment\_name) | Name of the Tecton deployment. | `string` | n/a | yes |
+| <a name="input_deployment_role_permissions_boundary_arn"></a> [deployment\_role\_permissions\_boundary\_arn](#input\_deployment\_role\_permissions\_boundary\_arn) | ARN of the policy that is used to set the permissions boundary for the deployment role | `string` | `null` | no |
 | <a name="input_emr_read_ecr_repositories"></a> [emr\_read\_ecr\_repositories](#input\_emr\_read\_ecr\_repositories) | List of ECR repositories that EMR roles are granted read access to. | `list(string)` | `[]` | no |
 | <a name="input_emr_spark_role_name"></a> [emr\_spark\_role\_name](#input\_emr\_spark\_role\_name) | Override the default name Tecton uses for emr spark role | `string` | `null` | no |
 | <a name="input_include_crossaccount_bucket_access"></a> [include\_crossaccount\_bucket\_access](#input\_include\_crossaccount\_bucket\_access) | Whether to grant direct cross-account bucket access | `bool` | `true` | no |

modules/dataplane_rift/README.md

@@ -82,12 +82,14 @@ output "tecton" {
 | <a name="input_controlplane_access_only"></a> [controlplane\_access\_only](#input\_controlplane\_access\_only) | Whether to only grant control-plane account access to the cross-account role | `bool` | `true` | no |
 | <a name="input_cross_account_external_id"></a> [cross\_account\_external\_id](#input\_cross\_account\_external\_id) | The external ID for cross-account access. Obtain this from your Tecton representative. | `string` | n/a | yes |
 | <a name="input_deployment_name"></a> [deployment\_name](#input\_deployment\_name) | The name of the Tecton deployment. Must be less than 22 characters due to AWS limitations. | `string` | n/a | yes |
+| <a name="input_deployment_role_permissions_boundary_arn"></a> [deployment\_role\_permissions\_boundary\_arn](#input\_deployment\_role\_permissions\_boundary\_arn) | ARN of the policy that is used to set the permissions boundary for the deployment role | `string` | `null` | no |
 | <a name="input_existing_rift_compute_security_group_id"></a> [existing\_rift\_compute\_security\_group\_id](#input\_existing\_rift\_compute\_security\_group\_id) | (Optional) The ID of the existing security group to use for Rift compute instances. | `string` | `null` | no |
 | <a name="input_existing_vpc"></a> [existing\_vpc](#input\_existing\_vpc) | (Optional) Configuration for using an existing VPC. If provided, both vpc\_id and private\_subnet\_ids must be provided together. | <pre>object({<br/>    vpc_id               = string<br/>    private_subnet_ids   = list(string)<br/>  })</pre> | `null` | no |
 | <a name="input_include_crossaccount_bucket_access"></a> [include\_crossaccount\_bucket\_access](#input\_include\_crossaccount\_bucket\_access) | Whether to grant direct cross-account bucket access | `bool` | `true` | no |
 | <a name="input_kms_key_id"></a> [kms\_key\_id](#input\_kms\_key\_id) | (Optional) The customer-managed key for encrypting data at rest. | `string` | `null` | no |
 | <a name="input_outputs_location_config"></a> [outputs\_location\_config](#input\_outputs\_location\_config) | Configuration for where to store the outputs. Defaults to creating a dedicated bucket. | <pre>object({<br/>    type = string # "new_bucket", "offline_store_bucket_path", or "tecton_hosted_presigned"<br/>    <br/>    # For offline_store_bucket_path (bucket name is automatically set to the deployment's offline store bucket)<br/>    offline_store_bucket_name    = optional(string)<br/>    offline_store_bucket_path_prefix = optional(string, "internal/tecton-outputs/")<br/>    <br/>    # For tecton_hosted_presigned<br/>    tecton_presigned_write_url = optional(string)<br/>    trigger_upload             = optional(bool, false)<br/>  })</pre> | <pre>{<br/>  "type": "tecton_hosted_presigned"<br/>}</pre> | no |
 | <a name="input_region"></a> [region](#input\_region) | The AWS region for the Tecton deployment. | `string` | n/a | yes |
+| <a name="input_rift_role_permissions_boundary_arn"></a> [rift\_role\_permissions\_boundary\_arn](#input\_rift\_role\_permissions\_boundary\_arn) | ARN of the policy that is used to set the permissions boundary for the rift compute roles | `string` | `null` | no |
 | <a name="input_subnet_azs"></a> [subnet\_azs](#input\_subnet\_azs) | A list of Availability Zones for the subnets. | `list(string)` | n/a | yes |
 | <a name="input_tecton_control_plane_account_id"></a> [tecton\_control\_plane\_account\_id](#input\_tecton\_control\_plane\_account\_id) | The AWS account ID of the Tecton control plane. Obtain this from your Tecton representative. | `string` | n/a | yes |
 | <a name="input_tecton_control_plane_role_name"></a> [tecton\_control\_plane\_role\_name](#input\_tecton\_control\_plane\_role\_name) | The name of the Tecton control plane IAM role. Obtain this from your Tecton representative. | `string` | n/a | yes |

rift_compute/README.md

@@ -17,13 +17,16 @@
 | <a name="input_cross_account_role_arn"></a> [cross\_account\_role\_arn](#input\_cross\_account\_role\_arn) | Name of cross-account role Tecton control-plane will assume in your account. | `string` | `null` | no |
 | <a name="input_enable_rift_legacy_secret_manager_access"></a> [enable\_rift\_legacy\_secret\_manager\_access](#input\_enable\_rift\_legacy\_secret\_manager\_access) | Flag to indicate if supporting legacy secret management or not. Directly accessing secret manager from Rift jobs is no longer supported. Tecton Secrets should be used instead | `bool` | `false` | no |
 | <a name="input_existing_rift_compute_security_group_id"></a> [existing\_rift\_compute\_security\_group\_id](#input\_existing\_rift\_compute\_security\_group\_id) | Optional. The ID of an existing security group to use for Rift compute instances. If provided, the module will not create a new security group. | `string` | `null` | no |
-| <a name="input_existing_vpc"></a> [existing\_vpc](#input\_existing\_vpc) | Optional. Configuration for using an existing VPC. If provided, the module will not create a new VPC or related core networking resources (subnets, IGW, NAT GWs, Route Tables). Both vpc\_id and private\_subnet\_ids must be provided together. | <pre>object({<br/>    vpc_id               = string<br/>    private_subnet_ids   = list(string)<br/>  })</pre> | `null` | no |
+| <a name="input_existing_vpc"></a> [existing\_vpc](#input\_existing\_vpc) | Optional. Configuration for using an existing VPC. If provided, the module will not create a new VPC or related core networking resources (subnets, IGW, NAT GWs, Route Tables). Both vpc\_id and private\_subnet\_ids must be provided together. | <pre>object({<br/>    vpc_id             = string<br/>    private_subnet_ids = list(string)<br/>  })</pre> | `null` | no |
 | <a name="input_is_internal_workload"></a> [is\_internal\_workload](#input\_is\_internal\_workload) | Flag to indicate if the workload is internal to Tecton. Set it to true if for dev and demo clusters. | `bool` | `false` | no |
 | <a name="input_kms_key_arn"></a> [kms\_key\_arn](#input\_kms\_key\_arn) | ARN of KMS key used to encrypt online/offline feature store. | `string` | `null` | no |
 | <a name="input_offline_store_bucket_arn"></a> [offline\_store\_bucket\_arn](#input\_offline\_store\_bucket\_arn) | ARN of offline store bucket. | `string` | n/a | yes |
 | <a name="input_offline_store_key_prefix"></a> [offline\_store\_key\_prefix](#input\_offline\_store\_key\_prefix) | Prefix used for offline store keys. | `string` | `"offline-store/"` | no |
+| <a name="input_offline_store_kms_key_arn"></a> [offline\_store\_kms\_key\_arn](#input\_offline\_store\_kms\_key\_arn) | ARN of KMS key used to encrypt offline feature store. If given, will override the kms\_key\_arn. | `string` | `null` | no |
+| <a name="input_online_store_kms_key_arn"></a> [online\_store\_kms\_key\_arn](#input\_online\_store\_kms\_key\_arn) | ARN of KMS key used to encrypt online feature store. If given, will override the kms\_key\_arn. | `string` | `null` | no |
 | <a name="input_resource_name_overrides"></a> [resource\_name\_overrides](#input\_resource\_name\_overrides) | map of Terraform resource names, to cloud provider names. Used to override any named resource. | `map(string)` | `{}` | no |
 | <a name="input_rift_compute_manager_assuming_role_arns"></a> [rift\_compute\_manager\_assuming\_role\_arns](#input\_rift\_compute\_manager\_assuming\_role\_arns) | ARNs of the IAM roles that will be assuming `tecton-rift-compute-manager` to start rift materialization jobs. Typically `eks-worker-node`. | `list(string)` | n/a | yes |
+| <a name="input_rift_role_permissions_boundary_arn"></a> [rift\_role\_permissions\_boundary\_arn](#input\_rift\_role\_permissions\_boundary\_arn) | ARN of the policy that is used to set the permissions boundary for the rift compute roles | `string` | `null` | no |
 | <a name="input_s3_log_destination"></a> [s3\_log\_destination](#input\_s3\_log\_destination) | S3 destination for rift job logs, Example: arn:aws:s3:::tecton-log-bucket/rift-logs | `string` | n/a | yes |
 | <a name="input_subnet_azs"></a> [subnet\_azs](#input\_subnet\_azs) | A list of Availability Zones for the subnets. Not used if existing\_vpc is provided. | `list(string)` | `[]` | no |
 | <a name="input_tecton_privatelink_egress_rules"></a> [tecton\_privatelink\_egress\_rules](#input\_tecton\_privatelink\_egress\_rules) | List of egress rules for the Tecton PrivateLink security group. If empty and PrivateLink is enabled, a default 'allow all' rule will be created. | <pre>list(object({<br/>    cidr        = string<br/>    from_port   = number<br/>    to_port     = number<br/>    protocol    = string<br/>    description = string<br/>  }))</pre> | `[]` | no |

rift_compute/iam.tf

@@ -1,5 +1,5 @@
 locals {
-  use_kms_key = var.kms_key_arn != null
+  use_kms_key = var.kms_key_arn != null || var.offline_store_kms_key_arn != null || var.online_store_kms_key_arn != null
 }
 
 # rift-compute-manager role, used by orchestrator for creating/managing EC2 instances running rift materialization jobs.
@@ -21,10 +21,10 @@ resource "aws_iam_role" "rift_compute_manager" {
 }
 
 resource "aws_iam_policy" "manage_rift_compute" {
-  name   = lookup(var.resource_name_overrides, "manage_rift_compute", "manage-rift-compute")
+  name = lookup(var.resource_name_overrides, "manage_rift_compute", "manage-rift-compute")
   policy = templatefile("${path.module}/../templates/manage_rift_compute_policy.json", {
-    ACCOUNT_ID                  = local.account_id,
-    RIFT_COMPUTE_ROLE_ARN       = aws_iam_role.rift_compute.arn,
+    ACCOUNT_ID            = local.account_id,
+    RIFT_COMPUTE_ROLE_ARN = aws_iam_role.rift_compute.arn,
     ALLOW_RUN_INSTANCES_RESOURCES = jsonencode(flatten([
       "arn:aws:ec2:*:${local.account_id}:volume/*",
       local.rift_security_group.arn,
@@ -70,13 +70,15 @@ resource "aws_iam_policy" "rift_dynamodb_access" {
   name = "tecton-rift-dynamodb-access"
   policy = templatefile("${path.module}/../templates/rift_dynamodb_access_policy.json", {
     ACCOUNT_ID   = local.account_id,
-    CLUSTER_NAME = var.cluster_name
+    CLUSTER_NAME = var.cluster_name,
+    USE_KMS_KEY  = local.use_kms_key,
+    KMS_KEY_ARN  = var.online_store_kms_key_arn != null ? var.online_store_kms_key_arn : var.kms_key_arn
   })
 }
 
 resource "aws_iam_policy" "rift_legacy_secrets_manager_access" {
   count = var.enable_rift_legacy_secret_manager_access ? 1 : 0
-  name = "tecton-rift-legacy-secrets-manager-access"
+  name  = "tecton-rift-legacy-secrets-manager-access"
   policy = templatefile("${path.module}/../templates/rift_legacy_secrets_manager_access_policy.json", {
     ACCOUNT_ID = local.account_id
   })
@@ -110,7 +112,7 @@ resource "aws_iam_policy" "offline_store_access" {
     OFFLINE_STORE_KEY_PREFIX = var.offline_store_key_prefix,
     ACCOUNT_ID               = local.account_id,
     USE_KMS_KEY              = local.use_kms_key,
-    KMS_KEY_ARN              = var.kms_key_arn
+    KMS_KEY_ARN              = var.offline_store_kms_key_arn != null ? var.offline_store_kms_key_arn : var.kms_key_arn,
   })
 }
 
@@ -142,11 +144,11 @@ resource "aws_iam_policy" "additional_rift_compute_policy" {
 locals {
   # Base set of policies to attach to the rift_compute role
   rift_compute_policies_base = {
-    rift_compute_logs    = aws_iam_policy.rift_compute_logs,
+    rift_compute_logs      = aws_iam_policy.rift_compute_logs,
     rift_bootstrap_scripts = aws_iam_policy.rift_bootstrap_scripts,
-    offline_store_access = aws_iam_policy.offline_store_access,
-    dynamo_db_access     = aws_iam_policy.rift_dynamodb_access,
-    ecr_readonly         = aws_iam_policy.rift_ecr_readonly
+    offline_store_access   = aws_iam_policy.offline_store_access,
+    dynamo_db_access       = aws_iam_policy.rift_dynamodb_access,
+    ecr_readonly           = aws_iam_policy.rift_ecr_readonly
   }
   # Include conditional policies when enabled/specified
   rift_compute_policies = merge(

rift_compute/variables.tf

@@ -78,6 +78,18 @@ variable "kms_key_arn" {
   default     = null
 }
 
+variable "online_store_kms_key_arn" {
+  type        = string
+  description = "ARN of KMS key used to encrypt online feature store. If given, will override the kms_key_arn."
+  default     = null
+}
+
+variable "offline_store_kms_key_arn" {
+  type        = string
+  description = "ARN of KMS key used to encrypt offline feature store. If given, will override the kms_key_arn."
+  default     = null
+}
+
 variable "use_network_firewall" {
   type        = bool
   default     = false
@@ -124,8 +136,8 @@ variable "tecton_privatelink_egress_rules" {
 variable "existing_vpc" {
   description = "Optional. Configuration for using an existing VPC. If provided, the module will not create a new VPC or related core networking resources (subnets, IGW, NAT GWs, Route Tables). Both vpc_id and private_subnet_ids must be provided together."
   type = object({
-    vpc_id               = string
-    private_subnet_ids   = list(string)
+    vpc_id             = string
+    private_subnet_ids = list(string)
   })
   default = null
 

templates/rift_dynamodb_access_policy.json

@@ -25,5 +25,16 @@
       ],
       "Resource": ["arn:aws:iam::${ACCOUNT_ID}:role/${CLUSTER_NAME}-cross-account-intermediate"]
     }
+    %{ if USE_KMS_KEY ~}
+    ,
+    {
+      "Effect": "Allow",
+      "Action": [
+        "kms:Decrypt",
+        "kms:GenerateDataKey"
+      ],
+      "Resource": ["${KMS_KEY_ARN}"]
+    }
+    %{ endif ~}
   ]
-} 
+}