Only one level of includes is allowed

[cstavro]

I'm trying to enable a multi-include scenario where I can cascade includes at different points in my configurations. Given the below layout:

└── root
    ├── regions
    │   ├── us-east-1
    │   │   ├── regional_deployment(lambda)
    │   │   │      └── terraform.tfvars
    │   │   ├── config.tfvars
    │   │   └── terraform.tfvars
    │   └── us-east-2
    │       ├── regional_deployment(lambda)
    │       │      └── terraform.tfvars
    │       ├── config.tfvars
    │       └── terraform.tfvars
    ├── config.tfvars
    └── terraform.tfvars

I would like to be able to set the remote_state in the root terraform.tfvars while allowing for the notion of regional configurations. Some deployments are regional by nature (aws lambdas as an example) and so I think it makes sense to allow for this kind of modeling.

Right now I'm adding a separate extra_arguments block in each regional deployment's terraform.tfvars that includes required_var_files = ["${get_tfvars_dir()}/../config.tfvars]" . It works but it's not very DRY. Being able to have the regional terraform.tfvars leverage an include block to roll up to its parent would be even more so.

[brikis98]

Could you add the extra_arguments block in the root too?

[mike-schiller]

We have a similar use case with regions, platforms deployed in regions, and then apps deployed on those platforms. I hadn't thought of @cstavro 's approach, but will see if I can adapt it for our 3 tier use case.

That said, support for multiple levels of includes would be ideal. This was also referenced (along with deep merging) in @jocgir 's comment on: #147 . It seems some work has been done in that area, though in a fork.

As for the suggestion of putting the extra_arguments in the root, would that have to apply identically to everything in the tree? If so, I don't think that would work for our use case with the additional tier. Also, we do have some resources that are only 1 level deep, so those would have issues if extra_arguments in the root are applied identically to everything.

[brikis98]

As for the suggestion of putting the extra_arguments in the root, would that have to apply identically to everything in the tree? If so, I don't think that would work for our use case with the additional tier. Also, we do have some resources that are only 1 level deep, so those would have issues if extra_arguments in the root are applied identically to everything.

Yes, every child with an include would pull in all the extra_arguments in the root. That said, it doesn't mean every child gets identical behavior. Using some of the helper functions, the child nodes can have fairly dynamic behavior.

For example, in the root, we can define something like this:

terragrunt = {
  terraform {
    extra_arguments "common_vars" {
      commands = ["${get_terraform_commands_that_need_vars()}"]

      optional_var_files = [
        "${get_tfvars_dir()}/${find_in_parent_folders("account.tfvars", "skip-account-if-does-not-exist")}",
        "${get_tfvars_dir()}/${find_in_parent_folders("region.tfvars", "skip-region-if-does-not-exist")}",
        "${get_tfvars_dir()}/${find_in_parent_folders("env.tfvars", "skip-env-if-does-not-exist")}",
        "${get_tfvars_dir()}/terraform.tfvars"
      ]
    }
  }
}

Now every child will automatically find an account.tfvars, region.tfvars, and env.tfvars in its parent directory and skip that file if it doesn't exist.

[dan-rose]

Thanks for the above - Its just the answer I was looking for!!!

[tomdavidson]

im a bit green to terragrunt - can optional_var_files pull in a tf file that uses locals {} or only tfvars files?

can the optional_var_files contain terragrunt config such as all the regions use the dependencies defined in cross-region.tfvars which depend on root/terraform.tfvars?

└── root
├── regions
│ ├── us-east-1
│ │ ├── regional_deployment(lambda)
│ │ │ └── terraform.tfvars
│ │ ├── config.tfvars
│ │ └── terraform.tfvars
│ ├── us-east-2
│ | ├── regional_deployment(lambda)
│ | │ └── terraform.tfvars
│ | ├── config.tfvars
│ | └── terraform.tfvars
| └── cross-region.tfvars
├── config.tfvars
└── terraform.tfvars

[brikis98]

im a bit green to terragrunt - can optional_var_files pull in a tf file that uses locals {} or only tfvars files?

optional_var_files simply sets the -var-file param when running terraform. AFAIK, the -var-file param only supports .tfvars files.

[tomdavidson]

Great, thank you. This might be obvious too, but can terragrunt config be brought in with the optional_var_files or just terraform vars?

[brikis98]

optional_var_files currently only sets the -var-file flag and nothing else, so it cannot be used to bring in Terragrunt configuration.

[mazzy89]

I have more or less the same scenario however I'm not able to get to the bottom of this. I've the following dir struct:

.
├── account.tfvars
├── terraform.tfvars
└── test
    ├── _global
    │   └── iam
    │       └── terraform.tfvars
    ├── terraform.tfvars
    ├── test-blue
    │   └── frontend
    │       └── terraform.tfvars
    └── test-green
        └── frontend
            └── terraform.tfvars

basically under the test environment i have multiple environments blue and green. Both of them must share the same IAM resources. I have the following error when I run terragrunt plan from the frontend dir

test/test-blue/frontend/terraform.tfvars includes ../../terraform.tfvars, which itself includes ../../terraform.tfvars. Only one level of includes is allowed.

basically under the test-blue/frontend/terraform.tfvars I have this:

  include = {
    path = "${find_in_parent_folders()}"
  }

so it is under also test/terraform.tfvars

Update - 1
I was able to get rid off the issue by removing the include under test/terraform.tfvars and have there just terragrunt {} however now I can't retrieve the remote state config set up in the terraform.tfvars file in the root dir

Update - 2

I've removed the intermediary terraform.tfvars located under test/terraform.tfvars and repeated the shared config in each terraform.tfvars of every envs. In this case I lose the DRY but it works. I mean the not nice thing is that I have to repeat the common test vars both under test-blue and test-green tfvars files.

So I could say that the use case with the intermediary terraform.tfvars is not supported?

Here the repositories

https://github.com/mazzy89/terragrunt-poc-live
https://github.com/mazzy89/terragrunt-poc-modules

[mazzy89]

All good after spent the entire afternoon reading the PR and all the issues around the terragrunt repo is now clear. This is my commit that implement the shared file across the test env. mazzy89/terragrunt-poc-live@0b37b2a

[brikis98]

Yes, as the error indicates, only one level of includes is currently supported. This is mainly to keep the code simpler & more maintainable. I'm not against multiple levels, but someone would have to roll up their sleeves for a PR. In the mean time, extra_arguments offers a workaround for many use cases. Looks like you got that figured out.