(cloudformation-include): Cannot use computed outputs from nested stack

[ozeebee]

Describe the bug

Hi,
I must integrate some pre-built components (provided as CFN templates) in my app as nested stacks.
When I try to use computed outputs (whose value use an intrinsic function for instance) exposed by the nested stack in my CDK stack, deployment fails with message:

❌ Deployment failed: Error [ValidationError]: Template error: every Fn::Join object requires two parameters, (1) a string delimiter and (2) a list of strings to be joined or a function that returns a list of strings (such as Fn::GetAZs) to be joined.

It looks like the generated CFN is incorrect.
This is confirmed by running cloudformation validate-template with the CLI.

Expected Behavior

To be able to use nested stack's outputs in my CDK constructs.

Current Behavior

Invalid CFN template generates an error during deployment:

❌ Deployment failed: Error [ValidationError]: Template error: every Fn::Join object requires two parameters, (1) a string delimiter and (2) a list of strings to be joined or a function that returns a list of strings (such as Fn::GetAZs) to be joined.

Reproduction Steps

For instance, I have a CFN template creating some VPC Endpoint, exposing outputs like this:

  "Outputs": {
    "SimpleOutput": {
      "Description": "Simple Output",
      "Value": {
        "Ref": "VpcEndpoint"
      }
    },
    "ComputedOutput": {
      "Description": "Computed Output",
      "Value": {
        "Fn::Join": [
          ",",
          {
            "Fn::GetAtt": "VpcEndpoint.DnsEntries"
          }
        ]
      }
    }
  }

Now consider this stack:

interface MyNestedStackProps extends NestedStackProps {
  VpcId: string,
  SubnetIDs: string[]
}

class MyNestedStack extends NestedStack {
  readonly cfnInc: cfninc.CfnInclude
  constructor(scope: Construct, id: string, props: MyNestedStackProps) {
    super(scope, id)
    // Include CFN template
    this.cfnInc = new cfninc.CfnInclude(this, 'CfnTemplate', {
      templateFile: 'cfn-template.json',
      parameters: {
        'VpcId': props.VpcId,
        'SubnetIDs': props.SubnetIDs
      }
    })
  }
}

export interface CdkbugreproducerStackProps extends StackProps {
  vpcName: string
}

export class CdkbugreproducerStack extends Stack {
  constructor(scope: Construct, id: string, props: CdkbugreproducerStackProps) {
    super(scope, id, props)

    const vpc = Vpc.fromLookup(this, 'MyVPC', { vpcName: 'My-VPC' })
    const privateSubnetIds = vpc.privateSubnets.map(_ => _.subnetId)

    // Include CFN template as a NESTED STACK in the stack
    const nested = new MyNestedStack(this, 'MyNestedStack', {
      VpcId: vpc.vpcId,
      SubnetIDs: privateSubnetIds
    })
    const simpleOutput = nested.cfnInc.getOutput('SimpleOutput').value
    const computedOutput = nested.cfnInc.getOutput('ComputedOutput').value

    new ssm.StringParameter(this, 'MyParamSimpleOutput', {
      parameterName: 'MyParamFromStackSimpleOutput',
      stringValue: simpleOutput
    })
    new ssm.StringParameter(this, 'MyParamComputedOutput', {
      parameterName: 'MyParamFromStackComputedOutput',
      stringValue: computedOutput
    })
  }
}

So, in this simple example, I just want to get the DNS entries of the VPC Endpoint (created in the nested stack) and create an SSM parameter with its value, which fails.
In practice, we have similar error with other stacks exposing outputs with Fn::Sub, like for instance in:

"DomainEndpoint": {
    "Description": "ElasticSearch Service domain endpoint",
    "Value": {
        "Fn::Sub": "https://${ElasticSearch.DomainEndpoint}"
    }
},

Possible Solution

No idea. I don't even have work-around for this.

Additional Information/Context

No response

CDK CLI Version

2.96.2 (build 3edd240)

Framework Version

2.96.2

Node.js Version

v18.12.0

OS

macos

Language

Typescript

Language Version

3.9.10

Other information

I have a full reproducer project available in case it can help (can push it to GitHub or GitLab if needed).

Some observations after testing various things:

• This error does not happen when including the CFN template directly into the CDK stack, but I must add them as nested stacks (for compliance reasons)
• In the code above, If I comment the SSM parameter for "ComputedOutput", it works.

Here's the output of the synthesis for this "ComputedOutput" SSM parameter:

...
  "MyParamComputedOutput8C461DF5": {
   "Type": "AWS::SSM::Parameter",
   "Properties": {
    "Name": "MyParamFromStackComputedOutput",
    "Type": "String",
    "Value": {
     "Fn::Join": [
      ",",
      {
       "Fn::GetAtt": [
        "MyNestedStackNestedStackMyNestedStackNestedStackResource9C617903",
        "Outputs.cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11DnsEntries"
       ]
      }
     ]
    }
   },
...

And the generated CF for the outputs in the included nested stack looks like this:

 "Outputs": {
  "SimpleOutput": {
   "Description": "Simple Output",
   "Value": {
    "Ref": "VpcEndpoint"
   }
  },
  "ComputedOutput": {
   "Description": "Computed Output",
   "Value": {
    "Fn::Join": [
     ",",
     {
      "Fn::GetAtt": "VpcEndpoint.DnsEntries"
     }
    ]
   }
  },
  "cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11Ref": {
   "Value": {
    "Ref": "VpcEndpoint"
   }
  },
  "cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11DnsEntries": {
   "Value": {
    "Fn::GetAtt": "VpcEndpoint.DnsEntries"
   }
  }
 }

[ozeebee]

I continued experimenting on this issue.
I found out that actually, this is issue is not related to cfn-include but to nested stacks in CDK.
If I re-write this purely in CDK, this fails too:

interface VpcEndpointNestedStackProps extends NestedStackProps {
  vpc: ec2.IVpc
}

/** A simple nested stack that creates a VPC Endpoint and tries to export some of its properties */
export class VpcEndpointNestedStack extends NestedStack {
  readonly vpcEndpointId: string
  readonly vpcEndpointDnsEntries: string[]
  readonly vpcEndpointDnsEntriesStr: string

  constructor(scope: Construct, id: string, props: VpcEndpointNestedStackProps) {
    super(scope, id)

    const vpce = new ec2.InterfaceVpcEndpoint(this, 'VpcEndpoint', {
      vpc: props.vpc,
      service: ec2.InterfaceVpcEndpointAwsService.S3,
      privateDnsEnabled: false
    })

    this.vpcEndpointId = vpce.vpcEndpointId
    this.vpcEndpointDnsEntries = vpce.vpcEndpointDnsEntries
    this.vpcEndpointDnsEntriesStr = vpce.vpcEndpointDnsEntries.join(',')
  }
}

export interface CdkbugreproducerStackProps extends StackProps {
  vpcName: string
}

export class CdkbugreproducerStack extends Stack {
  constructor(scope: Construct, id: string, props: CdkbugreproducerStackProps) {
    super(scope, id, props)

    const vpc = Vpc.fromLookup(this, 'MyVPC', { vpcName: 'My-VPC' })

    const nested = new VpcEndpointNestedStack(this, 'MyNestedStack', { vpc })

    new CfnOutput(this, 'CdkSimpleOutput', { value: nested.vpcEndpointId }) // OK, the "simple" prop works as expected
    // new CfnOutput(this, 'CdkComputedOutput', { value: nested.vpcEndpointDnsEntries.join(',') }) // Error: Found an encoded list token string in a scalar string context. Use 'Fn.select(0, list)' (not 'list[0]') to extract elements from token lists.
    // new CfnOutput(this, 'CdkComputedOutput', { value: Fn.join(',', nested.vpcEndpointDnsEntries) }) // Template error: every Fn::Join object requires two parameters, (1) a string delimiter and (2) a list of strings to be joined or a function that returns a list of strings (such as Fn::GetAZs) to be joined.
    // new CfnOutput(this, 'CdkComputedOutput', { value: nested.vpcEndpointDnsEntriesStr }) // Error: Found an encoded list token string in a scalar string context. Use 'Fn.select(0, list)' (not 'list[0]') to extract elements from token lists.
    // new CfnOutput(this, 'CdkComputedOutput', { value: Fn.select(0, nested.vpcEndpointDnsEntries) }) // Template error: Fn::Select requires a list argument with two elements: an integer index and a list
    // new CfnOutput(this, 'CdkComputedOutput', { value: nested.vpcEndpointDnsEntries[0] }) // Error: Found an encoded list token string in a scalar string context. Use 'Fn.select(0, list)' (not 'list[0]') to extract elements from token lists.
  }
}

In the stack above, there is no way to get the DNS entries from the VPC Endpoint created in the nested stack.

[indrora]

Thank you for your report.

Can you include the generated clouformation from your reproduction here to help debug?

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[ozeebee]

Sure,
Here's the synthesized root stack CFN:

{
 "Resources": {
  "MyNestedStackNestedStackMyNestedStackNestedStackResource9C617903": {
   "Type": "AWS::CloudFormation::Stack",
   "Properties": {
    "TemplateURL": {
     "Fn::Join": [
      "",
      [
       "https://s3.eu-west-1.",
       {
        "Ref": "AWS::URLSuffix"
       },
       "/cdk-hnb659fds-assets-123456789012-eu-west-1/fde4170827a9cd567630c317b8970bc9fe5bcd4754144208c276d15a5cb56856.json"
      ]
     ]
    }
   },
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
    "aws:cdk:path": "cdk-bug-reproducer-cf/MyNestedStack.NestedStack/MyNestedStack.NestedStackResource",
    "aws:asset:path": "cdkbugreproducercfMyNestedStackDDCA32DC.nested.template.json",
    "aws:asset:property": "TemplateURL"
   }
  },
  "MyParamSimpleOutputD4138D08": {
   "Type": "AWS::SSM::Parameter",
   "Properties": {
    "Name": "MyParamFromStackSimpleOutput",
    "Type": "String",
    "Value": {
     "Fn::GetAtt": [
      "MyNestedStackNestedStackMyNestedStackNestedStackResource9C617903",
      "Outputs.cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11Ref"
     ]
    }
   },
   "Metadata": {
    "aws:cdk:path": "cdk-bug-reproducer-cf/MyParamSimpleOutput/Resource"
   }
  },
  "MyParamComputedOutput8C461DF5": {
   "Type": "AWS::SSM::Parameter",
   "Properties": {
    "Name": "MyParamFromStackComputedOutput",
    "Type": "String",
    "Value": {
     "Fn::Join": [
      ",",
      {
       "Fn::GetAtt": [
        "MyNestedStackNestedStackMyNestedStackNestedStackResource9C617903",
        "Outputs.cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11DnsEntries"
       ]
      }
     ]
    }
   },
   "Metadata": {
    "aws:cdk:path": "cdk-bug-reproducer-cf/MyParamComputedOutput/Resource"
   }
  },
  "CDKMetadata": {
   "Type": "AWS::CDK::Metadata",
   "Properties": {
    "Analytics": "v2:deflate64:H4sIAAAAAAAA/zPSszTTM1JMLC/WTU7J1s3JTNKrDi5JTM7WCUotzi8tSk7VcU7Lg4gAFcUXF+eCFBRl5qUHJBYl5qaWpBaBVMA5tSAeTG+tTl5+SqpeVrF+maGFnqGRnoFiVnFmpm5RaV5JZm6qXhCEBgAz2BWJhQAAAA=="
   },
   "Metadata": {
    "aws:cdk:path": "cdk-bug-reproducer-cf/CDKMetadata/Default"
   }
  }
 },
 "Parameters": {
  "BootstrapVersion": {
   "Type": "AWS::SSM::Parameter::Value<String>",
   "Default": "/cdk-bootstrap/hnb659fds/version",
   "Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"
  }
 },
 "Rules": {
  "CheckBootstrapVersion": {
   "Assertions": [
    {
     "Assert": {
      "Fn::Not": [
       {
        "Fn::Contains": [
         [
          "1",
          "2",
          "3",
          "4",
          "5"
         ],
         {
          "Ref": "BootstrapVersion"
         }
        ]
       }
      ]
     },
     "AssertDescription": "CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."
    }
   ]
  }
 }
}

And the nested stack CFN :

{
 "Description": "reproducer for CDK bug related to cfn outputs",
 "AWSTemplateFormatVersion": "2010-09-09",
 "Resources": {
  "VpcEndpoint": {
   "Type": "AWS::EC2::VPCEndpoint",
   "Properties": {
    "ServiceName": {
     "Fn::Sub": "com.amazonaws.${AWS::Region}.s3"
    },
    "SubnetIds": [
     "subnet-0ab01234556ccc123",
     "subnet-0b3482397dd0080d"
    ],
    "VpcEndpointType": "Interface",
    "VpcId": "vpc-0d2be234324ede232"
   }
  },
  "CDKMetadata": {
   "Type": "AWS::CDK::Metadata",
   "Properties": {
    "Analytics": "v2:deflate64:H4sIAAAAAAAA/yWLOw7CMBAFz5LeXogLBHVEQQMoSLSRsTfS5rOO7DUUiLtjSPXmjTQGDjswlX0l7fyoJ3rA+4xJ0N/EulG5KWTfhzhbocAdsZuyR2h6Pq2oStqhMz91vzZH9ksgFlXuJcuS/9RiCjk6/CgOpR7S5lnvoTawrYZEpGNmoRmhXfcLRYNAEZUAAAA="
   },
   "Metadata": {
    "aws:cdk:path": "cdk-bug-reproducer-cf/MyNestedStack/CDKMetadata/Default"
   }
  }
 },
 "Outputs": {
  "SimpleOutput": {
   "Description": "Simple Output",
   "Value": {
    "Ref": "VpcEndpoint"
   }
  },
  "ComputedOutput": {
   "Description": "Computed Output",
   "Value": {
    "Fn::Join": [
     ",",
     {
      "Fn::GetAtt": "VpcEndpoint.DnsEntries"
     }
    ]
   }
  },
  "cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11Ref": {
   "Value": {
    "Ref": "VpcEndpoint"
   }
  },
  "cdkbugreproducercfMyNestedStackCfnTemplateVpcEndpointC42D0C11DnsEntries": {
   "Value": {
    "Fn::GetAtt": "VpcEndpoint.DnsEntries"
   }
  }
 }
}

Hope this helps !