v1 - Using Resource Provider Schemas

[kddejong]

Description

The effort to go to v1 will be driven by the goal to convert from CloudFormation specs to CloudFormation resource provider schemas. This will be a large change for how cfn-lint works and will result in rules having to be updated and changed. This issue will also serve to communicate the migration efforts.

Details

The CloudFormation resource provider schema is based on JSON Schema draft-07 but has modifications to handle the CloudFormation service These schemas allow us to do more straight JSON schema validation against resource properties. There are modifications to the schema and how the JSON schema validators work to handle CloudFormation specific capabilities. We find it important to provide the functionality and features that cfn-lint has had in v0 including the ability to disable and configure rules. As a result we will integrate in JSON schema validation into cfn-lint and cfn-lint will provide the functionality to massage the schemas and handle CloudFormation specific capabilities. Additionally there are checks in cfn-lint (best practices, etc.) that cannot be written into JSON schema validation.

Rule changes

All rules that have been modified or where the logic will change:

• E1027 - DynamicReferenceSecureString - Depended on match_resource_sub_properties. Was re-written to use match
• E1010 - GetAtt - Used the specs for valid GetAtt values
• E1022 - Join - Used the specs to determine the type of a GetAtt
• E1019 - Sub - Used the specs to determine the type of a GetAtt
• E1029 - SubNeeded - Used the specs to determine valid GetAtt values
• E6003 - Value (Outputs) - Used the specs to determine the type of a GetAtt
• W2031 - AllowedPattern (Parameters) - Replaced by JSON Schema validation logic
• W2030 - AllowedValue (Parameters) - Replaced by JSON Schema validation logic
• E3001 - Configuration (Resources) - Replaced by JSON Schema validation. Logic is outside of the registry
• E3000 - JsonSchema (Resources) - Expanded to handle registry resource schemas. Parent rule for all JSON schema validation rules
• E3031 - AllowedPattern (Resource/Properties) - Replaced by JSON Schema validation logic
• E3030 - AllowedValue (Resource/Properties) - Replaced by JSON Schema validation logic
• E2522 - AtLeastOne (Resource/Properties) - Deleted?
• E3017 - CfnSchema (Resource/Properties) (New Rule) - New rule to extend JSON Schema validation to handle scenarios not covered by the registry schema
• E2520 - Exclusive (Resource/Properties) - Deleted. Handled by if/then/else logic in JSON schema validation
• E2521 - Inclusive (Resource/Properties) - Deleted. Handled by if/then/else logic in JSON schema validation
• E3502 - JsonSize (Resource/Properties) - Deleted. Converted to string maxLength and minLength validation
• E3037 - ListDuplicates (Resource/Properties) - Replaced by JSON Schema validation logic
• I3037 - ListDuplicatesAllowed (Resource/Properties) - Replaced by JSON Schema validation logic
• E3032 - ListSize (Resource/Properties) - Replaced by JSON Schema validation logic
• E3034 - NumberSize (Resource/Properties) - Replaced by JSON Schema validation logic
• E2523 - OnlyOne (Resource/Properties) - Replaced by oneOf logic in JSON schema
• E3002 - Properties (Resource/Properties) - Replaced by properties and additionalProperties in JSON schema
• E3003 - Required (Resource/Properties) - Replaced by required in JSON schema
• E3017 - RequiredBasedOnValue (Resource/Properties) - Replaced by if/then/else logic using cfnSchema and JSON schema
• E3033 - StringSize (Resource/Properties) - Replaced by minLength and maxLength in JSON schema. May still need to add exceptions for dynamic references
• E3018 - UnwantedBasedOnValue (Resource/Properties) - Replaced by if/then/else logic using cfnSchema and JSON schema
• E3012 - ValuePrimitiveType (Resource/Properties) - Replaced by type in JSON schema
• E3008 - ValueRefGetAtt (Resource/Properties) - WIP

Breaking changes

This will serve as a list of changes that will occur in the migration from v0 to v1

• match_resource_sub_properties will be deprecated. This function was based on the specs and is not easily converted into the resource provider schema approach

Issues

• Some resource schemas have readOnlyProperties that are still write-able. Since we are removing them for validation we need to know the list of exceptions
• Packaging doesn't like linked files so they are dropped. Need a different approach to handling marking resources as cached

Action Items

• Convert custom or 3rd party registry schema validation to the new rule E3000
• Don't run property checks multiple times if the specs across regions are cached
• Convert enum values taken from boto
• Convert other manually created allowedvalues, patterns, etc.
• Write tests to validate JSON Schema docs and extensions
• Add in AWS::CDK::Metadata schema

[PatMyron]

• match_resource_sub_properties will be deprecated

undocumented and no other public repos show up in sourcegraph search using that, which bodes well

[kddejong]

When dealing with oneOf, allOf, anyOf, if/then/else logic JSON schema will fail with a rolled up error and the context will include all the errors determined from the sub schemas. While this works it hides some of the information for the user to determine what actually caused their issues and how to fix it.

Do not raise any generic errors for these types
allOf create multiple errors for all sub errors
anyOf create multiple errors for all sub errors
oneOf return the best error
if/then/else return the best error

[whoDoneItAgain]

i saw the rc1 released yesterday. i ran it against one my pipelines CF stacks. It returned a many findings, though basically the same 3-5 findings repeatedly. Where would you like the feedback?

[kddejong]

I would love the feedback. I've been trying to find as many templates as I can to run against behind the scenes but I'm running out of additional issues to chase. Thanks!

[whoDoneItAgain]

E3001 and E3030 are the most seen errors. Below are the unique errors and a snippet example.

E3030 - {'Fn::FindInMap': ['StaticParams', 'S3EmptierLambda', 'LambdaOSArchitechture']} is not one of ['x86_64', 'arm64']

LambdaLogGroup:
Type: AWS::Logs::LogGroup
DeletionPolicy: Delete
UpdateReplacePolicy: Delete
Properties:
LogGroupName: !Sub '/aws/lambda/${Region}-${VPCName}-${AppName}-${LambdaPurpose}'
RetentionInDays: !FindInMap [GlobalSettings, GlobalSettings, LogRetention]`

E3001 - {'Fn::If': ['ConditionDev', 'Delete', 'Retain']} is not one of ['Delete', 'Retain', 'Snapshot']

E3001 - {'Fn::If': ['ConditionDev', 'Delete', 'Retain']} is not of type 'string'

ALBLogsBucket:
Type: AWS::S3::Bucket
DeletionPolicy: !If
- ConditionDev
- Delete
- Retain
UpdateReplacePolicy: !If
- ConditionDev
- Delete
- Retain
Properties:

E0002 - Unknown exception while processing rule E6003: Resource type 'Custom::R53EndpointIp' is not found in 'us-east-1'

E1010 - 'EndpointIds' is not one of ['FirewallId', 'ResourceArn'] (This is an interesting one. Erroring on a custom resource)

NetworkFirewallVpceAz1:
Type: Custom::FirewallVpceFinder
Condition: ConditionCreatePublicSubnets
Properties:
ServiceToken:
Fn::ImportValue: !Sub '${Region}-${VpcName}-lambda-firewall-vpce-finder-fn-arn'
EndpointIds: !GetAtt NetworkFirewall.EndpointIds
AvailabilityZone: !GetAtt FirewallSubnet1.AvailabilityZone

E3032 - [] is too short

(Its barking about the Regions section)

ClientVpnAuthStackSet:
Type: AWS::CloudFormation::StackSet
Condition: ConditionCreateCrossRegionAuthRules
Properties:
...
StackInstancesGroup:
- DeploymentTargets:
Accounts:
- !Ref AWS::AccountId
Regions:
- !If
- ConditionCreateAuthorizationRulesUse1
- us-east-1
- !Ref AWS::NoValue
- !If
- ConditionCreateAuthorizationRulesUse2
- us-east-2
- !Ref AWS::NoValue
- !If
- ConditionCreateAuthorizationRulesUsw2
- us-west-2
- !Ref AWS::NoValue
- !If
- ConditionCreateAuthorizationRulesAps1
- ap-south-1
- !Ref AWS::NoValue

E3012 - Specify only 'SubnetMappings' or 'Subnets'

ApplicationLoadBalancer:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Condition: ConditionCreateLambda
Properties:
Type: application
Name: !Sub '${Region}-cross-vpc-testing-alb'
Scheme: internal
SecurityGroups:
- !Ref ALBSecurityGroup
Subnets:
- Fn::ImportValue: !Sub '${Region}-${VPCName}-private-subnet-1-id'
- Fn::ImportValue: !Sub '${Region}-${VPCName}-private-subnet-2-id'
- !If
- ConditionAzCount3
- Fn::ImportValue: !Sub '${Region}-${VPCName}-private-subnet-3-id'
- !Ref AWS::NoValue

[kddejong]

Action items:

• Language extension schema changes. This will handle E3001 conditions in the DeletionPolicy issue. Docs
• Update enum, pattern, type to handle functions
• Fix an issue with GetAtts where we dropped arrays that have a ref in the items portion of the schema
• E3012 - handle intrinsic functions in cfnSchema validation
• Update conditions logic to include resource or output level Condition scenarios

I'll add more details as I investigate and figure out the solutions.