cfn-lint hangs

[praf]

cfn-lint version: (cfn-lint --version)
cfn-lint 0.28.4

Platform
Darwin $hostname.domain.redacted$ 18.7.0 Darwin Kernel Version 18.7.0: Thu Jan 23 06:52:12 PST 2020; root:xnu-4903.278.25~1/RELEASE_X86_64 x86_64
MacOS Mojave 10.14.6

Also verified behavior in:
Linux $hostname.domain.redacted$ 4.9.184-0.1.ac.235.83.329.metal1.x86_64 #1 SMP Tue Jul 30 17:17:50 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Description of issue.
cfn-lint template.yml hangs indefinitely.

Please provide as much information as possible:

• Template linting issues:
  • Please provide a CloudFormation sample that generated the issue.

---
Resources:
  MyDashboard:
    Properties:
      DashboardBody:
        Fn::Sub:
        - " { \"widgets\": [{ \"type\": \"text\", \"width\": 24, \"height\": 1, \"properties\": {\"markdown\": \"# ${service_name} (${Stage} ${RegionCode} Cell-${Index})\"} },{ \"type\": \"text\", \"width\": 24, \"height\": 1, \"properties\": {\"markdown\": \"# API Gateway\"} } ,{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"AWS/ApiGateway\", \"Count\" ,\"ApiName\",\"${api_name}\" ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"API Call Count\", \"period\": 300,\"stat\": \"Sum\"} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"AWS/ApiGateway\", \"Latency\" ,\"ApiName\",\"${api_name}\" ,{\"label\": \"p50\",\"stat\": \"p50\",\"yAxis\": \"left\"} ],[ \"AWS/ApiGateway\", \"Latency\" ,\"ApiName\",\"${api_name}\" ,{\"label\": \"p90\",\"stat\": \"p90\",\"yAxis\": \"left\"} ],[ \"AWS/ApiGateway\", \"Latency\" ,\"ApiName\",\"${api_name}\" ,{\"label\": \"p99\",\"stat\": \"p99\",\"yAxis\": \"left\"} ],[ \"AWS/ApiGateway\", \"Latency\" ,\"ApiName\",\"${api_name}\" ,{\"label\": \"p100\",\"stat\": \"p100\",\"yAxis\": \"left\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"Latency\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"AWS/ApiGateway\", \"IntegrationLatency\" ,\"ApiName\",\"${api_name}\" ,{\"label\": \"p50\",\"stat\": \"p50\",\"yAxis\": \"left\"} ],[ \"AWS/ApiGateway\", \"IntegrationLatency\" ,\"ApiName\",\"${api_name}\" ,{\"label\": \"p90\",\"stat\": \"p90\",\"yAxis\": \"left\"} ],[ \"AWS/ApiGateway\", \"IntegrationLatency\" ,\"ApiName\",\"${api_name}\" ,{\"label\": \"p99\",\"stat\": \"p99\",\"yAxis\": \"left\"} ],[ \"AWS/ApiGateway\", \"IntegrationLatency\" ,\"ApiName\",\"${api_name}\" ,{\"label\": \"p100\",\"stat\": \"p100\",\"yAxis\": \"left\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"Integration Latency\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"AWS/ApiGateway\", \"4XXError\" ,\"ApiName\",\"${api_name}\" ],[ \"AWS/ApiGateway\", \"5XXError\" ,\"ApiName\",\"${api_name}\" ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"Error\", \"period\": 300,\"stat\": \"Sum\"} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[\n\n{\"expression\": \"SEARCH(\\u0027{AWS/ApiGateway,ApiName,Method,Resource,Stage} ApiName=\\\"${api_name}\\\" (\\\"4XXError\\\" OR \\\"5XXError\\\")\\u0027, \\u0027Sum\\u0027, 300)\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"API Method Errors\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[\n\n{\"expression\": \"SEARCH(\\u0027{AWS/ApiGateway,ApiName,Method,Resource,Stage} ApiName=${api_name} MetricName=\\\"Latency\\\"\\u0027, \\u0027p90\\u0027, 300)\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"API Method p90 Latency\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[\n\n{\"expression\": \"SEARCH(\\u0027{AWS/ApiGateway,ApiName,Method,Resource,Stage} ApiName=${api_name} MetricName=\\\"Latency\\\"\\u0027, \\u0027p99\\u0027, 300)\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"API Method p99 Latency\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[\n\n{\"expression\": \"SEARCH(\\u0027{AWS/ApiGateway,ApiName,Method,Resource,Stage} ApiName=${api_name} MetricName=\\\"IntegrationLatency\\\"\\u0027, \\u0027p90\\u0027, 300)\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"API Method p90 Integration Latency\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[\n\n{\"expression\": \"SEARCH(\\u0027{AWS/ApiGateway,ApiName,Method,Resource,Stage} ApiName=${api_name} MetricName=\\\"IntegrationLatency\\\"\\u0027, \\u0027p99\\u0027, 300)\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"API Method p99 Integration Latency\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[\n\n{\"expression\": \"SEARCH(\\u0027{AWS/ApiGateway,ApiName,Method,Resource,Stage} ApiName=\\\"${api_name}\\\" MetricName=\\\"Count\\u0027, \\u0027Sum\\u0027, 300)\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"API Method Call Count\", \"period\": 300} },{ \"type\": \"text\", \"width\": 24, \"height\": 1, \"properties\": {\"markdown\": \"# ${service_name} Lambda\"} } ,{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"AWS/Lambda\", \"Invocations\" ,\"FunctionName\",\"${api_lambda}\" ,{\"label\": \"Count\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"Invocations\", \"period\": 300,\"stat\": \"Sum\"} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"AWS/Lambda\", \"Duration\" ,\"FunctionName\",\"${api_lambda}\" ,{\"label\": \"p50\",\"stat\": \"p50\",\"yAxis\": \"left\"} ],[ \"AWS/Lambda\", \"Duration\" ,\"FunctionName\",\"${api_lambda}\" ,{\"label\": \"p90\",\"stat\": \"p90\",\"yAxis\": \"left\"} ],[ \"AWS/Lambda\", \"Duration\" ,\"FunctionName\",\"${api_lambda}\" ,{\"label\": \"p99\",\"stat\": \"p99\",\"yAxis\": \"left\"} ],[ \"AWS/Lambda\", \"Duration\" ,\"FunctionName\",\"${api_lambda}\" ,{\"label\": \"p100\",\"stat\": \"p100\",\"yAxis\": \"left\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"Duration\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"AWS/Lambda\", \"Errors\" ,\"FunctionName\",\"${api_lambda}\" ,{\"label\": \"Count\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"Errors\", \"period\": 300,\"stat\": \"Sum\"} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"AWS/Lambda\", \"Throttles\" ,\"FunctionName\",\"${api_lambda}\" ,{\"label\": \"Count\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"Throttles\", \"period\": 300,\"stat\": \"Sum\"} },{ \"type\": \"text\", \"width\": 24, \"height\": 1, \"properties\": {\"markdown\": \"# ${service_name} Lambda by API : POST_/object/put\"} } ,{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"${service_name}.APILambda\", \"SomeObject.SomeMethod\" ,\"metricType\",\"timerStats\",\"EntityType\",\"some.class.path.SomeObject\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/put\" ,{\"label\": \"p50\",\"stat\": \"p50\"} ],[ \"${service_name}.APILambda\", \"SomeObject.SomeMethod\" ,\"metricType\",\"timerStats\",\"EntityType\",\"some.class.path.SomeObject\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/put\" ,{\"label\": \"p90\",\"stat\": \"p90\"} ],[ \"${service_name}.APILambda\", \"SomeObject.SomeMethod\" ,\"metricType\",\"timerStats\",\"EntityType\",\"some.class.path.SomeObject\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/put\" ,{\"label\": \"p99\",\"stat\": \"p99\"} ],[ \"${service_name}.APILambda\", \"SomeObject.SomeMethod\" ,\"metricType\",\"timerStats\",\"EntityType\",\"some.class.path.SomeObject\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/put\" ,{\"label\": \"p100\",\"stat\": \"p100\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"SomeObject.SomeMethod (ms) - Object\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"${service_name}.APILambda\", \"LambdaContainerMetrics.newLambdaContainer\" ,\"metricType\",\"counterCount\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/put\" ,{\"label\": \"Count\",\"stat\": \"Sum\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"LambdaContainerMetrics.newLambdaContainer\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"${service_name}.APILambda\", \"LambdaContainerMetrics.lambdaContainerAge\" ,\"metricType\",\"timerStats\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/put\" ,{\"label\": \"p50\",\"stat\": \"p50\"} ],[ \"${service_name}.APILambda\", \"LambdaContainerMetrics.lambdaContainerAge\" ,\"metricType\",\"timerStats\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/put\" ,{\"label\": \"p90\",\"stat\": \"p90\"} ],[ \"${service_name}.APILambda\", \"LambdaContainerMetrics.lambdaContainerAge\" ,\"metricType\",\"timerStats\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/put\" ,{\"label\": \"p99\",\"stat\": \"p99\"} ],[ \"${service_name}.APILambda\", \"LambdaContainerMetrics.lambdaContainerAge\" ,\"metricType\",\"timerStats\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/put\" ,{\"label\": \"p100\",\"stat\": \"p100\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"LambdaContainerMetrics.lambdaContainerAge\", \"period\": 300} },{ \"type\": \"text\", \"width\": 24, \"height\": 1, \"properties\": {\"markdown\": \"# ${service_name} Lambda by API : POST_/object/load\"} } ,{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"${service_name}.APILambda\", \"SomeObject.SomeMethod\" ,\"metricType\",\"timerStats\",\"EntityType\",\"some.class.path.SomeObject\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/load\" ,{\"label\": \"p50\",\"stat\": \"p50\"} ],[ \"${service_name}.APILambda\", \"SomeObject.SomeMethod\" ,\"metricType\",\"timerStats\",\"EntityType\",\"some.class.path.SomeObject\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/load\" ,{\"label\": \"p90\",\"stat\": \"p90\"} ],[ \"${service_name}.APILambda\", \"SomeObject.SomeMethod\" ,\"metricType\",\"timerStats\",\"EntityType\",\"some.class.path.SomeObject\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/load\" ,{\"label\": \"p99\",\"stat\": \"p99\"} ],[ \"${service_name}.APILambda\", \"SomeObject.SomeMethod\" ,\"metricType\",\"timerStats\",\"EntityType\",\"some.class.path.SomeObject\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/load\" ,{\"label\": \"p100\",\"stat\": \"p100\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"SomeObject.SomeMethod (ms) - Object\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"${service_name}.APILambda\", \"LambdaContainerMetrics.newLambdaContainer\" ,\"metricType\",\"counterCount\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/load\" ,{\"label\": \"Count\",\"stat\": \"Sum\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"LambdaContainerMetrics.newLambdaContainer\", \"period\": 300} },{ \"type\": \"metric\", \"properties\": { \"region\": \"${AWS::Region}\", \"view\": \"timeSeries\", \"stacked\": false, \"metrics\": [[ \"${service_name}.APILambda\", \"LambdaContainerMetrics.lambdaContainerAge\" ,\"metricType\",\"timerStats\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/load\" ,{\"label\": \"p50\",\"stat\": \"p50\"} ],[ \"${service_name}.APILambda\", \"LambdaContainerMetrics.lambdaContainerAge\" ,\"metricType\",\"timerStats\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/load\" ,{\"label\": \"p90\",\"stat\": \"p90\"} ],[ \"${service_name}.APILambda\", \"LambdaContainerMetrics.lambdaContainerAge\" ,\"metricType\",\"timerStats\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/load\" ,{\"label\": \"p99\",\"stat\": \"p99\"} ],[ \"${service_name}.APILambda\", \"LambdaContainerMetrics.lambdaContainerAge\" ,\"metricType\",\"timerStats\",\"stage\",\"${Stage}\" ,\"region\",\"${Region}\" ,\"environmentIndex\",\"${Index}\",\"operation\",\"POST_/object/load\" ,{\"label\": \"p100\",\"stat\": \"p100\"} ]], \"yAxis\": {\"left\": {\"min\": 0}}, \"title\": \"LambdaContainerMetrics.lambdaContainerAge\", \"period\": 300} }] } "
        - api_name: myserviceapi
    Type: AWS::CloudWatch::Dashboard

• Repro Instructions:
  • Save the above template to a file named template.yml
  • run cfn-lint template.yml
  • ... hangs ... (waited for 5 minutes)
  • on ctrl-c:

^CTraceback (most recent call last):
  File "/usr/local/bin/cfn-lint", line 11, in <module>
    load_entry_point('cfn-lint==0.28.4', 'console_scripts', 'cfn-lint')()
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/__main__.py", line 23, in main
    cfnlint.core.run_cli(
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/core.py", line 45, in run_cli
    return run_checks(filename, template, rules, regions, mandatory_rules)
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/core.py", line 186, in run_checks
    matches.extend(runner.run())
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/__init__.py", line 1017, in run
    self.rules.run(
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/__init__.py", line 329, in run
    self.run_check(
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/__init__.py", line 203, in run_check
    return check(*args)
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/__init__.py", line 89, in wrapper
    results = match_function(self, filename, cfn, *args, **kwargs)
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/__init__.py", line 116, in matchall
    return self.match(cfn)  # pylint: disable=E1102
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/functions/SubNeeded.py", line 116, in match
    parameter_string_paths = self.match_values(parameter_search, cfn)
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/functions/SubNeeded.py", line 89, in match_values
    results.extend(self._match_values(searchRegex, cfn.template, []))
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/functions/SubNeeded.py", line 68, in _match_values
    values.extend(self._match_values(searchRegex, cfnelem[key], pathprop))
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/functions/SubNeeded.py", line 68, in _match_values
    values.extend(self._match_values(searchRegex, cfnelem[key], pathprop))
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/functions/SubNeeded.py", line 68, in _match_values
    values.extend(self._match_values(searchRegex, cfnelem[key], pathprop))
  [Previous line repeated 2 more times]
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/functions/SubNeeded.py", line 73, in _match_values
    values.extend(self._match_values(searchRegex, item, pathprop))
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/site-packages/cfnlint/rules/functions/SubNeeded.py", line 76, in _match_values
    if isinstance(cfnelem, str) and re.match(searchRegex, cfnelem):
  File "/usr/local/Cellar/cfn-lint/0.28.4_1/libexec/lib/python3.8/re.py", line 189, in match
    return _compile(pattern, flags).match(string)
KeyboardInterrupt

[PatMyron]

@praf thanks for that traceback and template! Looks like ReDoS here:

https://github.com/aws-cloudformation/cfn-python-lint/blob/3d2294d4f7c00aaaba787cc1289ac1a8c6e94106/src/cfnlint/rules/functions/SubNeeded.py#L76

"Unescaped version doesn’t hang. Escaped version hangs."

[kddejong]

specifically it looks like re.match is causing the larger string to hang.