"Blocked host" error after upgrading to 5.0.41

[hrak]

Chef Supermarket Version

5.0.41

Platform Details

Ubuntu 18.04.6 LTS

Configuration

private supermarket using supermarket-omnibus-cookbook

Scenario

Last night our supermarket was automatically upgraded to 5.0.41 by supermarket-omnibus-cookbook and now it throws the following error:

Blocked host: supermarket.example.com
To allow requests to supermarket.example.com, add the following to your environment configuration:

config.hosts << "supermarket.example.com"

It worked fine before the upgrade (version 4.2.89)

We are using the stable channel, i don't expect major upgrades to happen in a channel called stable.

Our config:

{
  "features": "tools, gravatar, collaborator_groups",
  "ssl": {
    "certificate": "/etc/ssl/wildcard_example_com.crt",
    "certificate_key": "/etc/ssl/wildcard_example_com.key",
    "ssl_dhparam": "/etc/ssl/ffdhe2048.pem",
    "protocols": "TLSv1.2",
    "ciphers": "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384"
  },
  "nginx": {
    "log_rotation": {
      "num_to_keep": 10
    }
  },
  "fqdn": "supermarket.example.com",
  "chef_server_url": "https://chef.example.com",
  "chef_oauth2_app_id": "<redacted>",
  "chef_oauth2_secret": "<redacted>",
  "chef_oauth2_verify_ssl": true
}

Steps to Reproduce

Upgrade a 4.2.89 installation using supermarket-omnibus-cookbook to 5.0.41

Expected Result

A working supermarket

Actual Result

red and black page with error mentioned above.

[hrak]

I noticed that v5.x.x adds export ALLOWED_HOST="fqdn.of.host" to the env. In our case the FQDN of the host does not match the public hostname of the supermarket. This is probably what's causing this.

[hrak]

I managed to solve this issue by explicitly setting node['supermarket_omnibus']['config']['allowed_host'] to my supermarket hostname (which i found here)

# Setting allowed host for supermarket to avoid arbitrary "Host" header injection
# Set this value to the domain name of your supermarket website e.g. supermarket.chef.io
default['supermarket']['allowed_host'] = node['supermarket']['fqdn']

it defaults to node['supermarket']['fqdn'], and although i do set node['supermarket_omnibus']['config']['fqdn'] to my supermarket hostname, it was not picked up as the default for default['supermarket']['allowed_host'], probably because of how the attributes are processed there (it defaults to (node['fqdn'] || node['hostname']).downcase):

# The fully qualified domain name. Will use the node's fqdn if nothing is
# specified.
default['supermarket']['fqdn'] = (node['fqdn'] || node['hostname']).downcase

[pdohertybcov]

I ran into something similar. I intentionally upgraded to 5.0.41, and updated my cookbook to add node.default['supermarket']['allowed_host'] before using the chef-ingredient cookbook to install supermarket, but it didn't seem to work, as I also got the same Blocked host error. There's a chance I just messed up my attribute precedence. But I tried manually added a normal attribute, and reconfiguring supermarket, and that didn't fix it either.

[pdohertybcov]

I eventually got this working, but later noticed that the AWS ELB healthchecks were all getting a 403. Before this change, the ELB health checks would get a 200, just doing a get on /.
I don't believe the ELB sets a Host Header at all. What' the appropriate way for an ELB healthcheck since this change was added?

[RajeshPaul38]

We have analyzed this issue and taken it up on priority. We are building a provision for allowing/disallowing host header injection. By allowing this the healthcheck APIs will not be blocked with status code 403. So it would be at customer's discretion to set this flag according to the need.

[bdwyertech]

#2698 adds support for multiple hostnames so you could use more than one

[RajeshPaul38]

@hrak @pdohertybcov @bdwyertech The following PR is to solve this issue:
#2733

We need to specify the private IP address of the supermarket instance along with the supermarket hostname in the list of allowed_hosts. This way the internal ELB healthcheck calls will not get blocked. We are also keeping this flag by default disabled to not get blocked due to the current error. In case someone wants to enable the protection they have to set the flag: disable_host_header_attack to true