from-to-www-redirect not working with a wildcard

[artemzakharov]

Is this a request for help?: Request for bug fix

What keywords did you search in NGINX Ingress controller issues before filing this one?: wildcard www redirect

---

Is this a BUG REPORT or FEATURE REQUEST?: Bug

NGINX Ingress controller version: not sure, installed via the latest chart on Helm yesterday

Kubernetes version (use kubectl version): 1.9.4-gke.1

Environment:

• Cloud provider or hardware configuration: GCP
• OS (e.g. from /etc/os-release): ContainerOS
• Kernel (e.g. uname -a):
• Install tools: Helm
• Others:

What happened:
I have a wildcard certificate that was issued to *.foo.com, in all lowercase. I have set up the ingress like this, with the intention of redirecting all naked domain traffic to www.foo.com, which the certificate covers.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: foo-https-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
spec:
  tls:
      - hosts:
          - foo.com
          - www.foo.com
        secretName: tls-wildcard-secret  
  rules:
    - host: foo.com
      http:
        paths:
          - backend:
              serviceName: foo-prod
              servicePort: 80
            path: /
    - host: www.foo.com
      http:
        paths:
          - backend:
              serviceName: foo-prod
              servicePort: 80
            path: /

The ingress, however, does not redirect traffic from the base domain - going to foo.com in the browser leaves me at https://foo.com instead of https://www.foo.com, and no redirect takes place.

What you expected to happen:
I expected all traffic going to the naked domain to be redirected.

How to reproduce it (as minimally and precisely as possible):

1. Get a wildcard certificate for a domain
2. Configure ingress as I have it
3. Attempt to go to the base domain

[oilbeater]

@artemzakharov there is a bug in redirect, you can try oilbeater/nginx-ingress-controller-amd64:0.12.0 to see if it's ok now

[artemzakharov]

@oilbeater thank you, how can I try it out? I assume I need to reinstall with helm?

[oilbeater]

@artemzakharov you can edit the ingress-controller deployment and change the image name

[artemzakharov]

@oilbeater I tried out the image, still doesn't redirect.

[oilbeater]

can you paste the /etc/nginx/nginx.conf in the container,there should be a section like
` server {

    listen 80;
    listen 443 ssl;

    listen [::]:80;
    listen [::]:443;

    server_name foo.com;

    return 308 $scheme://www.foo.com$request_uri;

}`

[artemzakharov]

Wasn't sure which one you wanted since there were many server sections, so here's the whole conf file.

daemon off;
worker_processes 1;
pid /run/nginx.pid;
worker_rlimit_nofile 1047552;
worker_shutdown_timeout 10s ;
events {
    multi_accept        on;
    worker_connections  16384;
    use                 epoll;
}
http {
    lua_package_cpath "/usr/local/lib/lua/?.so;/usr/lib/x86_64-linux-gnu/lua/5.1/?.so;;";
    lua_package_path "/etc/nginx/lua/?.lua;/etc/nginx/lua/vendor/?.lua;/usr/local/lib/lua/?.lua;;";
    lua_shared_dict configuration_data 5M;
    lua_shared_dict round_robin_state 1M;
    lua_shared_dict locks 512k;
    init_by_lua_block {
        require("resty.core")
        collectgarbage("collect")

        -- init modules
        local ok, res

        ok, res = pcall(require, "configuration")
        if not ok then
          error("require failed: " .. tostring(res))
        else
          configuration = res
        end

        ok, res = pcall(require, "balancer")
        if not ok then
          error("require failed: " .. tostring(res))
        else
          balancer = res
        end
    }

    init_worker_by_lua_block {
        balancer.init_worker()
    }

    real_ip_header      X-Forwarded-For;

    real_ip_recursive   on;

    set_real_ip_from    0.0.0.0/0;

    geoip_country       /etc/nginx/geoip/GeoIP.dat;
    geoip_city          /etc/nginx/geoip/GeoLiteCity.dat;
    geoip_org           /etc/nginx/geoip/GeoIPASNum.dat;
    geoip_proxy_recursive on;

    aio                 threads;
    aio_write           on;

    tcp_nopush          on;
    tcp_nodelay         on;

    log_subrequest      on;

    reset_timedout_connection on;

    keepalive_timeout  75s;
    keepalive_requests 100;

    client_header_buffer_size       1k;
    client_header_timeout           60s;
    large_client_header_buffers     4 8k;
    client_body_buffer_size         8k;
    client_body_timeout             60s;

    http2_max_field_size            4k;
    http2_max_header_size           16k;

    types_hash_max_size             2048;
    server_names_hash_max_size      1024;
    server_names_hash_bucket_size   64;
    map_hash_bucket_size            64;

    proxy_headers_hash_max_size     512;
    proxy_headers_hash_bucket_size  64;

    variables_hash_bucket_size      128;
    variables_hash_max_size         2048;

    underscores_in_headers          off;
    ignore_invalid_headers          on;

    limit_req_status                503;

    include /etc/nginx/mime.types;
    default_type text/html;

    gzip on;
    gzip_comp_level 5;
    gzip_http_version 1.1;
    gzip_min_length 256;
    gzip_types application/atom+xml application/javascript application/x-javascript application/json application/rss+xml application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/svg+xml image/x-icon text/css text/plain text/x-component;
    gzip_proxied any;
    gzip_vary on;

    # Custom headers for response

    server_tokens on;

    # disable warnings
    uninitialized_variable_warn off;

    # Additional available variables:
    # $namespace
    # $ingress_name
    # $service_name
    log_format upstreaminfo '$the_real_ip - [$the_real_ip] - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time [$proxy_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status';

    map $request_uri $loggable {

        default 1;
    }

    access_log /var/log/nginx/access.log upstreaminfo if=$loggable;

    error_log  /var/log/nginx/error.log notice;

    resolver 10.59.240.10 valid=30s;

    # Retain the default nginx handling of requests without a "Connection" header
    map $http_upgrade $connection_upgrade {
        default          upgrade;
        ''               close;
    }

    map $http_x_forwarded_for $the_real_ip {

        default          $remote_addr;

    }

    # trust http_x_forwarded_proto headers correctly indicate ssl offloading
    map $http_x_forwarded_proto $pass_access_scheme {
        default          $http_x_forwarded_proto;
        ''               $scheme;
    }

    # validate $pass_access_scheme and $scheme are http to force a redirect
    map "$scheme:$pass_access_scheme" $redirect_to_https {
        default          0;
        "http:http"      1;
        "https:http"     1;
    }

    map $http_x_forwarded_port $pass_server_port {
        default           $http_x_forwarded_port;
        ''                $server_port;
    }

    map $pass_server_port $pass_port {
        443              443;
        default          $pass_server_port;
    }

    # Obtain best http host
    map $http_host $this_host {
        default          $http_host;
        ''               $host;
    }

    map $http_x_forwarded_host $best_http_host {
        default          $http_x_forwarded_host;
        ''               $this_host;
    }

    server_name_in_redirect off;
    port_in_redirect        off;

    rewrite_log             on;

    ssl_protocols TLSv1.2;

    # turn on session caching to drastically improve performance

    ssl_session_cache builtin:1000 shared:SSL:10m;
    ssl_session_timeout 10m;

    # allow configuring ssl session tickets
    ssl_session_tickets on;

    # slightly reduce the time-to-first-byte
    ssl_buffer_size 4k;

    # allow configuring custom ssl ciphers
    ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
    ssl_prefer_server_ciphers on;

    ssl_ecdh_curve auto;

    proxy_ssl_session_reuse on;

    upstream default-foomainsite-prod-80 {
        least_conn;

        keepalive 32;

        server 10.56.2.25:80 max_fails=0 fail_timeout=0;

    }

    upstream upstream-default-backend {
        least_conn;

        keepalive 32;

        server 10.56.0.9:8080 max_fails=0 fail_timeout=0;

    }

    upstream upstream_balancer {
        server 0.0.0.1; # placeholder

        balancer_by_lua_block {
          balancer.call()
        }

        keepalive 32;

    }

    ## start server _
    server {
        server_name _ ;

        listen 80 default_server  backlog=511;

        listen [::]:80 default_server  backlog=511;

        set $proxy_upstream_name "-";

        listen 443  default_server  backlog=511 ssl http2;

        listen [::]:443  default_server  backlog=511 ssl http2;

        # PEM sha: aa618bf3921dc2efc0f9b4b3a84f67862b1c96be
        ssl_certificate                         /ingress-controller/ssl/default-fake-certificate.pem;
        ssl_certificate_key                     /ingress-controller/ssl/default-fake-certificate.pem;

        location / {

            if ($scheme = https) {
            more_set_headers                        "Strict-Transport-Security: max-age=15724800; includeSubDomains;";
            }

            access_log off;

            port_in_redirect off;

            set $proxy_upstream_name "upstream-default-backend";

            set $namespace      "";
            set $ingress_name   "";
            set $service_name   "";

            client_max_body_size                    "1m";

            proxy_set_header Host                   $best_http_host;

            # Pass the extracted client certificate to the backend

            proxy_set_header ssl-client-cert        "";
            proxy_set_header ssl-client-verify      "";
            proxy_set_header ssl-client-subject-dn  "";
            proxy_set_header ssl-client-issuer-dn   "";

            # Allow websocket connections
            proxy_set_header                        Upgrade           $http_upgrade;

            proxy_set_header                        Connection        $connection_upgrade;

            proxy_set_header X-Real-IP              $the_real_ip;

            proxy_set_header X-Forwarded-For        $the_real_ip;

            proxy_set_header X-Forwarded-Host       $best_http_host;
            proxy_set_header X-Forwarded-Port       $pass_port;
            proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
            proxy_set_header X-Original-URI         $request_uri;
            proxy_set_header X-Scheme               $pass_access_scheme;

            # Pass the original X-Forwarded-For
            proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;

            # mitigate HTTPoxy Vulnerability
            # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
            proxy_set_header Proxy                  "";

            # Custom headers to proxied server

            proxy_connect_timeout                   5s;
            proxy_send_timeout                      60s;
            proxy_read_timeout                      60s;

            proxy_buffering                         "off";
            proxy_buffer_size                       "4k";
            proxy_buffers                           4 "4k";
            proxy_request_buffering                 "on";

            proxy_http_version                      1.1;

            proxy_cookie_domain                     off;
            proxy_cookie_path                       off;

            # In case of errors try the next upstream server before returning an error
            proxy_next_upstream                     error timeout invalid_header http_502 http_503 http_504;
            proxy_next_upstream_tries               0;

            proxy_pass http://upstream-default-backend;

            proxy_redirect                          off;

        }

        # health checks in cloud providers require the use of port 80
        location /healthz {
            access_log off;
            return 200;
        }

        # this is required to avoid error if nginx is being monitored
        # with an external software (like sysdig)
        location /nginx_status {
            allow 127.0.0.1;
            allow ::1;
            deny all;

            access_log off;
            stub_status on;
        }

    }
    ## end server _

    ## start server foo.com
    server {
        server_name foo.com ;

        listen 80;

        listen [::]:80;

        set $proxy_upstream_name "-";

        listen 443  ssl http2;

        listen [::]:443  ssl http2;

        # PEM sha: 09ee2c5360071e60c711fb369f3ff940ed241543
        ssl_certificate                         /ingress-controller/ssl/default-tls-wildcard-secret.pem;
        ssl_certificate_key                     /ingress-controller/ssl/default-tls-wildcard-secret.pem;

        ssl_trusted_certificate                 /ingress-controller/ssl/default-tls-wildcard-secret-full-chain.pem;
        ssl_stapling                            on;
        ssl_stapling_verify                     on;

        location / {

            if ($scheme = https) {
            more_set_headers                        "Strict-Transport-Security: max-age=15724800; includeSubDomains;";
            }

            port_in_redirect off;

            set $proxy_upstream_name "default-foomainsite-prod-80";

            set $namespace      "default";
            set $ingress_name   "foo-https-ingress";
            set $service_name   "foomainsite-prod";

            # enforce ssl on server side
            if ($redirect_to_https) {

                return 308 https://$best_http_host$request_uri;

            }

            client_max_body_size                    "1m";

            proxy_set_header Host                   $best_http_host;

            # Pass the extracted client certificate to the backend

            proxy_set_header ssl-client-cert        "";
            proxy_set_header ssl-client-verify      "";
            proxy_set_header ssl-client-subject-dn  "";
            proxy_set_header ssl-client-issuer-dn   "";

            # Allow websocket connections
            proxy_set_header                        Upgrade           $http_upgrade;

            proxy_set_header                        Connection        $connection_upgrade;

            proxy_set_header X-Real-IP              $the_real_ip;

            proxy_set_header X-Forwarded-For        $the_real_ip;

            proxy_set_header X-Forwarded-Host       $best_http_host;
            proxy_set_header X-Forwarded-Port       $pass_port;
            proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
            proxy_set_header X-Original-URI         $request_uri;
            proxy_set_header X-Scheme               $pass_access_scheme;

            # Pass the original X-Forwarded-For
            proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;

            # mitigate HTTPoxy Vulnerability
            # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
            proxy_set_header Proxy                  "";

            # Custom headers to proxied server

            proxy_connect_timeout                   5s;
            proxy_send_timeout                      60s;
            proxy_read_timeout                      60s;

            proxy_buffering                         "off";
            proxy_buffer_size                       "4k";
            proxy_buffers                           4 "4k";
            proxy_request_buffering                 "on";

            proxy_http_version                      1.1;

            proxy_cookie_domain                     off;
            proxy_cookie_path                       off;

            # In case of errors try the next upstream server before returning an error
            proxy_next_upstream                     error timeout invalid_header http_502 http_503 http_504;
            proxy_next_upstream_tries               0;

            proxy_pass http://default-foomainsite-prod-80;

            proxy_redirect                          off;

        }

    }
    ## end server foo.com

    ## start server www.foo.com
    server {
        server_name www.foo.com ;

        listen 80;

        listen [::]:80;

        set $proxy_upstream_name "-";

        listen 443  ssl http2;

        listen [::]:443  ssl http2;

        # PEM sha: 09ee2c5360071e60c711fb369f3ff940ed241543
        ssl_certificate                         /ingress-controller/ssl/default-tls-wildcard-secret.pem;
        ssl_certificate_key                     /ingress-controller/ssl/default-tls-wildcard-secret.pem;

        ssl_trusted_certificate                 /ingress-controller/ssl/default-tls-wildcard-secret-full-chain.pem;
        ssl_stapling                            on;
        ssl_stapling_verify                     on;

        location / {

            if ($scheme = https) {
            more_set_headers                        "Strict-Transport-Security: max-age=15724800; includeSubDomains;";
            }

            port_in_redirect off;

            set $proxy_upstream_name "default-foomainsite-prod-80";

            set $namespace      "default";
            set $ingress_name   "foo-https-ingress";
            set $service_name   "foomainsite-prod";

            # enforce ssl on server side
            if ($redirect_to_https) {

                return 308 https://$best_http_host$request_uri;

            }

            client_max_body_size                    "1m";

            proxy_set_header Host                   $best_http_host;

            # Pass the extracted client certificate to the backend

            proxy_set_header ssl-client-cert        "";
            proxy_set_header ssl-client-verify      "";
            proxy_set_header ssl-client-subject-dn  "";
            proxy_set_header ssl-client-issuer-dn   "";

            # Allow websocket connections
            proxy_set_header                        Upgrade           $http_upgrade;

            proxy_set_header                        Connection        $connection_upgrade;

            proxy_set_header X-Real-IP              $the_real_ip;

            proxy_set_header X-Forwarded-For        $the_real_ip;

            proxy_set_header X-Forwarded-Host       $best_http_host;
            proxy_set_header X-Forwarded-Port       $pass_port;
            proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
            proxy_set_header X-Original-URI         $request_uri;
            proxy_set_header X-Scheme               $pass_access_scheme;

            # Pass the original X-Forwarded-For
            proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;

            # mitigate HTTPoxy Vulnerability
            # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
            proxy_set_header Proxy                  "";

            # Custom headers to proxied server

            proxy_connect_timeout                   5s;
            proxy_send_timeout                      60s;
            proxy_read_timeout                      60s;

            proxy_buffering                         "off";
            proxy_buffer_size                       "4k";
            proxy_buffers                           4 "4k";
            proxy_request_buffering                 "on";

            proxy_http_version                      1.1;

            proxy_cookie_domain                     off;
            proxy_cookie_path                       off;

            # In case of errors try the next upstream server before returning an error
            proxy_next_upstream                     error timeout invalid_header http_502 http_503 http_504;
            proxy_next_upstream_tries               0;

            proxy_pass http://default-foomainsite-prod-80;

            proxy_redirect                          off;

        }

    }
    ## end server www.foo.com

    # default server, used for NGINX healthcheck and access to nginx stats
    server {
        # Use the port 18080 (random value just to avoid known ports) as default port for nginx.
        # Changing this value requires a change in:
        # https://github.com/kubernetes/ingress-nginx/blob/master/controllers/nginx/pkg/cmd/controller/nginx.go
        listen 18080 default_server  backlog=511;
        listen [::]:18080 default_server  backlog=511;
        set $proxy_upstream_name "-";

        location /healthz {
            access_log off;
            return 200;
        }

        location /nginx_status {
            set $proxy_upstream_name "internal";

            access_log off;
            stub_status on;

        }

        location /configuration {
            allow 127.0.0.1;

            allow ::1;

            deny all;
            content_by_lua_block {
              configuration.call()
            }
        }

        location / {

            set $proxy_upstream_name "upstream-default-backend";

            proxy_pass          http://upstream-default-backend;

        }

    }
}

stream {
    log_format log_stream [$time_local] $protocol $status $bytes_sent $bytes_received $session_time;

    access_log /var/log/nginx/access.log log_stream;

    error_log  /var/log/nginx/error.log;

    # TCP services

    # UDP services

}

[oilbeater]

@artemzakharov still trying to figure out why this happens, you can try to remove www.foo.com ingress rules to see if works

[artemzakharov]

@oilbeater I tried removing it, and this is what happens:
foo.com redirects to www.foo.com, which just displays default backend - 404
https://foo.com and https://www.foo.com result in the browser getting the ingress controller fake certificate

[oilbeater]

the logical here is a little complex, try remove the www.foo.com rule and leave the foo.com rule

[artemzakharov]

I'm confused, did you want me to remove the logic from the nginx file itself? I removed it from the yaml file that defined the ingress resource, so I was left with this:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: foo-https-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
spec:
  tls:
      - hosts:
          - foo.com
        secretName: tls-wildcard-secret  
  rules:
    - host: foo.com
      http:
        paths:
          - backend:
              serviceName: foomainsite-prod
              servicePort: 80
            path: /

[oilbeater]

@artemzakharov Sorry, I found my mistake at the first place. Just rollback the image and only keep the www.foo.com rule. Hope this could work

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: foo-https-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
spec:
  tls:
      - hosts:
          - www.foo.com
        secretName: tls-wildcard-secret  
  rules:
    - host: www.foo.com
      http:
        paths:
          - backend:
              serviceName: foomainsite-prod
              servicePort: 80
            path: /

[artemzakharov]

Hey @oilbeater, that mostly works, but i get a certificate error and no redirect when i manually enterhttps://foo.com. I would like for users going directly to https://foo.com to be redirectred to https://www.foo.com too.

[djhmateer]

Duplicate of #2043 ?

[evelant]

I am being impacted by this. Users accessing the non-www domain is messing up my analytics data. Any workarounds? I can't find any.

[amardomingo]

Since we only have one relevant domain, we worked around this by using the "correct" certificate as the default tls ( '--default-ssl-certificate' ), but we are also being impacted by this.

Any other workaround or correct way to do this?

[artemzakharov]

@djhmateer It might be at this point. I opened this issue after a comment by @aledbf on the older one suggested that the behavior was intentional and I simply needed a wildcard certificate to fix things, but that turned out to be wrong, so I thought this was a separate issue altogether. In any case, we'll see if this issue persists when the original one is resolved.

[antoineco]

@artemzakharov this should be fixed in 0.16.2.

[evelant]

I found a hacky workaround that might be useful to others. I made an S3 bucket that does a redirect from root to www and setup a cloudfront distribution with my wildcard certificate pointing to that bucket. I then pointed my root dns to the cloudfront distribution. Things get redirected from root to www using https as expected with this setup.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[amih90]

same here! any news?

[kyma]

I'm also having this issue. Any news?