NGINX support

[sagitsofan]

Kowl does not support NGINX.

I added a NGINX in front of Kowl and the UI is broken, because that many static resources are absolute and not relative, according to the location configured inside the nginx.conf

I configured Kowl to be under admin/kafka path, but all the static resources are pointing to /
I saw in the Frontend a configuration with %PUBLIC_URL% but it needs to be exported as a environment variable from outside of the container ( like in docker compose yml example bellow )

Example for my nginx configuration:

      location /admin/kafka/ {
           proxy_pass             http://kowl:8077/;
           proxy_redirect  / /admin/kowl/;
           
           proxy_buffering                    off;
           proxy_set_header Host              $http_host;
           proxy_set_header X-Real-IP         $remote_addr;
           proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
           proxy_set_header X-Forwarded-Proto $scheme;
       }

[weeco]

Hi,
I think this overlaps with #107 . We are already working on this, but I think you'd need to add a X-Forwarded-For header for the new solution to work.

[rikimaru0345]

The header is named X-Forwarded-Prefix.
And there'll also be an option to set a static prefix, without having to use a proxy that sets (or strips) the header.
But yeah, as weeco said, it's still WIP but should be done soon.

[sagitsofan]

The solution with the X-Forwarded-Prefix looks good!
When will you release a new version and a new docker image?

[rikimaru0345]

When the feature is done, the github action should automatically create a new image.
I'll make sure to notify you here when its done though 😄

The feature itself should be done pretty soon.

[sagitsofan]

Thank you! appreciates that

[rikimaru0345]

@sagitsofan

The feature is done 😄
Please try it out and let me know if everything works.
See https://github.com/cloudhut/kowl/blob/master/docs/config/hosting.md#config-entries for documentation

[sagitsofan]

@rikimaru0345 It works good now with the X-Forwarded-Prefix header.
I used the master tag for the docker image, will you create a new 1.1.1 version?

[weeco]

This feature will be part of the 1.2 release. We don't have a due date for any of the upcoming releases but you can track the progress via the milestones here: https://github.com/cloudhut/kowl/milestone/3

[sagitsofan]

So using in production the `master ` tag is safe ?

…

On Tue, Aug 18, 2020 at 12:08 PM Martin Schneppenheim < ***@***.***> wrote: This feature will be part of the 1.2 release. We don't have a due date for any of the upcoming releases but you can track the progress via the milestones here: https://github.com/cloudhut/kowl/issues?q=is%3Aopen+is%3Aissue+milestone%3A%221.2+release%22 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#117 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN6JGQSUAYLRTZCJK2BCVDSBJAJZANCNFSM4QBLZ6CQ> .

[weeco]

No you should not use the master tag. Instead use one of the tags which refer to a specific commit. For example: master-f0aa871d: https://quay.io/repository/cloudhut/kowl?tab=tags. We always try to keep the master branch in a production ready state and we regularly use images from master as well. Obviously we can't provide extensive testing after each commit / merge to master.

We do test each version before we create a new release. The definition of "safe for production" varies a lot I guess.

[sagitsofan]

Just found a critical bug, when i run the application with the `X-Forwarded-Prefix` header, there is not messages although i have. I found an error, seems like the web socket is not working well with the new header Error in developer tools:  With the proxy:  Without the proxy: 

[rikimaru0345]

Which version (commit hash) were you testing?
Can you make sure it is f0aa871 ?

I'll try to reproduce the error 👍 😄 Can you please post your NGINX config?

edit:
You can see the commit hash in the lower left corner.
Don't mind the invalid "hashes" in my example picture they're just placeholders for development (where there are no commit hashes yet 😄)

[sagitsofan]

Using the latest commit: f0aa871
My NGINX configuration:

location /admin {
      resolver 127.0.0.11;

      location /admin/kafka/ {
          proxy_pass                              http://kowl:8077/;
          proxy_set_header X-Forwarded-Prefix     /admin/kafka/;
      }
  }

yml config:

kafka:
  brokers:
    - kafka1:19092

server:
  listenPort: 8077
  setBasePathFromXForwardedPrefix: true

docker config:

  kowl:
    image: quay.io/cloudhut/kowl:master-f0aa871d
    restart: unless-stopped
    hostname: kowl
    entrypoint: ./kowl --config.filepath=/etc/kowl/config.yaml
    networks:
      internal: null

[rikimaru0345]

Ah, do you happen to have a docker-compose file that I can use to reproduce the error? 😄

[sagitsofan]

Ah, not something completed that i can send, because its used in docker swarm with multiple compose overrides.
I can create something, but it will take me some time

[sagitsofan]

@rikimaru0345 I created a small project that you can clone and reproduce the error:
https://github.com/sagitsofan/KowlConfiguration

[rikimaru0345]

Thanks, I will take a look this evening!

[rikimaru0345]

@sagitsofan
Just tried it, seems like the nginx dockerfile is missing?

$ docker-compose -f docker-compose.yml up
WARNING: Some services (zoo1) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
Building nginx
ERROR: Cannot locate specified Dockerfile: Dockerfile 

[sagitsofan]

@rikimaru0345 Just pull the images manually and then run it again:

docker pull nginx:1.19-alpine
docker pull zookeeper:3.4.9
docker pull confluentinc/cp-kafka:5.5.0
docker pull quay.io/cloudhut/kowl:master

[rikimaru0345]

Seems like the issue is really simple. nginx must be configured to keep passing the upgrade header.
Just adding this to the nginx config did the trick.

        proxy_http_version 1.1;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

Here's how my complete nginx_prod.conf file looks now:

server {
    listen 80;

    location /admin/kafka/ {
        proxy_pass                              http://kowl:8077/;
        proxy_set_header X-Forwarded-Prefix     /admin/kafka/;


        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;


        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

See here for why this is needed:
https://nginx.org/en/docs/http/websocket.html

[sagitsofan]

Works great, Thank you @rikimaru0345