Stig-Manager in Docker as a shared resource, (not localhost only mode) resolvable by DNS #1018

jeremyatourville · 2023-06-16T17:22:08Z

jeremyatourville
Jun 16, 2023

USE CASE SCENARIO:

I would like to run Stig-Mamager so that it can be a shared resource by our Admin team. My short term goal is getting local accounts setup in keycloak with username/pw for authentication. As a long term goal I will see if I can setup PIV cards with certificiates so that keycloak can use tokens to authenticate.

I have taken the following steps so far as part of my short term goal:

Application is running as a docker container(s)
A dns "A" record was created for the docker server and a CNAME entry was created for Stigman that points to the docker server.
Generated certificates for the docker server that is running the app. We have our own internal CA. So I generated a cert for the docker host and created a SAN entry for "stigman"
Certs were put into the certs folders (ca, dod, localhost) SSL termination is happening at nginx frontend
A P12 truststore was generated and placed in the certs folder.
The setup was modified so that the relative path /kc resolves to Keycloak.

So far I am able to get to the landing page for keycloak. From there, I can click on the admin page. The page partially loads but doesn't fully complete loading and I just get a spinner icon on the page. Can anyone provide tips or suggestions on troubleshooting? I have included a copy of my logs and configs below.

Logs:

[root@gsil-docker2 nginx]# docker-compose up -d && docker-compose logs -f
...                                                                                                                                                           2.4s
nginx-nginx-1    | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx-nginx-1    | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx-nginx-1    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx-nginx-1    | 10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf is not a file or does not exist
nginx-nginx-1    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx-nginx-1    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx-nginx-1    | /docker-entrypoint.sh: Configuration complete; ready for start up
nginx-db-1       | [Entrypoint] MySQL Docker Image 8.0.20-1.1.16
nginx-db-1       | mysqld: [Warning] World-writable config file '/etc/my.cnf' is ignored.
nginx-db-1       | [Entrypoint] Starting MySQL 8.0.20-1.1.16
nginx-db-1       | mysqld: [Warning] World-writable config file '/etc/my.cnf' is ignored.
nginx-db-1       | 2023-06-16T16:01:41.764130Z 0 [Warning] [MY-013245] [Server] The SSL library function CRYPTO_set_mem_functions failed. This is typically caused by the SSL library already being used. As a result the SSL memory allocation will not be instrumented.
nginx-db-1       | 2023-06-16T16:01:41.767248Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.31) starting as process 1
nginx-db-1       | 2023-06-16T16:01:41.774666Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
nginx-db-1       | 2023-06-16T16:01:42.088374Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
nginx-stigman-1  | {"date":"2023-06-16T16:01:42.093Z","level":3,"component":"index","type":"starting","data":{"version":"1.3.5","env":{"STIGMAN_DB_PASSWORD":"*","NODE_VERSION":"18.16.0","STIGMAN_DB_HOST":"db","STIGMAN_DB_USER":"stigman","STIGMAN_CLIENT_OIDC_PROVIDER":"https://stigman.idm.example.org/kc/realms/stigman","STIGMAN_CLASSIFICATION":"U","STIGMAN_OIDC_PROVIDER":"http://auth:8080/kc/realms/stigman","STIGMAN_CLIENT_DIRECTORY":"./client","STIGMAN_DOCS_DIRECTORY":"./docs"},"dirname":"/home/node","cwd":"/home/node"}}
nginx-stigman-1  | {"date":"2023-06-16T16:01:42.097Z","level":3,"component":"index","type":"configuration","data":{"version":"1.3.5","commit":{"branch":"main","sha":"795a2b71a9","tag":"na","describe":"1.3.5-1-g795a2b7"},"settings":{"setClassification":"U","lastAccessResolution":60,"responseValidation":"none"},"client":{"clientId":"stig-manager","authority":"https://stigman.idm.example.org/kc/realms/stigman","apiBase":"api","disabled":false,"directory":"./client","refreshToken":{"disabled":false},"welcome":{"image":"","message":"","title":"","link":""}},"docs":{"disabled":false,"docsDirectory":"./docs"},"http":{"address":"0.0.0.0","port":54000,"maxJsonBody":"31457280","maxUpload":"1073741824"},"database":{"type":"mysql","host":"db","port":3306,"schema":"stigman","username":"stigman","maxConnections":25,"tls":{},"revert":false,"password":true},"init":{"importStigs":false},"swaggerUi":{"enabled":false,"authority":"http://auth:8080/kc/realms/stigman","server":"http://localhost:54000/api","oauth2RedirectUrl":"http://localhost:54000/api-docs/oauth2-redirect.html"},"oauth":{"authority":"http://auth:8080/kc/realms/stigman","claims":{"scope":"scope","username":"preferred_username","servicename":"clientId","name":"name","privileges":"['realm_access']?.['roles']","email":"email"}},"log":{"level":3,"mode":"combined"}}}
nginx-auth-1     | Changes detected in configuration. Updating the server image.
nginx-auth-1     | Updating the configuration and installing your custom providers, if any. Please wait.
nginx-db-1       | 2023-06-16T16:01:42.324707Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
nginx-db-1       | 2023-06-16T16:01:42.324741Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
nginx-db-1       | 2023-06-16T16:01:42.378372Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
nginx-db-1       | 2023-06-16T16:01:42.378651Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.31'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server - GPL.
nginx-stigman-1  | {"date":"2023-06-16T16:01:43.215Z","level":1,"component":"oidc","type":"discovery","data":{"success":false,"metadataUri":"http://auth:8080/kc/realms/stigman/.well-known/openid-configuration","message":"connect ECONNREFUSED 192.168.96.4:8080"}}
nginx-stigman-1  | {"date":"2023-06-16T16:01:43.787Z","level":3,"component":"mysql","type":"preflight","data":{"success":true,"version":"8.0.31"}}
nginx-stigman-1  | {"date":"2023-06-16T16:01:43.821Z","level":3,"component":"mysql","type":"migration","data":{"message":"MySQL schema is up to date"}}
nginx-stigman-1  | {"date":"2023-06-16T16:01:48.222Z","level":1,"component":"oidc","type":"discovery","data":{"success":false,"metadataUri":"http://auth:8080/kc/realms/stigman/.well-known/openid-configuration","message":"connect ECONNREFUSED 192.168.96.4:8080"}}
nginx-auth-1     | 2023-06-16 16:01:49,658 INFO  [io.quarkus.deployment.QuarkusAugmentor] (main) Quarkus augmentation completed in 5682ms
nginx-auth-1     | Server configuration updated and persisted. Run the following command to review the configuration:
nginx-auth-1     | 
nginx-auth-1     | 	kc.sh show-config
nginx-auth-1     | 
nginx-auth-1     | Next time you run the server, just run:
nginx-auth-1     | 
nginx-auth-1     | 	kc.sh start --optimized --hostname-strict-backchannel=true
nginx-auth-1     | 
nginx-stigman-1  | {"date":"2023-06-16T16:01:53.226Z","level":1,"component":"oidc","type":"discovery","data":{"success":false,"metadataUri":"http://auth:8080/kc/realms/stigman/.well-known/openid-configuration","message":"connect ECONNREFUSED 192.168.96.4:8080"}}
nginx-auth-1     | 2023-06-16 16:01:51,864 INFO  [org.keycloak.quarkus.runtime.hostname.DefaultHostnameProvider] (main) Hostname settings: Base URL: http://auth:8080/kc, Hostname: auth, Strict HTTPS: false, Path: /kc, Strict BackChannel: true, Admin URL: https://stigman.idm.example.org/kc/, Admin: stigman.idm.example.org, Port: 8080, Proxied: true
nginx-auth-1     | 2023-06-16 16:01:53,216 INFO  [org.keycloak.common.crypto.CryptoIntegration] (main) Detected crypto provider: org.keycloak.crypto.def.DefaultCryptoProvider
nginx-auth-1     | 2023-06-16 16:01:54,983 WARN  [org.infinispan.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
nginx-auth-1     | 2023-06-16 16:01:54,997 WARN  [org.infinispan.PERSISTENCE] (keycloak-cache-init) ISPN000554: jboss-marshalling is deprecated and planned for removal
nginx-auth-1     | 2023-06-16 16:01:55,007 INFO  [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
nginx-auth-1     | 2023-06-16 16:01:55,198 INFO  [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000128: Infinispan version: Infinispan 'Triskaidekaphobia' 13.0.9.Final
nginx-auth-1     | 2023-06-16 16:01:55,309 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000078: Starting JGroups channel `ISPN`
nginx-auth-1     | 2023-06-16 16:01:55,310 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000088: Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
nginx-auth-1     | 2023-06-16 16:01:55,376 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
nginx-auth-1     | 2023-06-16 16:01:55,377 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 20.00MB, but the OS only allocated 212.99KB
nginx-auth-1     | 2023-06-16 16:01:55,378 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
nginx-auth-1     | 2023-06-16 16:01:55,378 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 25.00MB, but the OS only allocated 212.99KB
nginx-auth-1     | 2023-06-16 16:01:57,389 INFO  [org.jgroups.protocols.pbcast.GMS] (keycloak-cache-init) 0173181baafb-54722: no members discovered after 2002 ms: creating cluster as coordinator
nginx-auth-1     | 2023-06-16 16:01:57,397 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000094: Received new cluster view for channel ISPN: [0173181baafb-54722|0] (1) [0173181baafb-54722]
nginx-auth-1     | 2023-06-16 16:01:57,400 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000079: Channel `ISPN` local address is `0173181baafb-54722`, physical addresses are `[192.168.96.4:50219]`
nginx-auth-1     | 2023-06-16 16:01:57,912 INFO  [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: 0173181baafb-54722, Site name: null
nginx-stigman-1  | {"date":"2023-06-16T16:01:58.234Z","level":1,"component":"oidc","type":"discovery","data":{"success":false,"metadataUri":"http://auth:8080/kc/realms/stigman/.well-known/openid-configuration","message":"connect ECONNREFUSED 192.168.96.4:8080"}}
nginx-auth-1     | 2023-06-16 16:01:59,059 INFO  [org.keycloak.quarkus.runtime.storage.legacy.liquibase.QuarkusJpaUpdaterProvider] (main) Initializing database schema. Using changelog META-INF/jpa-changelog-master.xml
nginx-auth-1     | 2023-06-16 16:02:00,888 INFO  [org.keycloak.services] (main) KC-SERVICES0050: Initializing master realm
nginx-stigman-1  | {"date":"2023-06-16T16:02:03.240Z","level":1,"component":"oidc","type":"discovery","data":{"success":false,"metadataUri":"http://auth:8080/kc/realms/stigman/.well-known/openid-configuration","message":"connect ECONNREFUSED 192.168.96.4:8080"}}
nginx-auth-1     | 2023-06-16 16:02:03,856 INFO  [org.keycloak.services] (main) KC-SERVICES0004: Imported realm stigman from file /opt/keycloak/bin/../data/import/import_realm.json.
nginx-auth-1     | 2023-06-16 16:02:04,007 INFO  [org.keycloak.services] (main) KC-SERVICES0003: Not importing realm stigman from file /opt/keycloak/bin/../data/import/stigman_realm.json.  It already exists.
nginx-auth-1     | 2023-06-16 16:02:04,011 INFO  [org.keycloak.services] (main) KC-SERVICES0003: Not importing realm stigman from file /opt/keycloak/bin/../data/import/stigman_realm.json.  It already exists.
nginx-auth-1     | 2023-06-16 16:02:04,225 INFO  [io.quarkus] (main) Keycloak 19.0.2 on JVM (powered by Quarkus 2.7.6.Final) started in 14.323s. Listening on: http://0.0.0.0:8080
nginx-auth-1     | 2023-06-16 16:02:04,225 INFO  [io.quarkus] (main) Profile prod activated. 
nginx-auth-1     | 2023-06-16 16:02:04,226 INFO  [io.quarkus] (main) Installed features: [agroal, cdi, hibernate-orm, jdbc-h2, jdbc-mariadb, jdbc-mssql, jdbc-mysql, jdbc-oracle, jdbc-postgresql, keycloak, logging-gelf, narayana-jta, reactive-routes, resteasy, resteasy-jackson, smallrye-context-propagation, smallrye-health, smallrye-metrics, vault, vertx]
nginx-auth-1     | 2023-06-16 16:02:04,404 INFO  [org.keycloak.services] (main) KC-SERVICES0009: Added user 'admin' to realm 'master'
nginx-stigman-1  | {"date":"2023-06-16T16:02:08.489Z","level":3,"component":"oidc","type":"discovery","data":{"success":true,"metadataUri":"http://auth:8080/kc/realms/stigman/.well-known/openid-configuration","jwksUri":"http://auth:8080/kc/realms/stigman/protocol/openid-connect/certs"}}
nginx-stigman-1  | {"date":"2023-06-16T16:02:08.497Z","level":3,"component":"index","type":"started","data":{"durationS":26.433772703,"port":54000,"api":"/api","client":"/","documentation":"/docs"}}
nginx-stigman-1  | {"date":"2023-06-16T16:03:42.325Z","level":3,"component":"rest","type":"transaction","data":{"request":{"date":"2023-06-16T16:03:42.138Z","source":"::ffff:127.0.0.1","method":"GET","url":"/api/op/configuration","headers":{"host":"localhost:54000","connection":"close","authorization":false}},"response":{"date":"2023-06-16T16:03:42.323Z","status":200,"headers":{"x-powered-by":"Express","access-control-allow-origin":"*","content-type":"application/json; charset=utf-8","content-length":"127","etag":"W/\"7f-Iz2iLsLl4gNXoC+opprc4XP1+EY\"","vary":"Accept-Encoding"}},"durationMs":185}}

Docker compose config:

# STIG Manager behind Proxy with DNS Hostname

version: '3.7'

networks:
  default:
    name: stigman-orchestration_default
    external: false

services:
  nginx:
    image: gsil-docker1.idm.example.org:5000/nginx:1.23.1
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/index.html:/usr/share/nginx/html/index.html
      - ./certs/localhost/gsil-docker2-chain.crt:/etc/nginx/localhost.crt
      - ./certs/localhost/gsil-docker2.key:/etc/nginx/localhost.key
      - ./certs/dod/gsil-docker2-chain.crt:/etc/nginx/gsil-docker2-chain.crt
    ports:
    - "443:443"
    restart: unless-stopped
    
  auth:
    image: gsil-docker1.idm.example.org:5000/stig-manager-auth:latest
    environment:
      - KEYCLOAK_ADMIN=admin
      - KEYCLOAK_ADMIN_PASSWORD=Pa55w0rd
      - KC_PROXY=edge
      - PROXY_ADDRESS_FORWARDING=true
      - KC_HOSTNAME_URL=http://auth:8080/kc
      - KC_HOSTNAME_ADMIN_URL=https://stigman.idm.example.org/kc/
      - KC_SPI_X509CERT_LOOKUP_PROVIDER=nginx
      - KC_SPI_X509CERT_LOOKUP_NGINX_SSL_CLIENT_CERT=SSL-CLIENT-CERT
      - KC_SPI_TRUSTSTORE_FILE_FILE=/tmp/truststore.p12
      - KC_SPI_TRUSTSTORE_FILE_PASSWORD=password
    command: start --http-relative-path /kc --hostname-strict-backchannel=true --import-realm 
    volumes:
      - ./certs/dod/truststore.p12:/tmp/truststore.p12
      - ./kc/stigman_realm.json:/opt/keycloak/data/import/stigman_realm.json

  db:
    image: gsil-docker1.idm.example.org:5000/mysql8:8.0.31
    environment:
      - MYSQL_ROOT_PASSWORD=rootpw
      - MYSQL_USER=stigman
      - MYSQL_PASSWORD=stigmanpw
      - MYSQL_DATABASE=stigman
    cap_add:
      - SYS_NICE  # workaround MySQL logging bug => mbind: Operation not permitted
    volumes:
      - ./mysql-data:/var/lib/mysql

  stigman:
    image: gsil-docker1.idm.example.org:5000/stig-manager:1.3.5
    environment:
      - STIGMAN_OIDC_PROVIDER=http://auth:8080/kc/realms/stigman
      - STIGMAN_CLIENT_OIDC_PROVIDER=https://stigman.idm.example.org/kc/realms/stigman  
      - STIGMAN_CLASSIFICATION=U
      - STIGMAN_DB_HOST=db
      - STIGMAN_DB_PASSWORD=stigmanpw
      - STIGMAN_DB_USER=stigman
    init: true
    depends_on: 
    - auth

NGINX Config:

events {
  worker_connections  4096;  ## Default: 1024
}
pid        /var/cache/nginx/nginx.pid;
http {
    server {
        listen 443 ssl;
        server_name stigman.idm.example.org;
        location /kc/ {
            proxy_pass              http://auth:8080;
            proxy_set_header        Host               $host;
            proxy_set_header        X-Real-IP          $remote_addr;
            proxy_set_header        X-Forwarded-For    $proxy_add_x_forwarded_for;
            proxy_set_header        X-Forwarded-Host   $host;
            proxy_set_header        X-Forwarded-Server $host;
            proxy_set_header        X-Forwarded-Port   $server_port;
            proxy_set_header        X-Forwarded-Proto  $scheme;
            proxy_set_header        ssl-client-cert    $ssl_client_escaped_cert;
            proxy_buffer_size       128k;
            proxy_buffers           4 256k;
            proxy_busy_buffers_size 256k;
        }

       location /stigman/ {
            proxy_pass              http://stigman:54000;
        }

        ssl_certificate             /etc/nginx/localhost.crt; ## localhost directory certs
        ssl_certificate_key         /etc/nginx/localhost.key;
        root                        /usr/share/nginx/html;
        client_max_body_size        100M;
        ssl_prefer_server_ciphers   on;
        ssl_client_certificate      /etc/nginx/gsil-docker2-chain.crt;
        ssl_verify_client           optional;
        ssl_verify_depth            4;
        error_log                   /var/log/nginx/error.log debug;
        if ($return_unauthorized) { return 496; }
        proxy_connect_timeout   600;
        proxy_send_timeout      600;
        proxy_read_timeout      600;
        send_timeout            600;

        location / {
            autoindex on;
            ssi on;
         }
    }

    # define which endpoints require mTLS
    map_hash_bucket_size 128;
    map $uri $secured_url {
        default false;
        "/kc/realms/stigman/protocol/openid-connect/auth" true;
    }

    map "$secured_url:$ssl_client_verify" $return_unauthorized {
            default 0;
            "true:FAILED" 1;
            "true:NONE" 1;
            "true:" 1;
    }
}

cd-rite · 2023-06-20T23:07:57Z

cd-rite
Jun 20, 2023
Maintainer

Hi @jeremyatourville
A couple issues do stand out in your orchestration...However, there are many ways to deploy STIGMan, and the project group are not experts in all those possible ways, so we offer general guidance and demos/samples only!

It seems like you are modifying our "Demo" orchestration that uses our demo Auth container from our Docker Hub page. Have you successfully run without making any changes?

The changes you posted will cause some issues....
I think you'll want KC_HOSTNAME_URL to be same/similar to your KC_HOSTNAME_ADMIN_URL since that is the address Keycloak will expect clients to be hitting.
You don't want ``--hostname-strict-backchannel=true, because that will prevent the API from contacting Keycloak with the "auth" alias, since it will be enforcing the external URLs specified above. PROXY_ADDRESS_FORWARDING` should not be needed, as it's function has been superseded by `KC_PROXY=edge`

We also don't typically run with --http-relative-path... I think that relative path is already added in your nginx configuration, so keycloak wouldn't have to do it itself. (So you'd want to remove the "kc" from your STIGMAN_OIDC_PROVIDER setting. The STIGMAN_CLIENT_OIDC_PROVIDER is for the client, which would be coming in through the proxy, so you'll need it there.)

I'm not sure if you've used it, but our stigman-orchestration repo offers a working setup (for localhost testing) with CAC and a proxy, but certain parts should still apply to your use case: https://github.com/NUWCDIVNPT/stigman-orchestration/blob/main/docker-compose.yml (This is a slightly more advanced example than the "demo" docker-compose on Docker Hub that doesn't use a proxy and just uses username/pw.)

You could try starting there, and then incrementally make some changes as needed for your deployment.

Hope that helps some!

1 reply

jeremyatourville Jun 21, 2023
Author

Hi @cd-rite

we offer general guidance and demos/samples only
I was hoping to get lucky with a little bit of support and advice but I can understand your position too.

Have you successfully run without making any changes?
Yes, that I am able to make work without issue.

then incrementally make some changes as needed for your deployment.
The changes you see here are my incremental changes

our stigman-orchestration repo offers a working setup (for localhost testing)
Acknowledged, but this is the very issue I am trying to solve in my use case scenario. Localhost only doesn't prove to be very useful for my team.

Many of your comments about my config are helpful. I have struggled some with the relative path issue because I am not using certs for authentication and therefore wanted to access the keycloak admin page so I can setup accounts with UN/PW. If I set relative path /kc as the root I can at least get to the keycloak page. If I set/as the root I only get the web/test page that says something about headers or certs or something along those lines, is working, and can never get to the keycloak admin page.

This is why I set --http-relative-path My understanding is that keycloak natively will go to / unless specified otherwise.
So this whole issue is a little bit of a chicken and egg problem unless your are knowledgeable enough to write out a correct .json config file for keycloak (which I am not) to import the needed user accounts. So, how else can you setup the needed accounts?

I think you'll want KC_HOSTNAME_URL to be same/similar to your KC_HOSTNAME_ADMIN_URL since that is the address Keycloak will expect clients to be hitting.
Respectfully, I am not sure if that is correct. My testing seems to show something very different. But that is the entire reason for this discussion. There are so many moving parts here and this is a somewhat complex project. I genuinely want to help the greater FOSS community here so I hope you won't take my comments as inflammatory. I don't intend them to be so. :-)

I'll keep plugging away at this with many or all of the changes you had suggested. Yes, overall your comments are helpful.

cd-rite · 2023-06-21T18:14:30Z

cd-rite
Jun 21, 2023
Maintainer

@jeremyatourville Ok.... Though you've got a lot of stuff in there that is CAC-specific, like all the keycloak KC_SPI_X509CERT_LOOKUP_PROVIDER stuff.

You are also importing a realm, but using our prebuilt image, which already has a stigman realm with some dummy username/pw accounts built in (https://hub.docker.com/repository/docker/nuwcdivnpt/stig-manager-auth/general).
Are you able to access the app with those sample accounts? Does the app come up, but you just can't administer keycloak?

You might also remove the proxy from the picture while you sort out any other issues, and add it back later. It might be complicating things.

4 replies

jeremyatourville Jun 23, 2023
Author

@cd-rite I got it working!!! There were 2 issues to fix:
#1 - KC_HOSTNAME_URL=http://auth:8080/kc needed to be replaced with - KC_HOSTNAME=stigman.idm.example.org
#2 nginx.conf required some extra configuration to allow the admin page to work.

Here is my working config:
docker-compose.yml

# STIG Manager behind Proxy with DNS Hostname

version: '3.7'

networks:
  default:
    name: stigman-orchestration_default
    external: false

services:
  nginx:
    image: gsil-docker1.idm.example.org:5000/nginx:1.23.1
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/index.html:/usr/share/nginx/html/index.html
      - ./certs/localhost/gsil-docker2-chain.crt:/etc/nginx/localhost.crt
      - ./certs/localhost/gsil-docker2.key:/etc/nginx/localhost.key
      - ./certs/dod/gsil-docker2-chain.crt:/etc/nginx/gsil-docker2-chain.crt
    ports:
    - "443:443"
    restart: unless-stopped
    
  auth:
    image: gsil-docker1.idm.example.org:5000/stig-manager-auth:latest
    environment:
      - KEYCLOAK_ADMIN=admin
      - KEYCLOAK_ADMIN_PASSWORD=Pa55w0rd
      - KC_PROXY=edge
      - KC_HOSTNAME=stigman.idm.example.org
#      - KC_HOSTNAME_URL=http://auth:8080/
      - KC_HOSTNAME_ADMIN_URL=https://stigman.idm.example.org/kc/
#      - KC_SPI_X509CERT_LOOKUP_PROVIDER=nginx
#      - KC_SPI_X509CERT_LOOKUP_NGINX_SSL_CLIENT_CERT=SSL-CLIENT-CERT
      - KC_SPI_TRUSTSTORE_FILE_FILE=/tmp/truststore.p12
      - KC_SPI_TRUSTSTORE_FILE_PASSWORD=password
    command: start --import-realm
    volumes:
      - ./certs/dod/truststore.p12:/tmp/truststore.p12
      - ./kc/stigman_realm.json:/opt/keycloak/data/import/stigman_realm.json
      - ./kc/h2:/opt/keycloak/data/h2/

  db:
    image: gsil-docker1.idm.example.org:5000/mysql8:8.0.31
    environment:
      - MYSQL_ROOT_PASSWORD=rootpw
      - MYSQL_USER=stigman
      - MYSQL_PASSWORD=stigmanpw
      - MYSQL_DATABASE=stigman
    cap_add:
      - SYS_NICE  # workaround MySQL logging bug => mbind: Operation not permitted
    volumes:
      - ./mysql-data:/var/lib/mysql

  stigman:
    image: gsil-docker1.idm.example.org:5000/stig-manager:1.3.5
    environment:
      - STIGMAN_OIDC_PROVIDER=http://auth:8080/realms/stigman
      - STIGMAN_CLIENT_OIDC_PROVIDER=https://stigman.idm.example.org/realms/stigman  
      - STIGMAN_CLASSIFICATION=U
      - STIGMAN_DB_HOST=db
      - STIGMAN_DB_PASSWORD=stigmanpw
      - STIGMAN_DB_USER=stigman
    init: true
    depends_on: 
    - auth

nginx.conf

events {
  worker_connections  4096;  ## Default: 1024
}
pid        /var/cache/nginx/nginx.pid;
http {
    server {
        listen 443 ssl;
        server_name stigman.idm.example.org;
        location /kc/ {
            proxy_pass              http://auth:8080/;
            proxy_set_header        Host               $host;
            proxy_set_header        X-Real-IP          $remote_addr;
            proxy_set_header        X-Forwarded-For    $proxy_add_x_forwarded_for;
            proxy_set_header        X-Forwarded-Host   $host;
            proxy_set_header        X-Forwarded-Server $host;
            proxy_set_header        X-Forwarded-Port   $server_port;
            proxy_set_header        X-Forwarded-Proto  $scheme;
            proxy_buffer_size       128k;
            proxy_buffers           4 256k;
            proxy_busy_buffers_size 256k;
        }
        location /admin/ {
            proxy_pass http://auth:8080/admin/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-Port $server_port;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
        location /resources/ {
           proxy_pass http://auth:8080/resources/;
           proxy_set_header Host $host;
           proxy_set_header X-Real-IP $remote_addr;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Server $host;
           proxy_set_header X-Forwarded-Port $server_port;
           proxy_set_header X-Forwarded-Proto $scheme;
        }
        location /js/ {
            proxy_pass http://auth:8080/js/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-Port $server_port;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
        location /realms/ {
            proxy_pass http://auth:8080/realms/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-Port $server_port;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
       location /stigman/ {
            proxy_pass              http://stigman:54000/;
        }

        ssl_certificate             /etc/nginx/localhost.crt; ## localhost directory certs
        ssl_certificate_key         /etc/nginx/localhost.key;
        root                        /usr/share/nginx/html;
        client_max_body_size        100M;
        ssl_prefer_server_ciphers   on;
        ssl_client_certificate      /etc/nginx/gsil-docker2-chain.crt;
        ssl_verify_client           optional;
        ssl_verify_depth            4;
        error_log                   /var/log/nginx/error.log debug;
        if ($return_unauthorized) { return 496; }
        proxy_connect_timeout   600;
        proxy_send_timeout      600;
        proxy_read_timeout      600;
        send_timeout            600;

#        location / {
#            autoindex on;
#            ssi on;
#         }
    }

    # define which endpoints require mTLS
    map_hash_bucket_size 128;
    map $uri $secured_url {
        default false;
        "/kc/realms/stigman/protocol/openid-connect/auth" true;
    }

    map "$secured_url:$ssl_client_verify" $return_unauthorized {
            default 0;
            "true:FAILED" 1;
            "true:NONE" 1;
            "true:" 1;
    }
}

I created a PR in the stiman-orchestraton repo based on theses changes. I spent a lot of time troubleshooting and saw many others who were having issues with keycloak behind nginx reverse proxy. While I could get to the keycloak main page, when I attempted to go to the admin page the page would only partially load and get a spinning icon that never went away. Hopefully this will help some people get their instance up running more quickly :-)

scjohnson1988 Oct 27, 2023

@jeremyatourville Sorry for the random follow-up, but is the configuration still working for you? I've been trying to get yours working, but running into some issues. I ended up toggling/cycling through a few of your configs and the example deployments here, but I also found a few issues that I've fixed (can't use a certificate with a wildcard SAN for the host and it seemed like the default .p12 wasn't working for me). Right now, when I enter 'example.com/stigman', I'm redirected from stigman to keycloak where it prompts for my CAC. After entering my PIN, keycloak throws an error stating I have an incorrect username and password, even though I'm not prompted for a username/password at all. For reference, I'm using valid DoD certificates. I've verified with openssl that the keycloak truststore.p12 and the Nginx CA pem chain are validating with the provided host certificate.

With that said, I had a few questions if that's still working for you:

Which config should I use if yours is working? I noticed that the Nginx.conf and docker-compose in your forked repo are a slightly different from the ones you listed here.
In your readme on the fork, you mention creating a CNAME for stigman. Is that necessary versus having an A record for the host Docker is running on? If it is necessary, which record would I associate the CNAME to?
With the current setup, am I supposed to be prompted for my CAC at all after I go to https://example.com/stigman? I'm assuming no, especially since it looks like you commented out some of the SSL configs and specify this should be username/password.
Your readme specifies that a 'Username and Password configured in Keycloak' as a requirement. Is that something beyond the 'admin' account being created in the compose?

Also just wanted to say thanks for posting your info. I've never used keycloak and am very lightly experienced with Nginx, so it was really helpful. We have a team of people that manage STIGs, so it makes a lot of sense to be able to access this remotely.

jeremyatourville Oct 30, 2023
Author

@scjohnson1988

Use the ones in my forked repo.
You associate the CNAME to the host it is running on. In my case I have many apps running on the same host so I can call each app by name. For example, I am running prometheus, guacamole and kibana all on the same host. I create an A record for the docker host and create CNAME records for prometheus, gucamole and kibana that all point to the A record. Hope that helps. You could possibly create just an A record but it gets easier to use CNAME when you have multiple apps on the same host . Hope that makes sense!
It depends. My setup presumes you are using UN/PW and not CAC auth. If you are using UN/PW you should get prompted to login from keycloak but you should not be prompted for CAC auth. You'll notice stigman as the realm.
Yes, You need to sign into keycloak as the admin and create the user accounts. Review my fork in the section entitled Authenticating to STIG Manager with Username and Password. You can rename/reuse one of the 5 existing user accounts that automatically get created with Stigman. I added some additional directions that should help clarify how to setup the accounts.
I'll make sure I add some more notes to my forked repo so others can benefit from our thread here!

scjohnson1988 Oct 30, 2023

So I ended up re-doing it all from scratch from your fork, and I had little-to-no issues this time. For some reason I didn't have any issues using a hostname in the wildcard SAN either. I think in my original config, I was using the keycloak image, but I was using the stigman-auth image for this one. I may play around a bit to see if I can get PKI going alongside username and password, but your repo was really helpful.

Either way,

cd-rite · 2023-06-25T16:50:22Z

cd-rite
Jun 25, 2023
Maintainer

Awesome!
We are leery of offering anything that purports to be "production" ready because deployers' requirements and environments can vary so widely. Our goal with that repo is to offer a sample CAC-enabled deployment, but we will use your inputs/PRs to offer additional guidance for those in your situation.

Thanks for the feedback, and good luck with the tool!!

1 reply

jeremyatourville Jun 26, 2023
Author

Understood about differing requirements. This does at least get you really close to something that is "production ready" NGINX is doing the TLS termination. It will be up to the users to generate proper certs for their environment. Like I said early on in this thread, my main goal was to have a shared environment where multiple users were working from a common interface, so for that reason localhost only was not going to work.

BTW, we are liking the tool so far. Having a common view across your environment has been very helpful.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stig-Manager in Docker as a shared resource, (not localhost only mode) resolvable by DNS #1018

{{title}}

Replies: 3 comments 6 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

Stig-Manager in Docker as a shared resource, (not localhost only mode) resolvable by DNS #1018

jeremyatourville Jun 16, 2023

Replies: 3 comments · 6 replies

cd-rite Jun 20, 2023 Maintainer

jeremyatourville Jun 21, 2023 Author

cd-rite Jun 21, 2023 Maintainer

jeremyatourville Jun 23, 2023 Author

scjohnson1988 Oct 27, 2023

jeremyatourville Oct 30, 2023 Author

scjohnson1988 Oct 30, 2023

cd-rite Jun 25, 2023 Maintainer

jeremyatourville Jun 26, 2023 Author

jeremyatourville
Jun 16, 2023

Replies: 3 comments 6 replies

cd-rite
Jun 20, 2023
Maintainer

jeremyatourville Jun 21, 2023
Author

cd-rite
Jun 21, 2023
Maintainer

jeremyatourville Jun 23, 2023
Author

jeremyatourville Oct 30, 2023
Author

cd-rite
Jun 25, 2023
Maintainer

jeremyatourville Jun 26, 2023
Author