Temporarily disabled exposing the Dask diagnostic dashboard (#872)

Temporarily disabled exposing the Dask diagnostic dashboard port to the
docker setup.
Executing docker containers after each other using docker compose, leads
sometimes to the case that the port is already in use.

We should come up with a solution if we want to use the dask dashboard
again.

Fix: #870

---------

Co-authored-by: Robbe Sneyders <robbe.sneyders@ml6.eu>

src/fondant/pipeline/compiler.py

@@ -27,7 +27,9 @@
 
 logger = logging.getLogger(__name__)
 
-DASK_DIAGNOSTIC_DASHBOARD_PORT = 8787
+# export DASK_DIAGNOSTICS_PORT="" to get a dynamic port assigned
+DASK_DIAGNOSTICS_PORT = os.environ.get("DASK_DIAGNOSTICS_PORT", ":8787")
+
 KubeflowCommandArguments = t.List[t.Union[str, t.Dict[str, str]]]
 
 
@@ -261,10 +263,7 @@ def _generate_spec(
             if extra_volumes:
                 volumes.extend(extra_volumes)
 
-            ports: t.List[t.Union[str, dict]] = []
-            ports.append(
-                f"{DASK_DIAGNOSTIC_DASHBOARD_PORT}:{DASK_DIAGNOSTIC_DASHBOARD_PORT}",
-            )
+            ports = [f"8787{DASK_DIAGNOSTICS_PORT}"]
 
             services[component_id] = {
                 "entrypoint": entrypoint,