Server/Linux: Listen on abstract X UDS by default

Recent versions of Linux configure the primary X server to listen on
both a pathname Unix domain socket (/tmp/.X11-unix/X$n, where $n is the
X display number) and an abstract Unix domain socket
(@/tmp/.X11-unix/X$n).  Referring to this long thread:

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/LDZNJJOQ5U74DRLBNSDOXJFPX5XIWCLT/

an issue exists with fixed-name abstract Unix domain sockets whereby an
unprivileged user can create a dummy abstract UDS with a specific name
and thus prevent an application that requires an abstract UDS with the
same name from starting.  However, in the case of X11, the use of a
shared world-writable filesystem directory (/tmp/.X11-unix) for all
pathname X11 Unix domain sockets already exposes the system to precisely
this type of attack.  That is a limitation of X11 that cannot be
addressed within the scope of TurboVNC.  (The same attack is possible
with local X11 sessions.)

In fact, I posit that it is safer for a TurboVNC session to claim the
abstract UDS associated with its X display number.  Otherwise a user
could conceivably build a special version of Xvnc that ignores the X11
lock file, and they could hijack the X display of another user's
TurboVNC session (that was started with '-nolisten local', which was the
default) by starting their special version of Xvnc with
'-listen local -nolisten unix', instructing Xvnc to listen on the same X
display as the other user's TurboVNC session, and instructing Xvnc to
listen on a currently unclaimed RFB port.  This exploit works because
modern Linux X11 applications prefer to communicate with the X server
using the abstract UDS, if available, rather than the pathname UDS.

The exploit described above is precisely what happened if a TurboVNC
session was listening on Display :1 and a local session was started
using GDM (if GDM was configured with WaylandEnable=false.)  GDM ignored
the presence of /tmp/.X1-lock and /tmp/.X11-unix/X1 and used Display :1
for the local session, thus stomping on the TurboVNC session's display.
(Refer to: https://bugzilla.redhat.com/show_bug.cgi?id=1673793.)
If the TurboVNC session listens on the abstract UDS as well as the
pathname UDS associated with Display :1, then GDM will choose Display :2
for the local session.

Should this change prove problematic, it can easily be reverted on a
system-wide or per-user basis by setting

$serverArgs = "-nolisten local";

in turbovncserver.conf.

This commit also documents bc73242.

CMakeLists.txt

@@ -16,7 +16,7 @@ project(TurboVNC NONE)
 string(TOLOWER ${CMAKE_PROJECT_NAME} CMAKE_PROJECT_NAME_LC)
 set(VERSION 3.0.4)
 set(DOCVERSION 3.0.4)
-set(COPYRIGHT_YEAR "1999-2023")
+set(COPYRIGHT_YEAR "1999-2024")
 set(COPYRIGHT "The VirtualGL Project and many others (see README.txt)")
 set(URLTEXT "Visit http://www.TurboVNC.org for more information on TurboVNC")
 

ChangeLog.md

@@ -21,6 +21,19 @@ CVE-2022-2320, CVE-2022-4283, CVE-2022-46340, CVE-2022-46341, CVE-2022-46342,
 CVE-2022-46343, CVE-2022-46344, CVE-2023-0494, and CVE-2023-1393) from the
 xorg-server 21.1.x code base.
 
+4. By default, each instance of the Linux TurboVNC Server now listens on the
+abstract Unix domain socket, in addition to the pathname Unix domain socket
+(under **/tmp/.X11-unix**), associated with its X display number.  This
+prevents recent versions of GDM, when configured with `WaylandEnable=false`,
+from attempting to use Display :1 for the local session if a TurboVNC session
+is already using Display :1.  The previous behavior can be restored by passing
+`-nolisten local` to `vncserver` or adding `-nolisten local` to the
+`$serverArgs` variable in **turbovncserver.conf**.
+
+5. The `vncserver` script now checks whether the abstract Unix domain socket
+associated with an X display number is in use before assuming that the display
+number is available.
+
 
 3.0.3
 =====

README.txt

@@ -24,7 +24,7 @@ Legal
 
 TurboVNC is
 
-    Copyright (C) 2009-2023 D. R. Commander
+    Copyright (C) 2009-2024 D. R. Commander
     Copyright (C) 2011-2019 Brian P. Hinz
     Copyright (C) 2010 University Corporation for Atmospheric Research
     Copyright (C) 2004-2008 Sun Microsystems, Inc.

unix/Xvnc/programs/Xserver/CMakeLists.txt

@@ -82,6 +82,9 @@ else()
 	set(XTRANS_SEND_FDS 1)
 endif()
 
+if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
+	set(LISTEN_LOCAL 1)
+endif()
 configure_file(include/dix-config.h.in include/dix-config.h)
 
 if(APPLE)

unix/Xvnc/programs/Xserver/include/dix-config.h.in

@@ -69,6 +69,9 @@
 /* Listen on Unix socket */
 #define LISTEN_UNIX
 
+/* Listen on abstract socket */
+#cmakedefine LISTEN_LOCAL
+
 /* Have setitimer support */
 #cmakedefine HAVE_SETITIMER