How to exclude DLLs in subdirectories?

[luziferius]

Hi,

I’d like to bundle a PyQt5-based application for deployment on Windows and would like to cut down the package size (currently about 200 MiB) by excluding some unused DLLs (about 45 MiB in total).

These get placed in build\exe.win-amd64-3.8\lib\PyQt5\Qt5\bin. I’m not using any 3D rendering, so I won’t need opengl32sw.dll (20MiB), d3dcompiler_47.dll (4MiB) and a bunch of other Qt5 libraries. The application works fine after deleting them manually from the build, but that’s not ideal. So the ideal solution would be to exclude them in setup.py.

I tried these variants:

• "bin_excludes": ["opengl32sw.dll",],
• "bin_excludes": ["opengl32sw",],
• "bin_excludes": ["Qt5/bin/opengl32sw.dll",],
• "bin_excludes": ["Qt5\bin\opengl32sw.dll",],
• "bin_excludes": ["lib/Qt5/bin/opengl32sw.dll",],
• "bin_excludes": ["lib\Qt5\bin\opengl32sw.dll",],

But none of them seemed to work.

Is there a way to make this work, other than deleting them after the build using a batch script?

Update 1

All attempts were unsuccessful, so I started digging in the cx_freeze source code. I added this to freeze.py

from pathlib import Path
log_file = Path("C:/Users/Thomas/freeze.txt")

def log(s: Any):
    with log_file.open("at") as f:
        print(s, file=f)

And then logged each file passed to def _ShouldCopyFile(self, path):

This is the result:

c:\windows\system32\user32.dll
c:\windows\system32\shlwapi.dll
c:\windows\system32\api-ms-win-crt-math-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-stdio-l1-1-0.dll
c:\windows\system32\kernel32.dll
c:\windows\system32\api-ms-win-crt-runtime-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-locale-l1-1-0.dll
c:\users\thomas\appdata\local\programs\python\python38\python38.dll
c:\windows\system32\api-ms-win-crt-heap-l1-1-0.dll
c:\users\thomas\appdata\local\programs\python\python38\vcruntime140.dll
c:\users\thomas\appdata\local\programs\python\python38\python3.dll
c:\users\thomas\downloads\mtgproxyprinter-fd13c997cf\venv\lib\site-packages\cx_freeze\bases\win32gui.exe
c:\users\thomas\downloads\mtgproxyprinter-fd13c997cf\venv\lib\site-packages\pyqt5\qtcore.pyd
c:\users\thomas\appdata\local\programs\python\python38\python3.dll
c:\windows\system32\kernel32.dll
c:\users\thomas\appdata\local\programs\python\python38\vcruntime140.dll
c:\windows\system32\api-ms-win-crt-runtime-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-stdio-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-string-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-heap-l1-1-0.dll
c:\users\thomas\downloads\mtgproxyprinter-fd13c997cf\venv\lib\site-packages\pyqt5\qtgui.pyd
c:\users\thomas\appdata\local\programs\python\python38\python3.dll
c:\windows\system32\kernel32.dll
c:\users\thomas\appdata\local\programs\python\python38\vcruntime140.dll
c:\windows\system32\api-ms-win-crt-heap-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-runtime-l1-1-0.dll
c:\users\thomas\downloads\mtgproxyprinter-fd13c997cf\venv\lib\site-packages\pyqt5\qtprintsupport.pyd
c:\users\thomas\appdata\local\programs\python\python38\python3.dll
c:\windows\system32\kernel32.dll
c:\users\thomas\appdata\local\programs\python\python38\vcruntime140.dll
c:\windows\system32\api-ms-win-crt-heap-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-runtime-l1-1-0.dll
c:\users\thomas\downloads\mtgproxyprinter-fd13c997cf\venv\lib\site-packages\pyqt5\qtwidgets.pyd
c:\users\thomas\appdata\local\programs\python\python38\python3.dll
c:\windows\system32\kernel32.dll
c:\users\thomas\appdata\local\programs\python\python38\vcruntime140.dll
c:\windows\system32\api-ms-win-crt-heap-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-runtime-l1-1-0.dll
c:\users\thomas\downloads\mtgproxyprinter-fd13c997cf\venv\lib\site-packages\ijson\backends\_yajl2.cp38-win_amd64.pyd
c:\users\thomas\appdata\local\programs\python\python38\python38.dll
c:\windows\system32\kernel32.dll
c:\users\thomas\appdata\local\programs\python\python38\vcruntime140.dll
c:\windows\system32\api-ms-win-crt-heap-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-stdio-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-runtime-l1-1-0.dll
c:\windows\system32\api-ms-win-crt-convert-l1-1-0.dll
c:\users\thomas\appdata\local\programs\python\python38\dlls\_bz2.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\_ctypes.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\_decimal.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\_elementtree.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\_hashlib.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\_lzma.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\_queue.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\_socket.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\_sqlite3.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\_ssl.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\pyexpat.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\select.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\unicodedata.pyd
c:\users\thomas\appdata\local\programs\python\python38\dlls\libcrypto-1_1.dll
c:\users\thomas\appdata\local\programs\python\python38\dlls\libssl-1_1.dll
c:\users\thomas\appdata\local\programs\python\python38\dlls\libffi-7.dll
c:\users\thomas\appdata\local\programs\python\python38\dlls\sqlite3.dll

As can be seen, the requested libs (e.g. lib\PyQt5\Qt5\bin\opengl32sw.dll) are not even present in the output. So it seems, the DLLs got copied in a recursive directory copy. Further investigating…

Update 2

It seems my initial question was misguided. cx_freeze copies the whole PyQt5 site-package directory, including everything in there.
If that can’t be customized, I can remove unnecessary stuff when creating the virtual environment.

[marcelotduarte]

The most propably is:
"bin_excludes": [r"lib\PyQt5\Qt5\bin\opengl32sw.dll",],
or
"bin_excludes": ["lib\\PyQt5\\Qt5\\bin\\opengl32sw.dll",],

[luziferius]

Alright, forgot about backslash escaping… I’ll try that (or use pathlib.Path, like I usually do to not run into this kind of issue…)

Thanks! I’ll try that.

[marcelotduarte]

Not only the backslash escaping, but the start of the path "lib\\PyQt5\\Qt5\\bin\\"

[luziferius]

Not only the backslash escaping, but the start of the path "lib\\PyQt5\\Qt5\\bin\\"

I copied that from the Explorer’s address bar. I have no clue why PyQt5 got missing. And I didn’t see that. You are right. Thanks again!

[luziferius]

Nope.
Doesn’t work. I tried this (adding sqlite3.dll as another test case):

 "bin_excludes": [
        str(Path("lib/PyQt5/Qt5/bin/opengl32sw.dll")),
        "lib\\PyQt5\\Qt5\\bin\\opengl32sw.dll",
        "PyQt5\\Qt5\\bin\\opengl32sw.dll",
        "Qt5\\bin\\opengl32sw.dll",
        "bin\\opengl32sw.dll",
        "opengl32sw.dll",
        "sqlite3.dll",
],

The only DLL removed was sqlite3.dll. So it works for DLLs directly below the lib directory.

opengl32sw.dll was still present. I think I’ll use a script to remove them after the build finished.

Edit: bdist_msi seems to re-build the application, so it’s not possible to reliably remove the DLLs in this case manually.

[marcelotduarte]

bdist_msi seems to re-build the application, so it’s not possible to reliably remove the DLLs in this case manually.

You can add 'skip_build': True option to the bdist_msi_options.

[marcelotduarte]

In the previous answer, I missed a point. The path of bin_excludes should be the full file name of the source binary (dll) or the base name of the binary. Not the relative target.

Source: https://github.com/marcelotduarte/cx_Freeze/blob/main/cx_Freeze/freezer.py#L314

def _should_copy_file(self, path):
        """
        Return true if the file should be copied to the target machine. This is
        done by checking the binPathIncludes, binPathExcludes, binIncludes and
        binExcludes configuration variables using first the full file name,
        then just the base file name, then the file name without any version
        numbers.
        Files are included unless specifically excluded but inclusions take
        precedence over exclusions."""

[luziferius]

I totally forgot to report back, sorry.

The root cause here was that PyQt5 includes the Qt5 distribution in the site-packages under PyQt5/Qt5, which in turn contains many files I don’t want to package, like plugins and unused DLLs.

I’m using the suggested 'skip_build': True option and a manual cleanup script that simply deletes all unwanted files from the distribution before packaging.

[marcelotduarte]

cx_Freeze 6.12 the issues with duplicated files using PyQt5.