November 12, 2024 Windows Updates Break Windowsfs + Groot on Windows Stemcell 2019.79

[geofffranks]

Current behavior

Windows stemcells built after Nov 12, 2024 via stembuild, or the pre-built 2019.79 version of stemcells contain Windows updates from the November patch Tuesday, which introduces an incompatibility with certain WIndowsFS releases.

The issue results in the pre-start script for groot hanging indefinitely when unpacking the filesystem.

Stemcells with the Nov 12, 2024 patches work with windowsfs 2.68.0 and prior, but not 2.69.0 and later. Stemcells built withou tthe Nov 12, 2024 patches work with all versions of windowsfs.

Desired behavior

Fully patched stemcells can successfully unpack the fully patched windowsfs release.

Affected Version

2019.79+ and windowsfs releases 2.69.0+

[amhuber]

FYI, I tested using the December patches and the behavior from the groot pre-start script is identical.

[geofffranks]

Opened microsoft/hcsshim#2342 with microsoft on this. We had no luck isolating individual layers of our image to see what might be hanging. We found one layer (setting no password expiry via the net command), modified it to use wmic instead, and that layer worked.. But then a subsequent layer failed, and all it did was delete an installer file after the installation step had completed. Two days later, the same combination of layers worked fine, even when deleting the installer file.

[winkingturtle-vmw]

@amhuber Fix for this issue has been released in v2.32.0

[amhuber]

Great, thank you.