Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protected Event Logging is breaking execution #188

Closed
chelnak opened this issue Jun 1, 2022 · 4 comments · Fixed by #189
Closed

Protected Event Logging is breaking execution #188

chelnak opened this issue Jun 1, 2022 · 4 comments · Fixed by #189
Assignees
Labels
bug Something isn't working

Comments

@chelnak
Copy link
Contributor

chelnak commented Jun 1, 2022

Both DSC resources and anything with a provider => powershell is immediately erroring out with errors as follows:

#<Thread:0x0000000005efc6a8@C:/ProgramData/PuppetLabs/puppet/cache/lib/pwsh.rb:587 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
        1: from C:/ProgramData/PuppetLabs/puppet/cache/lib/pwsh.rb:589:in `block in read_streams'
C:/ProgramData/PuppetLabs/puppet/cache/lib/pwsh.rb:589:in `sysread': end of file reached (EOFError)

See Slack thread for more discussion and a link to the ZenDesk ticket.

@chelnak chelnak added the bug Something isn't working label Jun 1, 2022
@chelnak chelnak self-assigned this Jun 7, 2022
@chelnak
Copy link
Contributor Author

chelnak commented Jun 9, 2022

I'm not having much success with the repro for this issue.

The following image shows some basic info from my test rig.. You can see that applying test.pp works with no issue:

image

Also Protected Event Logging is enabled and encrypting events as expected:

image

The main thing that perplexes me right now is how the encryption of the logs is correlated with the output of the PowerShell provider.

@cummingscody
Copy link

cummingscody commented Jun 20, 2022

image002
Attaching docs and further info from customer...

Also, this is coupled with the GPO "Turn on PowerShell Script Block Logging" set to "Enabled". In testing I've found that the errors don't occur if "Protected Event Logging" is enabled, but that policy is not.

@chelnak
Copy link
Contributor Author

chelnak commented Jun 23, 2022

Great news! That extra info helped me replicate the issue 😄

image

chelnak added a commit that referenced this issue Jun 23, 2022
When Protected Event Logging and PowerShell Script Block logging are
enabled together the SystemRoot environment variable is a requirement.

If it is removed as part of the environment purge it causes the
PowerShell process to crash, therefore breaking the pipe between Ruby
and the remote PowerShell session.

This commit fixes the issue by creating a filtered list of environment
variables which does not include the SystemRoot environment variable.

This list can then be purged safely.
chelnak added a commit that referenced this issue Jun 23, 2022
When Protected Event Logging and PowerShell Script Block logging are
enabled together the SystemRoot environment variable is a requirement.

If it is removed as part of the environment purge it causes the
PowerShell process to crash, therefore breaking the pipe between Ruby
and the remote PowerShell session.

This commit fixes the issue by creating a filtered list of environment
variables which does not include the SystemRoot environment variable.

This list can then be purged safely.
@chelnak
Copy link
Contributor Author

chelnak commented Jun 23, 2022

Fixed in #189

For each run we reset the session environment to a last known good state. We did this by removing all environment variables, then re-setting them from a cached set we took at the start of the run.

It appears that when ScriptBlock Logging and Protected Event Logging are both enabled, removing the SystemRoot environment variable from the current session causes PowerShell to crash.

I'm unclear at this time what the dependency is but suspect that there is a process that is consuming $env:\SystemRoot when building a path to an executable.

I was able to replicate by manually invoking init.ps1 with a custom name and redirecting pwshlib to the new pipe. Once you do this you can see the whole error message. It talks about a missing environment variable...

image

A bit of trial and error narrowed it down to SystemRoot.

#189 fixes the issue by filtering out SystemRoot from the list and only removing the remainder.

The questions remaining for me are:

  1. When the two settings are combined, what causes the dependency on SystemRoot?
  2. Would it be better to scope the filtering to only happen when the two settings are both enabled?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants