Feature: WebcamProcessSensor

[DennisGaida]

Is your feature request related to a problem? Please describe.
Windows Hello enabled cameras are also registered as webcams in Windows, meaning whenever you authenticate to your Windows machine using Windows Hello camera (infrared), all the automations based on webcam activity are triggered.

Describe the solution you'd like
When the webcam is active I want to know what process is using it. Many use cases for this:

• different automations based on the program I am using (e.g. Zoom/Teams call vs. Windows Camera app)
• Windows Hello authentication via camera is always leveraged using svchost.exe or SensorDataService.exe - If I would know that these processes are currently triggering the webcam, I can exclude automations - same issue here: Feature Request: Process which is using webcam sleevezipper/hass-workstation-service#92

Describe alternatives you've considered
Use HASS Workstation Service

[LAB02-Admin]

Hey @DennisGaida, good point! That'll really increase usability :)

Created a ticket for this request: hassagent-120.
I'm closing this GitHub ticket, not because I'm not doing it, but to keep everything centralised. I'll post back here when it's implemented :)

[LAB02-Admin]

Added in the latest beta: https://github.com/LAB02-Research/HASS.Agent/releases/tag/2022.13.0-beta2

[DennisGaida]

Do you want comments on the feature here or somewhere else since this issue is closed?

Currently isn't ideal in the beta. Microsoft Teams shows up as NonPackaged which is completely missing the exe name.

Checking a similar service I know that it is certainly possible to get more information from the registry (https://github.com/sleevezipper/hass-workstation-service/blob/master/hass-workstation-service/Domain/Sensors/WebcamProcessSensor.cs). Leveraging this service, Microsoft Teams shows up as:
C:#Users#<username>#AppData#Local#Microsoft#Teams#current#Teams.exe

Windows Camera shows up as WindowsCamera with the beta, but actually is Microsoft.WindowsCamera_8wekyb3d8bbwe in the registry.

[LAB02-Admin]

Do you want comments on the feature here or somewhere else since this issue is closed?

Here is fine :)

Currently isn't ideal in the beta. Microsoft Teams shows up as NonPackaged which is completely missing the exe name.

I'll tweak the parser, thanks for letting me know.

Windows Camera shows up as WindowsCamera with the beta, but actually is Microsoft.WindowsCamera_8wekyb3d8bbwe in the registry.

Yea that is by design. WindowsCamera is a lot more readable, so I parse the actual name from the reg entry.

[DennisGaida]

Just so you have some test data, here is a dump of the HKLM...\webcam hive. Please note that the SensorDataService and svchost are pretty much the most important things for automations because these represent Windows Hello camera authentication. You don't want your HA-automations every time you log into Windows ;-)

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam]
"Value"="Allow"

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Windows#System32#SensorDataService.exe]
"LastUsedTimeStart"=hex(b):fb,d5,6d,5f,c2,af,d8,01
"LastUsedTimeStop"=hex(b):b7,f0,2a,88,c2,af,d8,01

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Windows#System32#svchost.exe]
"LastUsedTimeStart"=hex(b):23,05,d2,bb,4c,b1,d8,01
"LastUsedTimeStop"=hex(b):6b,ff,d2,bc,4c,b1,d8,01

and here are a "couple" of entries for the HKCU...\webcam hive:

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam]
"Value"="Allow"

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\Microsoft.BioEnrollment_cw5n1h2txyewy]
"LastUsedTimeStart"=hex(b):7e,81,d0,d6,95,4d,d8,01
"LastUsedTimeStop"=hex(b):ce,40,0f,e4,95,4d,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\Microsoft.MicrosoftEdge_8wekyb3d8bbwe]
"Value"="Allow"

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\Microsoft.Office.OneNote_8wekyb3d8bbwe]
"Value"="Prompt"

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\Microsoft.Win32WebViewHost_cw5n1h2txyewy]
"Value"="Allow"

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\Microsoft.Windows.Photos_8wekyb3d8bbwe]
"Value"="Allow"

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\Microsoft.WindowsCamera_8wekyb3d8bbwe]
"Value"="Allow"
"LastUsedTimeStart"=hex(b):d3,4e,6c,4e,0c,af,d8,01
"LastUsedTimeStop"=hex(b):8c,c3,c4,17,0e,af,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\Microsoft.WindowsStore_8wekyb3d8bbwe]
"Value"="Allow"

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\MicrosoftCorporationII.PowerAppsforWindows10_8wekyb3d8bbwe]
"Value"="Prompt"

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged]
"Value"="Allow"

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Program Files (x86)#Common Files#LogiShrd#LogiUCDpp#LogitechCamera.exe]
"LastUsedTimeStart"=hex(b):12,ef,51,6c,69,55,d8,01
"LastUsedTimeStop"=hex(b):7e,a1,06,70,69,55,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Program Files (x86)#ManyCam Lite#ManyCamLite.exe]
"LastUsedTimeStart"=hex(b):1a,42,52,54,89,25,d8,01
"LastUsedTimeStop"=hex(b):61,c6,b7,bf,89,25,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Program Files (x86)#Microsoft#Edge Dev#Application#msedge.exe]
"LastUsedTimeStart"=hex(b):6b,b7,f7,e0,89,25,d8,01
"LastUsedTimeStop"=hex(b):81,99,75,e5,89,25,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Program Files#e2eSoft#iVCam#iVCam.exe]
"LastUsedTimeStart"=hex(b):ed,d9,39,75,d7,ff,d7,01
"LastUsedTimeStop"=hex(b):bb,b0,8b,ab,dd,ff,d7,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Program Files#Logitech#LogiCapture#bin#LogiCapture.exe]
"LastUsedTimeStart"=hex(b):44,96,82,0d,c0,7c,d8,01
"LastUsedTimeStop"=hex(b):13,98,e1,29,c0,7c,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Program Files#Microsoft Office#root#Office16#POWERPNT.EXE]
"LastUsedTimeStart"=hex(b):3c,6f,31,c1,9f,9d,d8,01
"LastUsedTimeStop"=hex(b):f6,5b,29,c3,9f,9d,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Program Files#Mozilla Firefox#firefox.exe]
"LastUsedTimeStart"=hex(b):59,a8,f3,3f,ac,1d,d8,01
"LastUsedTimeStop"=hex(b):c4,eb,ea,43,ac,1d,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Program Files#obs-studio#bin#64bit#obs64.exe]
"LastUsedTimeStart"=hex(b):dd,4d,47,a3,62,a6,d8,01
"LastUsedTimeStop"=hex(b):fb,c7,33,ab,62,a6,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Program Files#Snap Inc#Snap Camera#Snap Camera.exe]
"LastUsedTimeStart"=hex(b):90,ca,47,dc,98,1e,d8,01
"LastUsedTimeStop"=hex(b):72,0b,dc,c0,b9,1e,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#PROGRA~1#Logitech#LOGICA~1#bin#LogiCapture.exe]
"LastUsedTimeStart"=hex(b):9b,c8,6c,ef,32,7c,d8,01
"LastUsedTimeStop"=hex(b):8d,5b,35,96,33,7c,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Users#username#AppData#Local#GoToMeeting#19796#g2mvideoconference.exe]
"LastUsedTimeStart"=hex(b):f3,a1,f6,49,b4,19,d8,01
"LastUsedTimeStop"=hex(b):25,09,20,4a,b4,19,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Users#username#AppData#Local#Microsoft#Teams#current#Teams.exe]
"LastUsedTimeStart"=hex(b):0a,79,dd,ff,49,b1,d8,01
"LastUsedTimeStop"=hex(b):6d,2c,14,00,4a,b1,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Users#username#AppData#Local#Programs#ferdi#Ferdi.exe]
"LastUsedTimeStart"=hex(b):6d,8a,cd,a6,49,2a,d8,01
"LastUsedTimeStop"=hex(b):b5,d3,55,a8,49,2a,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\NonPackaged\C:#Users#username#AppData#Local#WebEx#WebEx64#Meetings#atmgr.exe]
"LastUsedTimeStart"=hex(b):49,9d,d0,ab,bf,48,d8,01
"LastUsedTimeStop"=hex(b):d4,9c,e7,d9,cb,48,d8,01

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\windows.immersivecontrolpanel_cw5n1h2txyewy]
"LastUsedTimeStart"=hex(b):01,6c,ab,3b,91,54,d8,01
"LastUsedTimeStop"=hex(b):0a,dd,ff,44,91,54,d8,01

[LAB02-Admin]

That's great stuff @DennisGaida, thank you!

[dimatx]

@LAB02-Admin, with 2022.13.0-beta3 [BETA], Microsoft Teams shows up as "non-packaged" for the WebcamProcess sensor as does Windows authentication/login prompt via Windows Hello. Is that expected behavior for this version?

[LAB02-Admin]

No, I think I forgot to add the actual fix :x I'll re-release this beta later and let you know

[dimatx]

Thanks :) Appreciate the fast turnaround!

[LAB02-Admin]

I've re-released beta3: https://github.com/LAB02-Research/HASS.Agent/releases/download/2022.13.0-beta3/HASS.Agent.Installer.exe

You can just run the installer again, no need to uninstall. Let me know how it goes!

[DennisGaida]

Latest re-released beta works (also the microphoneprocess sensor).

Any reason the WebcamProcess, WebcamActive (and Microphone equivalents) are hass.agent only? I believe the Windows service could easily just read the registry also while not being logged in.

[LAB02-Admin]

It could get the HKLM key, but not the user one when the user's logged in - the hive will be locked iirc. And no point getting it when the user's not logged in, since no app will be using anything. By logged in I mean a session, connected or not, by the way.

I guess I could add it to the service if that would be useful, with the notion that it only reads elevated usage.

[DennisGaida]

Not that important to me personally. https://github.com/sleevezipper/hass-workstation-service was able to do this (and maybe only the HKLM key, not sure). It actually led to "problems" with Windows Hello because then it is not svchost.exe using the camera, but another service.

Personally I don't need it prior to login, but it may be interesting to others (e.g. to do something especially when you authenticate with Windows Hello).

[LAB02-Admin]

Looking at the code, he gets both keys - it depends under what user the service runs whether that's useful. But I'll enable it for hass.agent's service, doesn't require extra coding anyway and users can decide for themselves.

At any rate, glad it's working for you, let me know whenever it might need tweaks :)

[dimatx]

Works for me also. Thanks for the re-release and for your contributions in general!