[3.x] Implement Microsoft Media Foundation on camera_win.cpp

[choonway]

Proof of concept for webcam on windows on media foundation based on their examples (MFCaptureD3D)
quality is Frankenstein.
not able to test on master branch due to bugs (texture related?)
test files can be obtained from https://github.com/choonway/godot-camera_win-test

tested to work on camera on HTC Vive.
If camera can't be accessed by camera app on windows, chances are that it won't work.

limitations:
virtual cameras and some webcams don't work (HTC Vive Pro Eye cameras)
hotplug not working/implemented

[BastiaanOlij]

This looks really good at first glance, thanks !

I get why some of the stuff is all done within the CPP source file while it should really be split into a header file I can see that can be an issue with some of the MF stuff.

This won't work on the master branch because the camera server at it's heart is still very much based on OpenGL, That said if this all works fine and has its rough sides polished a bit we should merge it into master even if disabled so we can make it all work on both OpenGL and Vulkan renderers.

Definitely would suggest getting the logic to work to detect when cameras are attached after Godot has started running.

[BastiaanOlij]

For the MacOS camera stuff I made a shader that can handle YUV so we didn't need to do conversions on the CPU side. Is it worth looking at this further and adding YUY2 support as well or does it make more sense to have this RGB conversion in?

I believe there is a conversion library already linked in as a 3rd party module?

[choonway]

can't seem to find the conversion library, but found some reference code to do it within MF itself. Curious why the provided sample files didn't use them. Going to implement them.

[BastiaanOlij]

Nitpicking but it is better to use Godots internal print methods so the messages get locked to the log in the editor. You can also switch between what is considered info and what is actually an error.

[choonway]

ok. i am going to do some tidying up as well.

[BastiaanOlij]

Yes we should add logic that ensures we detect cameras being connected or disconnected to the PC.

[choonway]

there is an existing mechanism - the one used by the joypad to detect hotplug hardware - using WM_DEVICECHANGE - but it seems it's going to touch a lot of other files so I'll leave it for some other time.

[BastiaanOlij]

I've left a few remarks for improvements, I hope others have some time to put this through its paces, but it looks good enough to me for a first version on which we can improve.

[Chaosus]

@choonway Please, merge commits together as required by our pipeline (https://docs.godotengine.org/en/latest/community/contributing/pr_workflow.html#modifying-a-pull-request)

[choonway]

@BastiaanOlij There is also a bug with the R and B swapped. Give me a few days to implement the changes.

[Ansraer]

@choonway Hey, are you still working on this?

[gotnospirit]

@BastiaanOlij There is also a bug with the R and B swapped

this is because RGBQUAD is a BGR struct

typedef struct tagRGBQUAD {
        BYTE    rgbBlue;
        BYTE    rgbGreen;
        BYTE    rgbRed;
        BYTE    rgbReserved;
} RGBQUAD;

The problem with WM_DEVICECHANGE is we need the HWND handler in order to register
https://github.com/microsoft/Windows-classic-samples/blob/7cbd99ac1d2b4a0beffbaba29ea63d024ceff700/Samples/Win7Samples/multimedia/mediafoundation/MFCaptureD3D/winmain.cpp#L260-L264

For joypad, this is directly handled by OS_windows (or DisplayServerWindows in master) because its a member of this class

godot/platform/windows/os_windows.cpp

Lines 1147 to 1149 in 5600be9

	case WM_DEVICECHANGE: {
	joypad->probe_joypads();
	} break;

godot/platform/windows/display_server_windows.cpp

Lines 3186 to 3188 in 20d6672

	case WM_DEVICECHANGE: {
	joypad->probe_joypads();
	} break;

but Camera is a module!

We could maybe introduce a signal to solve this

[KnightNine]

Wanted to try this out, but trying to compile: https://github.com/godotengine/godot/tree/23e3ce8d8724ee3965dceeed3fc5bbbc84793083
results in this error:

scons: Reading SConscript files ...
Configuring for Windows: target=debug, bits=default
AttributeError: 'SConsEnvironment' object has no attribute 'has_key':
  File "C:\Users\User\Desktop\Godot_v3.1.1-stable_mono_win64\Camera Godot\SConstruct", line 348:
    detect.configure(env)
  File "C:\Users\User\Desktop\Godot_v3.1.1-stable_mono_win64\Camera Godot\./platform/windows\detect.py", line 445:
    setup_msvc_auto(env)
  File "C:\Users\User\Desktop\Godot_v3.1.1-stable_mono_win64\Camera Godot\./platform/windows\detect.py", line 153:
    if env.has_key("msvc_version"):

I tried adding the changes to Camera_win.cpp manually to my 3.5 build to run the demo project, and it results in a monochromatic camera:

😕

[saierXP]

@KnightNine Can you test this? I tried this last year.But I can't remember if I tested the #49763 or this. The conversion shader from yuv to rgb is used.
testcam.zip

[KnightNine]

@saierXP

oh good, i'll try #49763 instead.

[KnightNine]

@saierXP
here it is with #49763 instead:

Your shader was probably meant to work with this PR which I can't get the CbCr feed from. so it only appears in monocolor.
Whereas 49763 just gives me the formatted colors from the first which_feed.

thus this shader works with 49763:

shader_type canvas_item;

uniform sampler2D camera;



void fragment() {
	vec3 color;
	color.rgb = texture(camera, UV).rgb;
	
	COLOR = vec4(color,1.0);	
}

[fire]

Is there interest in migrating this to godot engine master 4? I am also investigating video playback.