Assertion failed: g.IO.KeyMods with GLFW backend

[michaelquigley]

Dear ImGui 1.77 (17700)
--------------------------------
sizeof(size_t): 8, sizeof(ImDrawIdx): 4, sizeof(ImDrawVert): 20
define: __cplusplus=199711
define: _WIN32
define: _WIN64
define: _MSC_VER=1926
--------------------------------
io.BackendPlatformName: imgui_impl_glfw
io.BackendRendererName: imgui_impl_opengl3
io.ConfigFlags: 0x00000000
io.ConfigInputTextCursorBlink
io.ConfigWindowsResizeFromEdges
io.ConfigWindowsMemoryCompactTimer = 60.0f
io.BackendFlags: 0x0000000E
 HasMouseCursors
 HasSetMousePos
 RendererHasVtxOffset
--------------------------------
io.Fonts: 1 fonts, Flags: 0x00000000, TexSize: 512,64
io.DisplaySize: 1280.00,720.00
io.DisplayFramebufferScale: 1.00,1.00
--------------------------------
style.WindowPadding: 8.00,8.00
style.WindowBorderSize: 1.00
style.FramePadding: 4.00,3.00
style.FrameRounding: 0.00
style.FrameBorderSize: 0.00
style.ItemSpacing: 8.00,4.00
style.ItemInnerSpacing: 4.00,4.00

I'm running into something that I can't seem to get to the bottom of, and I'm hoping you can help. I'm using a framework on top of ImGui, which implements key bindings and actions. I'm running into a situation where when I execute an action from a key binding, and then I launch a native file dialog, I'm running into an assert in ImGui when the dialog returns control back to ImGui.

I get the following assert:

Assertion failed: g.IO.KeyMods == expected_key_mod_flags && "Mismatching io.KeyCtrl/io.KeyShift/io.KeyAlt/io.KeySuper vs io.KeyMods", file ..\imgui\imgui.cpp, line 6765

My code is structured like this:

inline void host::draw(const ImVec2& size) {
    ImGui_ImplOpenGL3_NewFrame();
    ImGui_ImplGlfw_NewFrame();
    ImGui::NewFrame();

    _app->draw(ImGui::GetIO().DisplaySize);
    _app->respond();

    ImGui::Render();
    glViewport(0, 0, size.x, size.y);
    glClearColor(_bg.x, _bg.y, _bg.z, _bg.w);
    glClear(GL_COLOR_BUFFER_BIT);
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());
    glfwSwapBuffers(_window);
}

void workspace_app::respond() {
    auto io = ImGui::GetIO();
    if(!io.WantCaptureKeyboard) {
        for(const auto& action : _actions) {
            if(action->triggered(io)) {
                action->execute();
                return;
            }
        }
    }
}

bool action::triggered(const ImGuiIO& io) const {
    return _key_binding->triggered(io);
}

bool key_binding::triggered(const ImGuiIO& io) const {
    return (io.KeyCtrl == ctrl && io.KeyAlt == alt && io.KeyShift == shift && ImGui::IsKeyPressed(key));
}

The workspace_app::respond is the concrete overload responding to the _app->respond() line in host::draw above.

When I invoke this same code path through clicking on a user interface element, the code path works perfectly. It only triggers this assert when I use the keyboard shortcut (CTRL-S) in my case, to invoke the code path. If I change my keyboard shortcut to just S instead of CTRL-S, it also works perfectly. It only seems to hit this assert when modifier keys are involved.

Any insight would be very appreciated.

[ocornut]

Hello,

The assert generally triggers when you have code modifying the io.KeyCtrl io.KeyAlt or io.KeyShift modifiers after ImGui::NewFrame(). I think you should search in your codebase for such modifications.

[ocornut]

Note the comment exactly above this assert:

// Verify that io.KeyXXX fields haven't been tampered with. Key mods should not be modified between NewFrame() and EndFrame()
// One possible reason leading to this assert is that your backends update inputs _AFTER_ NewFrame().
const ImGuiKeyModFlags expected_key_mod_flags = GetMergedKeyModFlags();
IM_ASSERT(g.IO.KeyMods == expected_key_mod_flags && "Mismatching io.KeyCtrl/io.KeyShift/io.KeyAlt/io.KeySuper vs io.KeyMods");

[michaelquigley]

Yes. I already saw that. But my code does is not directly modify those values at all. That was the first thing I looked for.

[michaelquigley]

In fact, if I add:

        auto io = ImGui::GetIO();
        auto mods = io.KeyMods;
        ImGui::NewFrame();

and after my code:

        ImGui::GetIO().KeyMods = mods;
        ImGui::Render();

I still get the same assertion.

I also tried capturing those fields after NewFrame:

        ImGui::NewFrame();
        auto io = ImGui::GetIO();
        auto mods = io.KeyMods;
        auto ctrl = io.KeyCtrl;
        auto alt = io.KeyAlt;
        auto shift = io.KeyShift;

Same result.

Could something be occuring because the dialog is blocking this loop for a period of seconds (or more)?

[ocornut]

Well, something does and it's not core imgui.
The only piece of code that you are running which intentionally write to io.KeyXXX are the key handlers in imgui_impl_glfw.cpp which are generally triggerd when your app calls glfwPollEvents().

Could something be occuring because the dialog is blocking this loop for a period of seconds (or more)?

Yes if you keep polling GLFW events, as stated above.

I still get the same assertion.

Because that's making it wrong. If you move the first mods = assignment after ImGui::NewFrame() that's a correct check.

[michaelquigley]

I don't see how anything in my code could be modifying ImGui::GetI().KeyMods within the following block, yet the assert persists:

inline void host::draw(const ImVec2& size) {
    _size = size;

    ImGui_ImplOpenGL3_NewFrame();
    ImGui_ImplGlfw_NewFrame();
    ImGui::NewFrame();
    auto saved_mods = ImGui::GetIO().KeyMods;
    
    _app->draw(ImGui::GetIO().DisplaySize);
    _app->respond();

    ImGui::GetIO().KeyMods = saved_mods;
    ImGui::Render();
    glViewport(0, 0, size.x, size.y);
    glClearColor(_bg.x, _bg.y, _bg.z, _bg.w);
    glClear(GL_COLOR_BUFFER_BIT);
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());
    glfwSwapBuffers(_window);
}

And keyboard shortcuts work delightfully everywhere other than when I launch a native file dialog, delaying this loop.

Code down the stack, below _app->respond() is launching a native file dialog. The entire loop is blocked when that happens. When it returns, it's throwing the assert.

[ocornut]

I gave you all the information you need to debug your issue. You can use printf or a debugger to track it down.

[michaelquigley]

The following code resolves the issue. I'll take it from here. Thank you.

inline void host::draw(const ImVec2& size) {
    _size = size;

    ImGui_ImplOpenGL3_NewFrame();
    ImGui_ImplGlfw_NewFrame();
    ImGui::NewFrame();
    auto saved_ctrl = ImGui::GetIO().KeyCtrl;
    auto saved_alt = ImGui::GetIO().KeyAlt;
    auto saved_shift = ImGui::GetIO().KeyShift;
    auto saved_super = ImGui::GetIO().KeySuper;

    _app->draw(ImGui::GetIO().DisplaySize);
    _app->respond();
 
    ImGui::GetIO().KeyCtrl = saved_ctrl;
    ImGui::GetIO().KeyAlt = saved_alt;
    ImGui::GetIO().KeyShift = saved_shift;
    ImGui::GetIO().KeySuper = saved_super;
    ImGui::Render();
    glViewport(0, 0, size.x, size.y);
    glClearColor(_bg.x, _bg.y, _bg.z, _bg.w);
    glClear(GL_COLOR_BUFFER_BIT);
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());
    glfwSwapBuffers(_window);
}

[michaelquigley]

(Also, thank you for the help... Surely you can understand how these things go. My code doesn't directly alter those values. There's got to be some kind of a compiler side effect or some minor mistake or something. And I'm kind of puzzled how this only happens in this one specific case... but yes, you're right, something is changing those fields between those 2 statements, and it's ultimately my problem... But thanks!)

[ocornut]

As a quick test,

I've tried today to use https://github.com/mlabbe/nativefiledialog to create a long blocking dialog in the middle of the Dear ImGui frame, calling it while modifiers were held, and tested this with GLFW, SDL and Win32 backends without a problem.

Maybe your and @str0yd app have some kind of reentrant/threaded code calling GLFW back-end events during the blocking call. I'm not sure.

[str0yd]

This is strange. I use my own key callback:

void key_callback(GLFWwindow* window, GLint key, GLint scancode, GLint action, GLint mode)
{
	if (!ImGui::GetIO().WantCaptureKeyboard) 
	{ // event not handled by gui
		if (key >= 0 && key < 1024)
		{
			if (action == GLFW_PRESS)
				sps_visual.keys[key] = GL_TRUE;
			else if (action == GLFW_RELEASE) {
				sps_visual.keys[key] = GL_FALSE;
				sps_visual.keys_processed[key] = GL_FALSE;
			}
		}
	}
}

I first set all my callbacks:

glfwSetKeyCallback(window, key_callback);
glfwSetCharModsCallback(window, char_callback);
glfwSetMouseButtonCallback(window, mouse_click_callback);
glfwSetScrollCallback(window, scroll_callback);
glfwSetCursorPosCallback(window, mouse_callback);

Then I initialize ImGui:

ImGui_ImplGlfw_InitForOpenGL(window, true);

Usually ImGui should call my own key callback function at the right time, shouldn't it?

I don't use the callbacks anywhere else.

Could it maybe be something with the Windows Openfile dialog? I use the GetOpenFileNameA function: https://docs.microsoft.com/en-us/windows/win32/api/commdlg/nf-commdlg-getopenfilenamea

[michaelquigley]

As a quick test,

I've tried today to use https://github.com/mlabbe/nativefiledialog to create a long blocking dialog in the middle of the Dear ImGui frame, calling it while modifiers were held, and tested this with GLFW, SDL and Win32 backends without a problem.

Maybe your and @str0yd app have some kind of reentrant/threaded code calling GLFW back-end events during the blocking call. I'm not sure.

I definitely do not have any threaded or re-entrant code as part of my application, and certainly none that interact with GLFW directly.

I do have a callback registered with GLFW to invoke my draw loop when the host window gets resized. That draw loop contains this code:

ImGui_ImplOpenGL3_NewFrame();
ImGui_ImplGlfw_NewFrame();
ImGui::NewFrame();

ImGuiIO& io = ImGui::GetIO();
auto saved_ctrl = io.KeyCtrl;
auto saved_alt = io.KeyAlt;
auto saved_shift = io.KeyShift;
auto saved_super = io.KeySuper;

// Application Drawing Here

io.KeyCtrl = saved_ctrl;
io.KeyAlt = saved_alt;
io.KeyShift = saved_shift;
io.KeySuper = saved_super;
ImGui::Render();
glViewport(0, 0, size.x, size.y);
glClearColor(_bg.x, _bg.y, _bg.z, _bg.w);
glClear(GL_COLOR_BUFFER_BIT);
ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());
glfwSwapBuffers(_window);

But I'm pretty sure that's not causing this issue.

I've left my workaround in place because it doesn't seem to be adversely affecting the behavior of any part of my application. I've been through my codebase extensively... there does not seem to be anything that's directly changing the state of any ImGui structures in any way. Reading them... extensively. Changing them... no.

[ocornut]

You know which function in the GLFW backends is altering the key mods so I am surprised you haven’t set breakpoints to figure out exactly what’s calling it and when.

[str0yd]

I had no time to test it yet, will do this tomorrow.

[str0yd]

I think the problem is in the ImGui_ImplGlfw_KeyCallback. I press "Ctrl + S" to quicksave my things. While pressing "Ctrl + S" the Windows file dialog opens. I relase "Ctrl + S" and choose a file. As soon as I leave the file dialog the Keykallback is triggered (I relased Ctrl and S) and resets the io.KeyCtrl. After resetting the program runs and ASSERTS

g.IO.KeyMods == expected_key_mod_flags // same as GetMergedKeyModFlags()
// g.IO.KeyMods is 1, expected_key_mod_flags  is 0

This evaluates FALSE because the ImGui_ImplGlfw_KeyCallback has already reset the Ctrl flag (g.IO.KeyMods is set at the beginning of the frame).

I think this is a bug beacause if you could press and release the Ctrl button in one frame you should also get this error message.

I'm not sure when exactly GLFW calls the kallback function. If it is periodically or inerrupting. But if I'm right with my theory here it should call the callback interrupting.

[ocornut]

I'm not sure when exactly GLFW calls the kallback function

You can set a breakpoint to find out but I've also written it above. They are called when you call glfwPollEvents() and the point is your should not call glfwPollEvents() in the middle of an ImGui frame, so you should look at where you are calling it.

[michaelquigley]

You can set a breakpoint to find out but I've also written it above. They are called when you call glfwPollEvents() and the point is your should not call glfwPollEvents() in the middle of an ImGui frame, so you should look at where you are calling it.

My code does not call glfwPollEvents anywhere, ever, between ImGui::NewFrame and ImGui::Render, and still has this issue.

Maybe there's some sort of strange side-effect happening in GLFW when opening a dialog, that's causing that callback to execute despite not calling glfwPollEvents? Unfortunately it's not super easy for me to fixture this up in a debugger in a way that makes this easy to isolate.

[str0yd]

I'm debugging the code right now. My main loop calls the pollevents before any imgui stuff:

glfwPollEvents();

...

// initialize GUI for new frame
ImGui_ImplOpenGL3_NewFrame();
ImGui_ImplGlfw_NewFrame();
// draw GUI
ImGui::NewFrame();
// get all the gui stuff
VisualGui::getInstance().DrawGui();
// render the gui
ImGui::Render();
ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());

...

I do not call the pollEvents anywhere else in my program.

Here is an image of the callstack:

As you can see the breakpoint is in the ImGui_ImplGlfw_KeyCallback function. I come from extern code. The curser (green arrow) points in my function VisualGui::openFile. In this function the curser is in the

iResult = GetSaveFileNameA(&ofn);

Sooo it looks like the GetSaveFileNameA (a Windows function) calls the glfwPollEvents.

[michaelquigley]

glfwPollEvents();

...

// initialize GUI for new frame
ImGui_ImplOpenGL3_NewFrame();
ImGui_ImplGlfw_NewFrame();
// draw GUI
ImGui::NewFrame();
// get all the gui stuff
VisualGui::getInstance().DrawGui();
// render the gui
ImGui::Render();
ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());

...

I do not call the pollEvents anywhere else in my program.

My code is structured exactly the same.

[ocornut]

I looked at GLFW codebase and at a first glance it seems like only the _glfwPlatformPollEvents() function dispatch win32 messages which turns into callback events.

I have created a minimum repro calling GetSaveFileNameA() in the middle of the dear imgui frame and couldn't get things to trigger.

As this point if you would like to correctly get to the end of this, could you try creating a minimum repro for it? Ideally sticking some code inside existing examples/ app.

[michaelquigley]

I can probably put something together this weekend.

[ocornut]

OK so weirdly I tried to build a repro in another sample project and it worked this time...

In examples/example_glfw_opengl3/main.cpp

#include <windows.h>
#include <commdlg.h>
#include <GLFW/glfw3.h>
#define GLFW_EXPOSE_NATIVE_WIN32
#include <GLFW/glfw3native.h>   // for glfwGetWin32Window

[...]

if (ImGui::IsKeyPressed(GLFW_KEY_O))
{
    CHAR buf[256] = "hello.txt";
    OPENFILENAME of;
    memset(&of, 0, sizeof(of));
    of.lStructSize = sizeof(OPENFILENAME);
    of.hwndOwner = NULL;
    of.lpstrFilter = NULL;
    of.lpstrFile = buf;
    of.hwndOwner = glfwGetWin32Window(window);
    of.nMaxFile = 256;
    of.lpstrFileTitle = NULL;
    of.nMaxFileTitle = 0;
    of.lpstrInitialDir = (LPSTR)NULL;
    of.Flags = OFN_SHOWHELP | OFN_OVERWRITEPROMPT;
    of.lpstrTitle = "lpstrTitle";
    ::GetSaveFileName(&of);
}

This repros it for me.

Installed GLFW with debug symbols
# vcpkg install glfw3:static

With callstack:

>	example_glfw_opengl3.exe!__glfwInputKey�()	C
 	example_glfw_opengl3.exe!__glfwInputWindowFocus�()	C
 	example_glfw_opengl3.exe!_glfwGetWin32Window�()	C
 	user32.dll!__InternalCallWinProc@20�()	Unknown
 	user32.dll!UserCallWinProcCheckWow()	Unknown
 	user32.dll!CallWindowProcAorW(long (*)(struct HWND__ *,unsigned int,unsigned int,long),struct HWND__ *,enum _WM_VALUE,unsigned int,long,int)	Unknown
 	user32.dll!_CallWindowProcW@20�()	Unknown
 	opengl32.dll!_wglWndProc@16�()	Unknown
 	user32.dll!__InternalCallWinProc@20�()	Unknown
 	user32.dll!UserCallWinProcCheckWow()	Unknown
 	user32.dll!DispatchClientMessage()	Unknown
 	user32.dll!___fnDWORD@4�()	Unknown
 	ntdll.dll!_KiUserCallbackDispatcher@12�()	Unknown
 	comdlg32.dll!_InvokeNewFileOpenSave(struct IFileDialog *,unsigned short,struct HWND__ *,struct _OFNINITINFO *,struct HWND__ *)	Unknown
 	comdlg32.dll!_CreateNewFileOpenSaveInProc(unsigned short,struct HWND__ *,struct _OFNINITINFO *)	Unknown
 	comdlg32.dll!NewGetFileName(struct OPENFILEINFO *,int)	Unknown
 	comdlg32.dll!_GetFileName@8�()	Unknown
 	comdlg32.dll!_GenericGetFileNameA@8�()	Unknown
 	comdlg32.dll!_GetSaveFileNameA@4�()	Unknown
 	example_glfw_opengl3.exe!main(int __formal, char * * __formal) Line 182	C++

[ocornut]

So GLFW is clearing keyboard input when the windows loses focus, and with those native modal dialog the events are pushed mid-frame.

With the current direct-write to IO i don't have a super satisfying solution right now. When we transitioning to queuing events (e.g. https://gist.github.com/ocornut/8417344f3506790304742b07887adf9f) it'll be trivial.

Considering the nature of that mid-frame modification my suggested solution would be to make the assert more lenient by only asserting on mid-frame presses rather than mid-frame release. This way, the assert would STILL catch on the common error it's trying to catch, while acting friendly on this sort of situation we stumbled on here.

[ocornut]

Commit with this fix: 3600ceb

Commit where the check was first added: ccf0cc8
It's an important check as a frequent error was to call events after ImGui::NewFrame() and while things would appear to mostly work, some of the nav queries done in NewFrame would essentially break.

Thanks all for your patience with this, glad we got this through.

[michaelquigley]

Confirmed. This fixes the issue in my codebase. Thank you!

[phynae]

I have a similar issue. When I press Ctrl + S the following error pops up. My BeginFrame -> EndFrame code looks like this.

I am using dx11 and win32 backends.

[ocornut]

I have a similar issue. When I press Ctrl + S the following error pops up. My BeginFrame -> EndFrame code looks like this.

What’s in the draw() function then?
Try to narrow it down to a minimum repro.

I don’t think the 2020 issue is related anymore, as nowadays events are queued so even if a backend receives event’s asynchronously it won’t interfere with what the assert is complaining about. It may be a problem on your end.

[phynae]

the draw function itself only uses ImGui::GetBackgroundDrawList and ImGui::GetTextSize functions to draw "certain stuff" over video games. until that error occured i have never heard of KeyMods in any situation. Therefore I certainly do not modify them activley and knowingly.

Fyi: I am internal, so maybe the game itself interfers with the keymods? idk i started that whole learning process a few weeks ago.

[ocornut]

Therefore I certainly do not modify them activley and knowingly.

You’ll need to investigate on your own. We can’t really help with that amount of information.