Fix overflow in Viewport::FromDimensions #12669
Conversation
ghost
added
Area-Server
This and |
| @@ -677,8 +677,7 @@ void Renderer::_PaintBufferOutput(_In_ IRenderEngine* const pEngine) | |||
|
|
|||
| for (const auto& dirtyRect : dirtyAreas) | |||
| { | |||
| // Shortcut: don't bother redrawing if the width is 0. | |||
| if (dirtyRect.left == dirtyRect.right) | |||
| if (!dirtyRect) | |||
There was a problem hiding this comment.
BTW This is a bug. Some engines return dirty rects where top == bottom, and I don't think any engine returns left == right actually. This causes an invalid Viewport to be created a few lines below. In either case though, this now checks both dimensions.
|
@msftbot merge this in 1 minute |
ghost
added
the
AutoMerge
|
Hello @DHowett! Because you've given me some instructions on how to help merge this pull request, I'll be modifying my merge approach. Here's how I understand your requirements for merging this pull request:
If this doesn't seem right to you, you can tell me to cancel these instructions and use the auto-merge policy that has been configured for this repository. Try telling me "forget everything I just told you". |
ghost
merged commit 
This removes one source of potential integer overflows from the Viewport class. Other parts were left untouched, as this entire class of overflow issues gets fixed all at once, as soon as we replace COORD with til::coord (etc.). * [x] Closes #5271 * [x] I work here * [x] Tests added/passed * Call `ScrollConsoleScreenBufferW` with out of bounds coordinates * Doesn't crash ✅ (cherry picked from commit a4a6dfc)
This removes one source of potential integer overflows from the Viewport class. Other parts were left untouched, as this entire class of overflow issues gets fixed all at once, as soon as we replace COORD with til::coord (etc.). ## PR Checklist * [x] Closes #5271 * [x] I work here * [x] Tests added/passed ## Validation Steps Performed * Call `ScrollConsoleScreenBufferW` with out of bounds coordinates * Doesn't crash ✅ (cherry picked from commit a4a6dfc) Signed-off-by: Dustin Howett <duhowett@microsoft.com>
This removes one source of potential integer overflows from the Viewport class. Other parts were left untouched, as this entire class of overflow issues gets fixed all at once, as soon as we replace COORD with til::coord (etc.). ## PR Checklist * [x] Closes #5271 * [x] I work here * [x] Tests added/passed ## Validation Steps Performed * Call `ScrollConsoleScreenBufferW` with out of bounds coordinates * Doesn't crash ✅ (cherry picked from commit a4a6dfc)
|
🎉 Handy links: |
ghost
mentioned this pull request
|
🎉 Handy links: |
This removes one source of potential integer overflows from the Viewport class.
Other parts were left untouched, as this entire class of overflow issues gets
fixed all at once, as soon as we replace COORD with til::coord (etc.).
PR Checklist
Validation Steps Performed
ScrollConsoleScreenBufferWwith out of bounds coordinates