Skip to content

Commit

Permalink
Fix overflow in Viewport::FromDimensions (#12669)
Browse files Browse the repository at this point in the history
This removes one source of potential integer overflows from the Viewport class.
Other parts were left untouched, as this entire class of overflow issues gets
 fixed all at once, as soon as we replace COORD with til::coord (etc.).

## PR Checklist
* [x] Closes #5271
* [x] I work here
* [x] Tests added/passed

## Validation Steps Performed
* Call `ScrollConsoleScreenBufferW` with out of bounds coordinates
* Doesn't crash ✅

(cherry picked from commit a4a6dfc)
  • Loading branch information
lhecker authored and DHowett committed Mar 23, 2022
1 parent 6cd3e6a commit 9c1fe1f
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 4 deletions.
3 changes: 1 addition & 2 deletions src/renderer/base/renderer.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -677,8 +677,7 @@ void Renderer::_PaintBufferOutput(_In_ IRenderEngine* const pEngine)

for (const auto& dirtyRect : dirtyAreas)
{
// Shortcut: don't bother redrawing if the width is 0.
if (dirtyRect.left == dirtyRect.right)
if (!dirtyRect)
{
continue;
}
Expand Down
14 changes: 12 additions & 2 deletions src/types/viewport.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,12 @@ Viewport Viewport::FromDimensions(const COORD origin,
const short width,
const short height) noexcept
{
return Viewport::FromExclusive({ origin.X, origin.Y, origin.X + width, origin.Y + height });
return Viewport::FromInclusive({
origin.X,
origin.Y,
base::saturated_cast<short>(origin.X + width - 1),
base::saturated_cast<short>(origin.Y + height - 1),
});
}

// Function Description:
Expand All @@ -60,7 +65,12 @@ Viewport Viewport::FromDimensions(const COORD origin,
Viewport Viewport::FromDimensions(const COORD origin,
const COORD dimensions) noexcept
{
return Viewport::FromExclusive({ origin.X, origin.Y, origin.X + dimensions.X, origin.Y + dimensions.Y });
return Viewport::FromInclusive({
origin.X,
origin.Y,
base::saturated_cast<short>(origin.X + dimensions.X - 1),
base::saturated_cast<short>(origin.Y + dimensions.Y - 1),
});
}

// Function Description:
Expand Down

0 comments on commit 9c1fe1f

Please sign in to comment.