Improve caching of shading patterns. (bug 1721949) #13808

brendandahl · 2021-07-28T03:00:49Z

The PDF in bug 1721949 uses many unique pattern objects
that references the same shading many times. This caused
a new canvas pattern to be created and cached many times
driving up memory use.

To fix, I've changed the cache in the worker to key off the
shading object and instead send the shading and matrix
separately. While that worked well to fix the above bug,
there could be PDFs that use many shading that could
cause memory issues, so I've also added a LRU cache
on the main thread for canvas patterns. This should prevent
memory use from getting too high.

brendandahl · 2021-07-28T03:00:56Z

/botio test

pdfjsbot · 2021-07-28T03:00:57Z

From: Bot.io (Linux m4)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://54.67.70.0:8877/ff14baf4af73c25/output.txt

pdfjsbot · 2021-07-28T03:00:57Z

From: Bot.io (Windows)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://3.101.106.178:8877/09e4f6a7ed401ca/output.txt

pdfjsbot · 2021-07-28T03:27:56Z

From: Bot.io (Linux m4)

Failed

Full output at http://54.67.70.0:8877/ff14baf4af73c25/output.txt

Total script time: 26.98 mins

Font tests: Passed
Unit tests: Passed
Integration Tests: Passed
Regression tests: FAILED

  errors: 602
  different ref/snapshot: 8
  different first/second rendering: 1

Image differences available at: http://54.67.70.0:8877/ff14baf4af73c25/reftest-analyzer.html#web=eq.log

pdfjsbot · 2021-07-28T03:39:05Z

From: Bot.io (Windows)

Failed

Full output at http://3.101.106.178:8877/09e4f6a7ed401ca/output.txt

Total script time: 38.12 mins

Font tests: Passed
Unit tests: Passed
Integration Tests: FAILED
Regression tests: FAILED

  different ref/snapshot: 11
  different first/second rendering: 1

Image differences available at: http://3.101.106.178:8877/09e4f6a7ed401ca/reftest-analyzer.html#web=eq.log

brendandahl · 2021-07-28T15:39:38Z

/botio-linux test

pdfjsbot · 2021-07-28T15:39:39Z

From: Bot.io (Linux m4)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://54.67.70.0:8877/d009884031f0799/output.txt

pdfjsbot · 2021-07-28T16:07:19Z

From: Bot.io (Linux m4)

Failed

Full output at http://54.67.70.0:8877/d009884031f0799/output.txt

Total script time: 27.65 mins

Font tests: Passed
Unit tests: Passed
Integration Tests: Passed
Regression tests: FAILED

  errors: 584
  different ref/snapshot: 7
  different first/second rendering: 1

Image differences available at: http://54.67.70.0:8877/d009884031f0799/reftest-analyzer.html#web=eq.log

Snuffleupagus

r=me, with two small comments; thank you!

Snuffleupagus · 2021-07-28T16:56:16Z

src/display/canvas.js

+ * by last insertion.
+ */
+class LRUCache {
+  constructor(maxSize) {


Given that you have a this._maxSize <= 0 check further down, you probably want to set a numerical default value here (since (undefined <= 0) === false):

Suggested change

constructor(maxSize) {

constructor(maxSize = 0) {

Snuffleupagus · 2021-07-28T17:11:57Z

src/display/pattern_helper.js

+    if (!shadingFill && this.cachedCanvasPatterns.has(this)) {
+      pattern = this.cachedCanvasPatterns.get(this);
    } else {
      if (!shadingFill) {
        const tmpCanvas = owner.cachedCanvases.getCanvas(


Nit: Unless I'm completely misreading this code, it'd feel slightly easier to reason about the conditions with the following small change:

Suggested change

if (!shadingFill && this.cachedCanvasPatterns.has(this)) {

pattern = this.cachedCanvasPatterns.get(this);

} else {

if (!shadingFill) {

const tmpCanvas = owner.cachedCanvases.getCanvas(

if (!shadingFill) {

if (this.cachedCanvasPatterns.has(this)) {

pattern = this.cachedCanvasPatterns.get(this);

} else {

const tmpCanvas = owner.cachedCanvases.getCanvas(

The PDF in bug 1721949 uses many unique pattern objects that references the same shading many times. This caused a new canvas pattern to be created and cached many times driving up memory use. To fix, I've changed the cache in the worker to key off the shading object and instead send the shading and matrix separately. While that worked well to fix the above bug, there could be PDFs that use many shading that could cause memory issues, so I've also added a LRU cache on the main thread for canvas patterns. This should prevent memory use from getting too high.

brendandahl · 2021-07-28T17:30:00Z

/botio test

pdfjsbot · 2021-07-28T17:30:01Z

From: Bot.io (Linux m4)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://54.67.70.0:8877/b9636a4283dfbfb/output.txt

pdfjsbot · 2021-07-28T17:30:02Z

From: Bot.io (Windows)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://3.101.106.178:8877/dd1c5e96ce9c865/output.txt

pdfjsbot · 2021-07-28T18:03:31Z

From: Bot.io (Linux m4)

Failed

Full output at http://54.67.70.0:8877/b9636a4283dfbfb/output.txt

Total script time: 33.48 mins

Font tests: Passed
Unit tests: Passed
Integration Tests: Passed
Regression tests: FAILED

  different ref/snapshot: 10
  different first/second rendering: 1

Image differences available at: http://54.67.70.0:8877/b9636a4283dfbfb/reftest-analyzer.html#web=eq.log

pdfjsbot · 2021-07-28T18:08:50Z

From: Bot.io (Windows)

Failed

Full output at http://3.101.106.178:8877/dd1c5e96ce9c865/output.txt

Total script time: 38.80 mins

Font tests: Passed
Unit tests: Passed
Integration Tests: Passed
Regression tests: FAILED

  different ref/snapshot: 485
  different first/second rendering: 1

Image differences available at: http://3.101.106.178:8877/dd1c5e96ce9c865/reftest-analyzer.html#web=eq.log

Snuffleupagus · 2021-07-28T18:11:25Z

Hmm, it looks like makeref failed in a previous PR, so the "failures" in #13808 (comment) should be unrelated.

brendandahl · 2021-07-28T18:17:22Z

/botio makreref

pdfjsbot · 2021-07-28T18:17:23Z

From: Bot.io (Windows)

Invalid

Command not implemented: makreref.

pdfjsbot · 2021-07-28T18:17:23Z

From: Bot.io (Linux m4)

Invalid

Command not implemented: makreref.

brendandahl · 2021-07-28T18:17:33Z

/botio makeref

pdfjsbot · 2021-07-28T18:17:35Z

From: Bot.io (Linux m4)

Received

Command cmd_makeref from @brendandahl received. Current queue size: 1

Live output at: http://54.67.70.0:8877/f64f67868399425/output.txt

pdfjsbot · 2021-07-28T18:17:35Z

From: Bot.io (Windows)

Received

Command cmd_makeref from @brendandahl received. Current queue size: 2

Live output at: http://3.101.106.178:8877/b8ea7dc718f5d1d/output.txt

pdfjsbot · 2021-07-28T19:04:11Z

From: Bot.io (Linux m4)

Success

Full output at http://54.67.70.0:8877/f64f67868399425/output.txt

Total script time: 30.30 mins

Lint: Passed
Make references: Passed
Check references: Passed

pdfjsbot · 2021-07-28T19:25:10Z

From: Bot.io (Windows)

Success

Full output at http://3.101.106.178:8877/b8ea7dc718f5d1d/output.txt

Total script time: 36.42 mins

Lint: Passed
Make references: Passed
Check references: Passed

brendandahl requested a review from Snuffleupagus July 28, 2021 15:40

Snuffleupagus approved these changes Jul 28, 2021

View reviewed changes

brendandahl force-pushed the pattern-cache-v2 branch from 66efc87 to c836e1f Compare July 28, 2021 17:29

brendandahl merged commit 4ad5c5d into mozilla:master Jul 28, 2021

timvandermeij added performance shading-pattern labels Jul 30, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve caching of shading patterns. (bug 1721949) #13808

Improve caching of shading patterns. (bug 1721949) #13808

brendandahl commented Jul 28, 2021

brendandahl commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

brendandahl commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

Snuffleupagus left a comment

Snuffleupagus Jul 28, 2021

Snuffleupagus Jul 28, 2021

brendandahl commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

Snuffleupagus commented Jul 28, 2021

brendandahl commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

brendandahl commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

Improve caching of shading patterns. (bug 1721949) #13808

Improve caching of shading patterns. (bug 1721949) #13808

Conversation

brendandahl commented Jul 28, 2021

brendandahl commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

From: Bot.io (Linux m4)

Received

pdfjsbot commented Jul 28, 2021

From: Bot.io (Windows)

Received

pdfjsbot commented Jul 28, 2021

From: Bot.io (Linux m4)

Failed

pdfjsbot commented Jul 28, 2021

From: Bot.io (Windows)

Failed

brendandahl commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

From: Bot.io (Linux m4)

Received

pdfjsbot commented Jul 28, 2021

From: Bot.io (Linux m4)

Failed

Snuffleupagus left a comment

Choose a reason for hiding this comment

Snuffleupagus Jul 28, 2021

Choose a reason for hiding this comment

Snuffleupagus Jul 28, 2021

Choose a reason for hiding this comment

brendandahl commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

From: Bot.io (Linux m4)

Received

pdfjsbot commented Jul 28, 2021

From: Bot.io (Windows)

Received

pdfjsbot commented Jul 28, 2021

From: Bot.io (Linux m4)

Failed

pdfjsbot commented Jul 28, 2021

From: Bot.io (Windows)

Failed

Snuffleupagus commented Jul 28, 2021

brendandahl commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

From: Bot.io (Windows)

Invalid

pdfjsbot commented Jul 28, 2021

From: Bot.io (Linux m4)

Invalid

brendandahl commented Jul 28, 2021

pdfjsbot commented Jul 28, 2021

From: Bot.io (Linux m4)

Received

pdfjsbot commented Jul 28, 2021

From: Bot.io (Windows)

Received

pdfjsbot commented Jul 28, 2021

From: Bot.io (Linux m4)

Success

pdfjsbot commented Jul 28, 2021

From: Bot.io (Windows)

Success