Problems with mesuring slow sites

[denar90]

Hi. We got some issue in pwmetrics project, where users and me personally faced some problems using ligthouse API for getting site metrics. So I used @paulirish suggestion and run trace stuff (cloned repo and run node lighthouse-cli --save-assets --save-artifacts https://www.cnet.com/special-reports/jony-ive-talks-about-putting-the-apple-touch-on-the-macbook-pro/ --output=json > results.json). I'm attaching archive with results for kinda slow site.
Hope it will help you guys.

P.S. Don't hesitate to ask me for additional info or smth.

results.zip

Thanks.

[denar90]

One more thing. After debugging I found out that Lighthouse throws error: No usable firstMeaningfulPaint event found in trace twice.

[wardpeet]

I always get a memory exception when saving the trace. I'll have a look at your results :)

[tmf]

Hi @denar90 @wardpeet,
First: a big thank you for this awesome project!

I also looked into this issue and found that there could be several reasons for this (not finding a firstMeaningfulPaint event in the trace):

• fMP (firstMeaningfulPaint) is recorded before the fCP (firstContentfulPaint) - FCPFMP_TOLERANCE: in this case, I think, the metric cannot be determined...
• fMP is not recorded by the trace: this can happen when the actual fMP happens after both waitForLoadEvent and waitForNetworkIdle resolve or after MAX_WAIT_FOR_FULLY_LOADED (The trace is ended after the page has "finished" loading and all gatherer's pass's have been executed).

There could be several ways to solve the problem:

• pass in a driver flag pauseAfterLoad to wait more after both the load event and network idle event have been detected... This will (unreliably) give the driver the chance to record the fMP.
• extend recording the trace by waiting for the fMP event in either a gatherer's pass method or by adding a promise to wait for the trace event in the driver's _waitForFullyLoaded method . This should only happen when the the pass is configured to record the trace from the beginning.
• MAX_WAIT_FOR_FULLY_LOADED should maybe be overridable by a flag: with a 5x CPU throttling it is more probable for slow sites to run into this hardcoded timeout of 25 seconds.

Hope that helps :)

[wardpeet]

Thanks for the breakdown, maybe a better solution is when no fMP at all can be found we can just classify your site as slow XD

[denar90]

Do you have any idea how can it be interpreted? Maybe return firstMeaningfulPaint value equals to some constant which means that site is to slow... We really need some indicator for this.

---

But I also like an idea about making MAX_WAIT_FOR_FULLY_LOADED changeable (change it value by flag value)...

[wardpeet]

@denar90 would you find it useful that you can change it?
I would find it more useful to show that pageload took longer than 30s. Than you know that you have a slow site no? (might be wrong 😄 )

[denar90]

I think so. Taking as example site we are having issue with, and using webpagetest I got these results
https://www.webpagetest.org/result/170121_0G_WY7/
It's very slooooow. Load Time - 71.087s !!!

[denar90]

Looks like fixed 🎉
Thanks guys 💃