docs: add results object explainer #3495

patrickhulce · 2017-10-06T17:54:04Z

brendankenny

nice work

brendankenny · 2017-10-07T00:35:21Z

readme.md

-* **[Calibre](https://calibreapp.com)** - Calibre is a web performance monitoring tool running Lighthouse continuously or on-demand via an API. Test using emulated devices and connection speeds from a number of geographical locations. Set budgets and improve performance with actionable guidelines. Note that Calibre is a paid product with a free 14-day trial. 
+* **[Calibre](https://calibreapp.com)** - Calibre is a web performance monitoring tool running Lighthouse continuously or on-demand via an API. Test using emulated devices and connection speeds from a number of geographical locations. Set budgets and improve performance with actionable guidelines. Note that Calibre is a paid product with a free 14-day trial.
+
+* **[HTTPArchive](http://httparchive.org/)** - HTTPArchive tracks how the web is built by crawling 500k pages with Web Page Test, including Lighthouse results, and stores the information in BigQuery where it is [publicly available](https://discuss.httparchive.org/t/quickstart-guide-to-exploring-the-http-archive/682).


sorry, made this conflict

brendankenny · 2017-10-07T00:43:15Z

docs/understanding-results.md

+| Name | Description |
+| - | - |
+| lighthouseVersion | The version of Lighthouse with which these results were generated. |
+| generatedTime | The ISO timestamp at which the results were generated. |


"The ISO-8601 timestamp of when the results were generated", maybe?

brendankenny · 2017-10-07T00:44:03Z

docs/understanding-results.md

+| userAgent | The user agent string of the version of Chrome that was used by Lighthouse. |
+| initialUrl | The URL that was supplied to Lighthouse and initially navigated to. |
+| url | The URL that Lighthouse ended up auditing after redirects were followed. |
+| score | The overall score `0-100`, a weighted average of all category scores. |


right now this is really the PWA category score, which we may have to maintain until LH 3.0 :)

are you sure?

lighthouse/lighthouse-core/report/v2/report-generator.js

Lines 109 to 110 in 320eb1f

const overallScore = ReportGeneratorV2.arithmeticMean(categories);

return {score: overallScore, categories};

are you sure?

technically correct that it's a weighted average of all category score, which is the best kind of correct, but it also happens that the weights are 0 except for the PWA category. So, maybe just add a sentence that by default it's currently weighted to only give the PWA score or something?

brendankenny · 2017-10-07T00:45:34Z

docs/understanding-results.md

+| name  | `string` | The string identifier of the audit in kebab case.  |
+| category | `string` | No longer used. *WARNING: Deprecated, will be removed in Lighthouse 3.0* |
+| description | `string` | The brief description of the audit's successful state. |
+| failureDescription | `string` | The brief description of the audit's failure state. |


this will always be description since failureDescription is copied over when the audit fails

Oh right, I was coming at it from the audit author perspective too which is confusing to mix here. I'll just remove?

brendankenny · 2017-10-07T00:48:10Z

docs/understanding-results.md

+| score | <code>boolean&#124;number</code> | The scored value determined by the audit as either boolean or a number `0-100`. If the audit is a boolean, the implication is `score ? 100 : 0`. |
+| rawValue | <code>boolean&#124;number</code> | The unscored value determined by the audit. Typically this will match the score if there's no additional information to impart. For performance audits, this value is typically a number indicating the metric value. |
+| displayValue | `string` | The string to display in the report alongside audit results. If empty, nothing additional is shown. This is typically used to explain additional information such as the number and nature of failing items. |
+| scoringMode | `string` | A string identifying how granular the score is meant to be indicating, i.e. is the audit pass/fail or are there shades of gray 0-100. *NOTE: This does not necessarily mean `typeof audit.score === audit.scoringMode`, an audit can have a score of 40 with a scoringMode of `"binary"` meant to indicate failure.* |


mention scoringMode can be binary or numeric?

brendankenny · 2017-10-07T00:49:21Z

docs/understanding-results.md

+| scoringMode | `string` | A string identifying how granular the score is meant to be indicating, i.e. is the audit pass/fail or are there shades of gray 0-100. *NOTE: This does not necessarily mean `typeof audit.score === audit.scoringMode`, an audit can have a score of 40 with a scoringMode of `"binary"` meant to indicate failure.* |
+| details | `Object` | Extra information found by the audit necessary for display. The structure of this object varies from audit to audit. The structure of this object is somewhat stable between minor version bumps as this object is used to render the HTML report.
+| extendedInfo | `Object` | Extra information found by the audit. The structure of this object varies from audit to audit and is generally for programmatic consumption and debugging, though there is typically overlap with `details`. *WARNING: The structure of this object is not stable and cannot be trusted to follow semver* |
+


what about debugString...and less importantly optimalValue, informative, and manual?

done, though skipping optimalValue since we don't know what it means or that we want it

brendankenny · 2017-10-07T00:53:36Z

docs/understanding-results.md

+| description | `string` | A brief description of the purpose of the category, supports markdown links. |
+| score | `string` | The overall score of the category, the weighted average of all its audits. |
+| weight | `string` | The weight of the category in the overall Lighthouse score. |
+| audits | `AuditEntry[]` | An array of all the audit results that comprise the category. Each entry contains a copy of the audit's result in the top-level [audits](#audits) object. |


should we warn that the result copy is going away?

s/comprise/compose again (or a different word if that sounds awkward)

brendankenny · 2017-10-07T01:03:17Z

docs/understanding-results.md

+<a name="report-categories"></a>
+## `reportCategories`
+
+An array containing the different categories, their scores, and the results of the audits that comprise them.


s/comprise/compose

got rid of it

brendankenny · 2017-10-07T01:05:53Z

docs/understanding-results.md

+| Name | Type | Description |
+| -- | -- | -- |
+| id | `string` | The string identifier of the category. |
+| score | `number` | The numeric score `0-100` of the audit, audits with a boolean score result are converted with `score ? 100 : 0`. |


paulirish

nice stuff.

some suggestions, but nothing i would block on.

paulirish · 2017-10-10T23:13:40Z

docs/understanding-results.md

+
+The results object contains all the audit information Lighthouse determined about the page. In fact, everything you see in the HTML report, even the screenshots, is a rendering of information contained in the results object. You might need to work directly with the results object if you use [Lighthouse programmatically](https://github.com/GoogleChrome/lighthouse/blob/master/docs/readme.md#using-programmatically), consume the JSON output of the [CLI](https://github.com/GoogleChrome/lighthouse#using-the-node-cli), explore [Lighthouse results in HTTPArchive](https://github.com/GoogleChrome/lighthouse#lighthouse-integrations), or work on the report generation code that reads the Lighthouse JSON and outputs HTML.
+
+## Top-level Object


can we call this

Lighthouse Result (LHR) object

a unique name makes it a bit easier to discuss

good call, I knew we had a name for it I was forgetting!

paulirish · 2017-10-10T23:15:09Z

docs/understanding-results.md

+| -- | -- | -- |
+| name  | `string` | The string identifier of the audit in kebab case.  |
+| category | `string` | No longer used. *WARNING: Deprecated, will be removed in Lighthouse 3.0* |
+| description | `string` | The brief description of the audit's successful state. |


missing failureDescription. see also https://github.com/GoogleChrome/lighthouse/blob/master/docs/architecture.md#auditreport-terminology

discussed offline, but not needed since failureDescription is just impl detail

paulirish · 2017-10-10T23:18:36Z

docs/understanding-results.md

+| description | `string` | The brief description of the audit's successful state. |
+| helpText | `string` | A more detailed description that describes why the audit is important and links to Lighthouse documentation on the audit, markdown links supported. |
+| debugString | <code>string&#124;undefined</code> | A string indicating some additional information to the user explaining an unusual circumstance or reason for failure. |
+| informative | `boolean` | Indicator used for display that the audit is intended to be informative only, not failable. |


Indicator used for display that the audit is intended to be informative only. It cannot be passed or failed.

paulirish · 2017-10-10T23:21:26Z

docs/understanding-results.md

+| displayValue | `string` | The string to display in the report alongside audit results. If empty, nothing additional is shown. This is typically used to explain additional information such as the number and nature of failing items. |
+| scoringMode | <code>"binary"&#124;"numeric"</code> | A string identifying how granular the score is meant to be indicating, i.e. is the audit pass/fail or are there shades of gray 0-100. *NOTE: This does not necessarily mean `typeof audit.score === audit.scoringMode`, an audit can have a score of 40 with a scoringMode of `"binary"` meant to indicate display should be failure.* |
+| details | `Object` | Extra information found by the audit necessary for display. The structure of this object varies from audit to audit. The structure of this object is somewhat stable between minor version bumps as this object is used to render the HTML report.
+| extendedInfo | `Object` | Extra information found by the audit. The structure of this object varies from audit to audit and is generally for programmatic consumption and debugging, though there is typically overlap with `details`. *WARNING: The structure of this object is not stable and cannot be trusted to follow semver* |


can you change the order of the properties here?

name
description
helpText
debugString
rawValue
displayValue
score
scoringMode
details
extendedInfo
manual
informative
category

paulirish · 2017-10-10T23:22:56Z

docs/understanding-results.md

+<a name="artifacts"></a>
+## `artifacts`
+
+An object containing gatherer results keyed by gatherer class name. The structure varies by artifact and is not stable. The values of artifacts are subject to change. This property is only available when consuming Lighthouse results programmatically as the artifacts contain trace data and can be quite large.


can be quite large (>50MB).

brendankenny

LGTM

brendankenny · 2017-10-11T01:23:51Z

a rebase will make the tests green :)

patrickhulce force-pushed the results_docs branch from 9fe7739 to f058053 Compare October 6, 2017 17:54

brendankenny requested changes Oct 7, 2017

View reviewed changes

paulirish approved these changes Oct 10, 2017

View reviewed changes

brendankenny approved these changes Oct 11, 2017

View reviewed changes

patrickhulce added 3 commits October 11, 2017 10:25

docs: add results object explainer

603b2bc

feedback

f6caecf

feedback

77bafe4

patrickhulce force-pushed the results_docs branch from 68a9f94 to 77bafe4 Compare October 11, 2017 17:26

add caveat for overall score

1430cdc

patrickhulce merged commit 6f9d593 into master Oct 11, 2017

patrickhulce deleted the results_docs branch October 11, 2017 21:53

patrickhulce mentioned this pull request Oct 12, 2017

Decide on & document stable vs. experimental properties of the LHR #2668

Closed

dependencies bot mentioned this pull request Dec 17, 2017

Update lighthouse in / from 2.5.0 to 2.7.0 chauncey-garrett/dotfiles#57

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: add results object explainer #3495

docs: add results object explainer #3495

patrickhulce commented Oct 6, 2017

brendankenny left a comment

brendankenny Oct 7, 2017

brendankenny Oct 7, 2017

patrickhulce Oct 9, 2017

brendankenny Oct 7, 2017

patrickhulce Oct 9, 2017

brendankenny Oct 11, 2017

brendankenny Oct 7, 2017

patrickhulce Oct 9, 2017

brendankenny Oct 7, 2017

patrickhulce Oct 9, 2017

brendankenny Oct 7, 2017

patrickhulce Oct 9, 2017 •

edited

Loading

brendankenny Oct 7, 2017

brendankenny Oct 7, 2017

patrickhulce Oct 9, 2017

brendankenny Oct 7, 2017

patrickhulce Oct 9, 2017

brendankenny Oct 7, 2017

patrickhulce Oct 9, 2017

paulirish left a comment

paulirish Oct 10, 2017

patrickhulce Oct 11, 2017

paulirish Oct 10, 2017

patrickhulce Oct 11, 2017

paulirish Oct 10, 2017

patrickhulce Oct 11, 2017

paulirish Oct 10, 2017

patrickhulce Oct 11, 2017

paulirish Oct 10, 2017

brendankenny left a comment

brendankenny commented Oct 11, 2017

	const overallScore = ReportGeneratorV2.arithmeticMean(categories);
	return {score: overallScore, categories};


		The results object contains all the audit information Lighthouse determined about the page. In fact, everything you see in the HTML report, even the screenshots, is a rendering of information contained in the results object. You might need to work directly with the results object if you use [Lighthouse programmatically](https://github.com/GoogleChrome/lighthouse/blob/master/docs/readme.md#using-programmatically), consume the JSON output of the [CLI](https://github.com/GoogleChrome/lighthouse#using-the-node-cli), explore [Lighthouse results in HTTPArchive](https://github.com/GoogleChrome/lighthouse#lighthouse-integrations), or work on the report generation code that reads the Lighthouse JSON and outputs HTML.

		## Top-level Object

docs: add results object explainer #3495

docs: add results object explainer #3495

Conversation

patrickhulce commented Oct 6, 2017

brendankenny left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

patrickhulce Oct 9, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulirish left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brendankenny left a comment

Choose a reason for hiding this comment

brendankenny commented Oct 11, 2017

patrickhulce Oct 9, 2017 •

edited

Loading