Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core(test): add user flows to generate some fixtures #15005

Merged
merged 12 commits into from
May 2, 2023
Merged

core(test): add user flows to generate some fixtures #15005

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
4673e15
core(test): add user flows to generate some fixtures
connorjclark Apr 20, 2023
559a0c3
dont use lighthouse pkg
connorjclark Apr 20, 2023
d608b26
change lint thing
connorjclark Apr 25, 2023
d41d1cc
fix interactive test
connorjclark Apr 26, 2023
4e5f3bc
fix snapshot
connorjclark Apr 26, 2023
2b75ae0
fix animation fixture
connorjclark Apr 27, 2023
a4706e0
revert test changes
connorjclark Apr 27, 2023
40f90bb
mv
connorjclark Apr 27, 2023
b80bbe5
oops
connorjclark Apr 27, 2023
d0d244c
move to new folder structure
connorjclark May 1, 2023
044fcef
bad c/p
connorjclark May 1, 2023
9838462
rename
connorjclark May 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cli/test/fixtures/perf/animations.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
width: 100px;
height: 100px;
background-color: green;
animation: gamma 4s,
animation: gamma 4s;
}

@keyframes alpha {
Expand Down
51 changes: 37 additions & 14 deletions core/test/audits/mainthread-work-breakdown-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ import {readJson} from '../test-utils.js';

const acceptableTrace = readJson('../fixtures/traces/progressive-app-m60.json', import.meta);
const siteWithRedirectTrace = readJson('../fixtures/traces/site-with-redirect.json', import.meta);
const loadTrace = readJson('../fixtures/traces/load.json', import.meta);
const loadTraceOld = readJson('../fixtures/traces/load.json', import.meta);
const loadTrace = readJson('../fixtures/traces/animation.json', import.meta);
const errorTrace = readJson('../fixtures/traces/no_fmp_event.json', import.meta);

const options = PageExecutionTimings.defaultOptions;
Expand All @@ -25,15 +26,6 @@ const acceptableTraceExpectations = {
garbageCollection: 48,
other: 663,
};
const siteWithRedirectTraceExpectations = {
parseHTML: 84,
styleLayout: 275,
paintCompositeRender: 12,
scriptEvaluation: 145,
scriptParseCompile: 38,
garbageCollection: 46,
other: 184,
};
const loadTraceExpectations = {
parseHTML: 25,
styleLayout: 131,
Expand Down Expand Up @@ -85,14 +77,25 @@ describe('Performance: page execution timings audit', () => {
const artifacts = {traces: {defaultPass: siteWithRedirectTrace}};

const output = await PageExecutionTimings.audit(artifacts, {options, computedCache: new Map()});
assert.deepStrictEqual(keyOutput(output), siteWithRedirectTraceExpectations);
assert.equal(Math.round(output.numericValue), 784);
expect(keyOutput(output)).toMatchInlineSnapshot(`
Object {
"garbageCollection": 14,
"other": 188,
"paintCompositeRender": 11,
"parseHTML": 52,
"scriptEvaluation": 577,
"scriptParseCompile": 67,
"styleLayout": 70,
}
`);
expect(Math.round(output.numericValue)).toMatchInlineSnapshot(`979`);
assert.equal(output.details.items.length, 7);
assert.equal(output.score, 1);
});

it('should compute the correct values for the load trace', async () => {
const artifacts = {traces: {defaultPass: {traceEvents: loadTrace}}};
it('should compute the correct values for the load trace (legacy)', async () => {
assert(loadTraceOld.find(e => e.name === 'TracingStartedInPage'));
const artifacts = {traces: {defaultPass: {traceEvents: loadTraceOld}}};

const output = await PageExecutionTimings.audit(artifacts, {options, computedCache: new Map()});
assert.deepStrictEqual(keyOutput(output), loadTraceExpectations);
Expand All @@ -101,6 +104,26 @@ describe('Performance: page execution timings audit', () => {
assert.equal(output.score, 1);
});

it('should compute the correct values for the load trace', async () => {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really any modern chrome trace will do here, which is why I didn't bother making a user flow for a basic page. instead just used animations here.

assert(loadTrace.traceEvents.find(e => e.name === 'TracingStartedInBrowser'));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not put this assertion in the trace verify function?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because that's pretty far removed from the one place that cares (this test)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, are we really worried about this event being missing in any new traces? The check kinda makes sense for the old trace but if this one is being continually refreshed then I'm not sure we need it.

Copy link
Collaborator Author

@connorjclark connorjclark Apr 27, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This trace will not be refreshed. We could remove the logic for the old event but only when all our test fixtures have been updated.

const artifacts = {traces: {defaultPass: loadTrace}};

const output = await PageExecutionTimings.audit(artifacts, {options, computedCache: new Map()});
expect(keyOutput(output)).toMatchInlineSnapshot(`
Object {
"other": 59,
"paintCompositeRender": 48,
"parseHTML": 3,
"scriptEvaluation": 11,
"scriptParseCompile": 1,
"styleLayout": 103,
}
`);
expect(Math.round(output.numericValue)).toMatchInlineSnapshot(`224`);
assert.equal(output.details.items.length, 6);
assert.equal(output.score, 1);
});

it('should get no data when no events are present', () => {
const artifacts = {traces: {defaultPass: errorTrace}};

Expand Down
5 changes: 2 additions & 3 deletions core/test/audits/metrics/interactive-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,8 @@ describe('Performance: interactive audit', () => {

const context = getFakeContext({formFactor: 'mobile', throttlingMethod: 'provided'});
return Interactive.audit(artifacts, context).then(output => {
assert.equal(output.score, 0.97);
assert.equal(Math.round(output.numericValue), 2712);
expect(output.displayValue).toBeDisplayString('2.7\xa0s');
assert.equal(output.score, 0.98);
expect(output.numericValue).toMatchInlineSnapshot(`2381.989`);
});
});
});
210 changes: 114 additions & 96 deletions core/test/audits/third-party-facades-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,106 +267,124 @@ describe('Third party facades audit', () => {
expect(results.displayValue).toBeDisplayString('2 facade alternatives available');
expect(results.details.items[0].product).toBeDisplayString('YouTube Embedded Player (Video)');
expect(results.details.items[1].product).toBeDisplayString('Vimeo Embedded Player (Video)');
expect(results.details.items).toMatchObject(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also just delete this?

[
{
entity: 'YouTube',
transferSize: 651350,
blockingTime: 0,
subItems: {
items: [
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 459603,
url: 'https://www.youtube.com/s/player/e0d83c30/player_ias.vflset/en_US/base.js',
},
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 66273,
url: 'https://i.ytimg.com/vi/tgbNymZ7vqY/maxresdefault.jpg',
},
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 50213,
url: 'https://www.youtube.com/s/player/e0d83c30/www-embed-player.vflset/www-embed-player.js',
},
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 46813,
url: 'https://www.youtube.com/s/player/e0d83c30/www-player.css',
},
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 11477,
url: 'https://www.youtube.com/s/player/e0d83c30/player_ias.vflset/en_US/embed.js',
},
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 16971,
url: {
formattedDefault: 'Other resources',
},
},
],
type: 'subitems',
expect(results.details.items).toMatchInlineSnapshot(`
Array [
Object {
"blockingTime": 0,
"entity": "YouTube",
"product": Object {
"formattedDefault": "YouTube Embedded Player (Video)",
"i18nId": "core/audits/third-party-facades.js | categoryVideo",
"values": Object {
"productName": "YouTube Embedded Player",
},
},
"subItems": Object {
"items": Array [
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 459603,
"url": "https://www.youtube.com/s/player/e0d83c30/player_ias.vflset/en_US/base.js",
},
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 66273,
"url": "https://i.ytimg.com/vi/tgbNymZ7vqY/maxresdefault.jpg",
},
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 50213,
"url": "https://www.youtube.com/s/player/e0d83c30/www-embed-player.vflset/www-embed-player.js",
},
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 46813,
"url": "https://www.youtube.com/s/player/e0d83c30/www-player.css",
},
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 11477,
"url": "https://www.youtube.com/s/player/e0d83c30/player_ias.vflset/en_US/embed.js",
},
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 16971,
"url": Object {
"formattedDefault": "Other resources",
"i18nId": "core/lib/i18n/i18n.js | otherResourcesLabel",
"values": undefined,
},
},
{
entity: 'Vimeo',
transferSize: 184495,
blockingTime: 0,
subItems: {
items: [
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 145772,
url: 'https://f.vimeocdn.com/p/3.22.3/js/player.js',
},
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 17633,
url: 'https://f.vimeocdn.com/p/3.22.3/css/player.css',
},
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 9313,
url: 'https://i.vimeocdn.com/video/784397921.webp?mw=1200&mh=675&q=70',
},
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 8300,
url: 'https://player.vimeo.com/video/336812660',
},
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 1474,
url: 'https://f.vimeocdn.com/js_opt/modules/utils/vuid.min.js',
},
{
blockingTime: 0,
mainThreadTime: 0,
transferSize: 2003,
url: {
formattedDefault: 'Other resources',
},
},
],
type: 'subitems',
],
"type": "subitems",
},
"transferSize": 651350,
},
Object {
"blockingTime": 0,
"entity": "Vimeo",
"product": Object {
"formattedDefault": "Vimeo Embedded Player (Video)",
"i18nId": "core/audits/third-party-facades.js | categoryVideo",
"values": Object {
"productName": "Vimeo Embedded Player",
},
},
"subItems": Object {
"items": Array [
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 145772,
"url": "https://f.vimeocdn.com/p/3.22.3/js/player.js",
},
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 17633,
"url": "https://f.vimeocdn.com/p/3.22.3/css/player.css",
},
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 9313,
"url": "https://i.vimeocdn.com/video/784397921.webp?mw=1200&mh=675&q=70",
},
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 8300,
"url": "https://player.vimeo.com/video/336812660",
},
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 1474,
"url": "https://f.vimeocdn.com/js_opt/modules/utils/vuid.min.js",
},
Object {
"blockingTime": 0,
"mainThreadTime": 0,
"transferSize": 2003,
"url": Object {
"formattedDefault": "Other resources",
"i18nId": "core/lib/i18n/i18n.js | otherResourcesLabel",
"values": undefined,
},
},
]
);
],
"type": "subitems",
},
"transferSize": 184495,
},
]
`);
});

describe('.condenseItems', () => {
Expand Down
41 changes: 15 additions & 26 deletions core/test/audits/user-timing-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,57 +9,46 @@ import assert from 'assert/strict';
import UserTimingsAudit from '../../audits/user-timings.js';
import {readJson} from '../test-utils.js';

const traceEvents = readJson('../fixtures/traces/trace-user-timings.json', import.meta);
const trace = readJson('../fixtures/traces/trace-user-timings.json', import.meta);

function generateArtifactsWithTrace(trace) {
return {
traces: {
defaultPass: {traceEvents: Array.isArray(trace) ? trace : trace.traceEvents},
defaultPass: trace,
},
};
}
describe('Performance: user-timings audit', () => {
it('evaluates valid input correctly', () => {
const artifacts = generateArtifactsWithTrace(traceEvents);
const artifacts = generateArtifactsWithTrace(trace);
return UserTimingsAudit.audit(artifacts, {computedCache: new Map()}).then(auditResult => {
const excludedUTs = auditResult.details.items.filter(timing => {
return UserTimingsAudit.excludedPrefixes.some(prefix => timing.name.startsWith(prefix));
});
assert.equal(excludedUTs.length, 0, 'excluded usertimings included in results');

assert.equal(auditResult.score, 0);
expect(auditResult.displayValue).toBeDisplayString('2 user timings');
expect(auditResult.displayValue).toBeDisplayString('4 user timings');

assert.equal(auditResult.details.items[0].name, 'measure_test');
assert.equal(auditResult.details.items[0].name, 'fetch-end');
assert.equal(auditResult.details.items[0].timingType, 'Measure');
assert.equal(auditResult.details.items[0].startTime, 0.002);
assert.equal(auditResult.details.items[0].duration, 1000.965);
expect(auditResult.details.items[0].startTime).toMatchInlineSnapshot(`1597.565`);
expect(auditResult.details.items[0].duration).toMatchInlineSnapshot(`6.7`);

assert.equal(auditResult.details.items[1].name, 'mark_test');
assert.equal(auditResult.details.items[1].name, 'start');
assert.equal(auditResult.details.items[1].timingType, 'Mark');
assert.equal(auditResult.details.items[1].startTime, 1000.954);
expect(auditResult.details.items[1].startTime).toMatchInlineSnapshot(`688.369`);
assert.equal(auditResult.details.items[1].duration, undefined);

assert.equal(auditResult.details.items[2].name, 'fetch-start');
assert.equal(auditResult.details.items[2].timingType, 'Mark');
expect(auditResult.details.items[2].startTime).toMatchInlineSnapshot(`1604.267`);
assert.equal(auditResult.details.items[2].duration, undefined);
});
});

it('doesn\'t throw when user_timing events have a colon', () => {
const extraTraceEvents = traceEvents.concat([
{
'pid': 41904,
'tid': 1295,
'ts': 1676836141,
'ph': 'R',
'id': 'fake-event',
'cat': 'blink.user_timing',
'name': 'Zone:ZonePromise',
'dur': 64,
'tdur': 61,
'tts': 881373,
'args': {},
},
]);

const artifacts = generateArtifactsWithTrace(extraTraceEvents);
const artifacts = generateArtifactsWithTrace(trace);
return UserTimingsAudit.audit(artifacts, {computedCache: new Map()}).then(result => {
const fakeEvt = result.details.items.find(item => item.name === 'Zone:ZonePromise');
assert.ok(fakeEvt, 'failed to find user timing item with colon');
Expand Down
Loading