Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

misc(treemap): i18n #12441

Merged
merged 20 commits into from
May 14, 2021
Merged

misc(treemap): i18n #12441

merged 20 commits into from
May 14, 2021

Conversation

connorjclark
Copy link
Collaborator

@connorjclark connorjclark commented May 4, 2021

Strings added via rendererFormattedStrings. Considered an approach where the app would work out its own strings, but that would require 1) an extra round trip to get strings based off the LHR/useragent before we can render the app or 2) bundle strings for all languages into the bundle. Using rendererFormattedStrings also keeps things simpler.

@connorjclark connorjclark requested a review from a team as a code owner May 4, 2021 20:57
@connorjclark connorjclark requested review from patrickhulce and removed request for a team May 4, 2021 20:57
@google-cla google-cla bot added the cla: yes label May 4, 2021
build/build-treemap.js Outdated Show resolved Hide resolved
/** Text for an option in a dropdown menu, when selected the current view of the app is set to all scripts. */
treemapAllScripts: 'All Scripts',
/** Label for a column where the values are URLs, JS module names, or arbitrary identifiers. For simplicity, just 'name' is used. */
treemapName: 'Name',
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Name or URL / Module ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which column is this in treemap app exactly? it's been awhile since I've used it should we make it part of the vercel deployments on PRs?

Identifier perhaps?

@@ -377,7 +391,7 @@ class TreemapViewer {
if (!dataRow.unusedBytes) return '';

const percent = Math.floor(100 * dataRow.unusedBytes / dataRow.resourceBytes);
return `${percent}% bytes unused`;
return `${Util.i18n.formatNumber(percent)}% bytes unused`;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we have a good way to handle this; thoughts? It's just a tooltip...

should we do the "nearly correct" thing and just string concat this formatted number w/ a Util.i18n.bytesUnused ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oof, um what about filling with a prefilled value and replacing in the report?

e.g. ask translators to do {percentUnused}% bytes unused and for rendererFormattedStrings we prefill percentUnused to be 123456789 or some specific predetermined number?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but how would we do the value replacement at runtime? can we do something like

const data = replaceIcuMessages({string: str_(myStringWithPlaceholders, {value: 123}})
// get string from data ....

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah IDK, but either way I think we can punt for now :)

Maybe we start an issue with i18n frontend use cases? This happens in the report to some extent too we just chopped off the explanations to make them less helpful.

@@ -458,10 +478,11 @@ class TreemapViewer {
];

if (bytes !== undefined && total !== undefined) {
let str = `${TreemapUtil.formatBytes(bytes)} (${Math.round(bytes / total * 100)}%)`;
const percentStr = `${Util.i18n.formatNumber(Math.round(bytes / total * 100))}%`;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is concating % here OK?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, good question, but there's a lot of concat going on here with the partitionByStr: and then parts.join too so I'm not sure that's even our biggest issue. Since we're best effort on this already, seems fine file an issue to solve this report use case? 🤷

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for plain % (not concatenating to another string), there's new Intl.NumberFormat(locale, {style: 'percent'}).format(value). You have to be careful with report formatting stuff because Safari formatting support has really lagged the other browsers (and Node), but 'percent' has been supported for a while now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but then this value isn't just ##%, it's put into let str = `${TreemapUtil.formatBytes(bytes)} (${percentStr})`;, so agree with the 🤷 :)

@brendankenny
Copy link
Member

bundle strings for all languages into the bundle

Can you talk a bit about the downsides of this approach? 8 pretty short strings seems not so bad. It seems like the hardest part would be replicating locales.js without actually replicating it, then trimming down the locale files to just strings under lighthouse-treemap/.

rendererFormattedStrings, on the other hand, have to be baked into the proto (forever), are less forward compatible, etc.

@connorjclark
Copy link
Collaborator Author

Rough estimate: at 100 bytes per string per locale, and 8 strings, that is 50 * 800 = 40 kB. Currently the bundle is ~200 kB (on disk). Is that acceptable?

As for implementation, I suppose it wouldn't be too bad. Instead of

const i18n = new I18n(report.configSettings.locale, {
    // Set missing renderer strings to default (english) values.
    ...Util.UIStrings,
    ...report.i18n.rendererFormattedStrings,
  });

we just do

const i18n = new I18n(report.configSettings.locale, {
    ...require('./locales.js')['en'],
    ...require('./locales.js')[report.configSettings.locale], // need to use lookupLocale ???
  });

where locales.js is a file just like the main locales. lighthouse-treemap/app/src/locales/ could contain all the locale files, and we'd have to modify the collect-strings script to input strings from lighthouse-treemap/app/src and output them to lighthouse-treemap/app/src/locales/.

Copy link
Collaborator

@patrickhulce patrickhulce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big picture wise, it feels really weird to have another app's strings just hanging out in the LHR on the chance that someone has source maps and then wants to open them up in the treemap viewer.

  1. an extra round trip to get strings based off the LHR/useragent before we can render the app

This really doesn't sound that bad to me. This would be a super small payload, could be loaded speculatively first based on user's locale (possibly even better than LHR's locale?), and on a warm connection even on moderate 3G get these in 300ms.

I'm not expecting treemap viewer to be the best mobile experience anyway (I mean the LH mobile report experience is already awful, the metrics get cutoff so you can't even see the most important values)

Was there an underlying issue that this approach was more difficult to accomplish or more work? I wouldn't let this extra round trip stop separation of these strings for treemap :)

/** Text for an option in a dropdown menu, when selected the current view of the app is set to all scripts. */
treemapAllScripts: 'All Scripts',
/** Label for a column where the values are URLs, JS module names, or arbitrary identifiers. For simplicity, just 'name' is used. */
treemapName: 'Name',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which column is this in treemap app exactly? it's been awhile since I've used it should we make it part of the vercel deployments on PRs?

Identifier perhaps?

treemapName: 'Name',
/** Label for a value associated with how many bytes a URL/file is, on-disk. */
treemapResourceBytes: 'Resource Bytes',
/** Label for a value associated with how many bytes a URL/file is, over-network. */
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one doesn't seem right

Suggested change
/** Label for a value associated with how many bytes a URL/file is, over-network. */
/** Label for a value stating how many bytes of a URL/file were not used. */

maybe?

treemapAllScripts: 'All Scripts',
/** Label for a column where the values are URLs, JS module names, or arbitrary identifiers. For simplicity, just 'name' is used. */
treemapName: 'Name',
/** Label for a value associated with how many bytes a URL/file is, on-disk. */
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand the on disk distinction. Just saying it's the uncompressed size?

@@ -636,7 +636,7 @@ if (require.main === module) {

if (collisions > 0) {
console.log(`MEANING COLLISION: ${collisions} string(s) have the same content.`);
assert.equal(collisions, 30, `The number of duplicate strings have changed, update this assertion if that is expected, or reword strings. Collisions: ${collisionStrings.join('\n')}`);
assert.equal(collisions, 32, `The number of duplicate strings have changed, update this assertion if that is expected, or reword strings. Collisions: ${collisionStrings.join('\n')}`);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name, and....?

This comment was marked as spam.

@@ -377,7 +391,7 @@ class TreemapViewer {
if (!dataRow.unusedBytes) return '';

const percent = Math.floor(100 * dataRow.unusedBytes / dataRow.resourceBytes);
return `${percent}% bytes unused`;
return `${Util.i18n.formatNumber(percent)}% bytes unused`;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oof, um what about filling with a prefilled value and replacing in the report?

e.g. ask translators to do {percentUnused}% bytes unused and for rendererFormattedStrings we prefill percentUnused to be 123456789 or some specific predetermined number?

@@ -458,10 +478,11 @@ class TreemapViewer {
];

if (bytes !== undefined && total !== undefined) {
let str = `${TreemapUtil.formatBytes(bytes)} (${Math.round(bytes / total * 100)}%)`;
const percentStr = `${Util.i18n.formatNumber(Math.round(bytes / total * 100))}%`;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, good question, but there's a lot of concat going on here with the partitionByStr: and then parts.join too so I'm not sure that's even our biggest issue. Since we're best effort on this already, seems fine file an issue to solve this report use case? 🤷

lighthouse-treemap/app/src/main.js Show resolved Hide resolved
lighthouse-treemap/app/src/util.js Outdated Show resolved Hide resolved
...report.i18n.rendererFormattedStrings,
});
Util.i18n = i18n;
Util.reportJson = report;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this? I thought this was just for the performance category renderer

build/build-treemap.js Outdated Show resolved Hide resolved
@google-cla
Copy link

google-cla bot commented May 4, 2021

☹️ Sorry, but only Googlers may change the label cla: yes.

@@ -636,7 +636,7 @@ if (require.main === module) {

if (collisions > 0) {
console.log(`MEANING COLLISION: ${collisions} string(s) have the same content.`);
assert.equal(collisions, 30, `The number of duplicate strings have changed, update this assertion if that is expected, or reword strings. Collisions: ${collisionStrings.join('\n')}`);
assert.equal(collisions, 32, `The number of duplicate strings have changed, update this assertion if that is expected, or reword strings. Collisions: ${collisionStrings.join('\n')}`);

This comment was marked as spam.

Copy link

@mspaansen mspaansen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ik probeer het. Maar kost me veel tijd

@connorjclark
Copy link
Collaborator Author

waiting for feedback on #12441 (comment)

@patrickhulce I believe I commented this at the exact time you submitted your review, so likely you missed it.

Copy link
Collaborator

@patrickhulce patrickhulce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just the minor concern about the collision check undefined strangeness

Comment on lines 11 to 24
function buildLocales() {
const locales = require('../lighthouse-core/lib/i18n/locales.js');
const clonedLocales = JSON.parse(JSON.stringify(locales));

for (const lhlMessages of Object.values(clonedLocales)) {
for (const key of Object.keys(lhlMessages)) {
if (!key.startsWith('lighthouse-treemap')) {
delete lhlMessages[key];
}
}
}

return 'const locales =' + JSON.stringify(clonedLocales, null, 2) + ';';
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just for fun since we can use fromEntries, what are your thoughts on the readability of the below compared to the delete?

Suggested change
function buildLocales() {
const locales = require('../lighthouse-core/lib/i18n/locales.js');
const clonedLocales = JSON.parse(JSON.stringify(locales));
for (const lhlMessages of Object.values(clonedLocales)) {
for (const key of Object.keys(lhlMessages)) {
if (!key.startsWith('lighthouse-treemap')) {
delete lhlMessages[key];
}
}
}
return 'const locales =' + JSON.stringify(clonedLocales, null, 2) + ';';
}
function buildLocales() {
const locales = Object.entries(require('../lighthouse-core/lib/i18n/locales.js'));
const treemapLocales = locales.map(([locale, messageMap]) => {
const lhlMessages = Object.entries(messageMap);
const treemapMessages = lhlMessages.filter(([key]) => key.startsWith('lighthouse-treemap'));
return [locale, Object.fromEntries(treemapMessages)];
});
return 'const locales =' + JSON.stringify(Object.fromEntries(treemapLocales), null, 2) + ';';
}

(not serious review, just curious) :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because it's nested, i prefer the current approach

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha, agree with @connorjclark on this one :)

}
}

return 'const locales =' + JSON.stringify(clonedLocales, null, 2) + ';';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with something this specially injected, WDYT about the name being a little more in your face?

BUILD_INJECTED_LOCALES/__buildInjectedLocales__/sOmEtHiNgElSe?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the cloak and dagger is necessary here, since we control the code that runs here (and it's not a library) :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm mostly interested in improving the experience for someone reading locales referenced without a definition and wondering where in the world it comes from. If you tried to grep locales in our codebase you'd get like 160 results. With localesInjectedAtBuildTime you'd get 1 :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comment then?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self-describing variable names >> comments :)

but yes a comment would somewhat address my concern too.

@@ -121,7 +122,10 @@ class TreemapViewer {
}

for (const [group, depthOneNodes] of Object.entries(this.depthOneNodesByGroup)) {
makeOption({type: 'group', value: group}, `All ${group}`);
const allLabel = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the purpose of this object? are there about to be many types here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe, i don't know, I'm being careful here.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there something dangerous that could happen by using the string directly? ;)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have just one group now (scripts).

there's also a small part of me thinking someone will send us their own treemap data, and their own groups. unlikely, but....

anyhow, the object will be needed regardless as soon as we have a second group (however that turns out–I think resource summary will be a group? idk yet)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aight, sg

const i18n = new I18n(options.lhr.configSettings.locale, {
// Set missing renderer strings to default (english) values.
...TreemapUtil.UIStrings,
...getStrings(locales[options.lhr.configSettings.locale]),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this handle all that fallback business? I can't remember off the top of my head if the configSettings locale is normalized to always match our defined locale (e.g. for user-provided de-CH which matches our de key, will this work or miss?)

EDIT: seems like we should be fine

settingsWithFlags.locale = locale;

@@ -8,6 +8,21 @@
const fs = require('fs');
const GhPagesApp = require('./gh-pages-app.js');

function buildLocales() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a jsdoc comment on this function would be appreciated so you don't have to know the structure of locales.js and the files it requires and what the significance of key.startsWith('lighthouse-treemap') is to know what this is doing to the locale files :)

Comment on lines 11 to 24
function buildLocales() {
const locales = require('../lighthouse-core/lib/i18n/locales.js');
const clonedLocales = JSON.parse(JSON.stringify(locales));

for (const lhlMessages of Object.values(clonedLocales)) {
for (const key of Object.keys(lhlMessages)) {
if (!key.startsWith('lighthouse-treemap')) {
delete lhlMessages[key];
}
}
}

return 'const locales =' + JSON.stringify(clonedLocales, null, 2) + ';';
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha, agree with @connorjclark on this one :)


/**
* @template T
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameterizing on strings when none of the I18n object uses strings except the getter for strings works for this PR but makes me think it really shouldn't be in this object at all and should just be a global (or on Util directly), but we can save that for a future report refactor PR :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving to Util/TreemapUtil SGTM, and so does worrying about it in a future refactor ;)

assert.ok(val in Util.UIStrings, `Invalid data-i18n value of: "${val}" not found.`);
}

// Do the same for the strings in treemap app.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this move to a treemap test since report-ui-features isn't involved?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, FWIW, if it's just the single string, could just set it manually and avoid the loop here and in main.js

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's just so much BLEGH to do re: jsdom setup... altho I guess it could go into the puppeteer page?

if it's just the single string,

I think there will be more later (if we do the settings cog).

Copy link
Member

@brendankenny brendankenny May 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's just so much BLEGH to do re: jsdom setup

normally true but I think it's all self contained in this case. Free test for you:

const assert = require('assert').strict;
const fs = require('fs');
const jsdom = require('jsdom');

describe('data-i18n', () => {
  it('should have only valid data-i18n values in treemap html', () => {
    const TreemapUtil = require('../../../../../lighthouse-treemap/app/src/util.js');
    const TREEMAP_INDEX = fs.readFileSync(__dirname +
      '/../../../../../lighthouse-treemap/app/index.html', 'utf8');
    const dom = new jsdom.JSDOM(TREEMAP_INDEX);
    for (const node of dom.window.document.querySelectorAll('[data-i18n]')) {
      const val = node.getAttribute('data-i18n');
      assert.ok(val in TreemapUtil.UIStrings, `Invalid data-i18n value of: "${val}" not found.`);
    }
  });
});

though the paths will have to be updated :)


TreemapUtil.UIStrings = {
/** Label for a button that alternates between showing or hiding a table. */
toggleTable: 'Toggle Table',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the main report we tend to be more self documenting with these, e.g.toggleTableButtonLabel for this. It seems like a good approach generally?

I guess I'm less concerned about 'toggleTable' and more about things like 'all'.

lighthouse-treemap/app/src/util.js Outdated Show resolved Hide resolved
lighthouse-treemap/app/src/util.js Outdated Show resolved Hide resolved
Comment on lines 213 to 214
/** Label for a value associated with how many bytes a URL/file is, over-network. */
unusedBytes: 'Unused Bytes',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did this description and string diverge?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait yeah, I had these exact same comments already, did those get those in the move?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The strings moved from Util to TreemapUtil, so github prob hid ur comments

havent fixed the original comments yet :P

lighthouse-treemap/app/src/util.js Outdated Show resolved Hide resolved
const [filename, varName] = icuMessageId.split(' | ');
if (!filename.endsWith('util.js')) throw new Error(`Unexpected message: ${icuMessageId}`);

const key = /** @type {keyof LH.I18NRendererStrings} */ (varName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

definitely not LH.I18NRendererStrings :)

Can chopping off the path be moved to the build step? If we don't use them anyways, it would certainly cut down that 40KiB to something much smaller :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea

Co-authored-by: Brendan Kenny <bckenny@gmail.com>
Copy link
Collaborator

@patrickhulce patrickhulce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still missing the straggler OG comments, but I'll approve :)

for (const lhlMessages of Object.values(clonedLocales)) {
for (const icuMessageId of Object.keys(lhlMessages)) {
const lhlMessage = lhlMessages[icuMessageId];
delete lhlMessages[icuMessageId];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, c'mon y'all now it's really not a clone 😆 still no construction love?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol ok i surrender

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also renamed this to strings since it is different from the real locales

tableColumnName: 'Name',
/** Label for column giving the size of a file in bytes. */
resourceBytesLabel: 'Resource Bytes',
/** Label for a value associated with how many bytes a URL/file is, over-network. */
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bump

Copy link
Collaborator

@patrickhulce patrickhulce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants