Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(config-cli): New config-cli package includes dump basemaps screenshots command line tool #2231

Merged
merged 8 commits into from
Jun 2, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions packages/config-cli/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Basemaps Config CLI

This package is to control the configuration in the LINZ basemaps product.

## Usage

Dump the screenshots from basemaps production

```bash
./screenshot.js screenshot
```

Dump the screenshots from different host and tag

```bash
./screenshot.js screenshot --host HOST --tag PR-TAG

```
36 changes: 36 additions & 0 deletions packages/config-cli/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"name": "@basemaps/config-cli",
"version": "6.27.0",
"repository": {
"type": "git",
"url": "https://github.com/linz/basemaps.git",
"directory": "packages/config"
},
"author": {
"name": "Land Information New Zealand",
"url": "https://linz.govt.nz",
"organization": true
},
"type": "module",
"engines": {
"node": "^12.20.0 || ^14.13.1 || >=16.0.0"
},
"license": "MIT",
"main": "./build/index.js",
"types": "./build/index.d.ts",
"scripts": {
"test": "ospec --globs 'build/**/*.test.js'"
},
"publishConfig": {
"access": "public"
},
"files": [
"build/"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing bin/

],
"dependencies": {
"@basemaps/geo": "^6.26.0",
"@basemaps/shared": "^6.27.0",
"@rushstack/ts-command-line": "^4.3.13",
"playwright": "^1.22.0"
}
}
3 changes: 3 additions & 0 deletions packages/config-cli/screenshot.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/usr/bin/env node
Error.stackTraceLimit = 100;
import './build/cli/screenshot/index.js';
16 changes: 16 additions & 0 deletions packages/config-cli/src/cli/screenshot/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/usr/bin/env node
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file should never be executed directly.

import { BaseCommandLine } from '@basemaps/shared/build/cli/base.js';
import 'source-map-support/register.js';
import { CommandScreenShot } from './screenshot.js';

export class ScreenshotCommandLine extends BaseCommandLine {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets call this BasemapsConfig as we will add more sub tools to it like the config import

bmc as the script name? or basemaps-config ?

constructor() {
super({
toolFilename: 'screenshot',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should the same name as the bin file

toolDescription: 'Dump screenshots from Basemaps',
});
this.addAction(new CommandScreenShot());
}
}

new ScreenshotCommandLine().run();
168 changes: 168 additions & 0 deletions packages/config-cli/src/cli/screenshot/screenshot.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
import { GoogleTms, Nztm2000QuadTms } from '@basemaps/geo';
import { Config, LogConfig, LogType } from '@basemaps/shared';
import { mkdir } from 'fs/promises';
import { Browser, chromium } from 'playwright';
import { CommandLineAction, CommandLineFlagParameter, CommandLineStringParameter } from '@rushstack/ts-command-line';

const TileTest = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after this pull request it would be great to store these as a JSON file.

{
name: 'health-3857-z5',
tileMatrix: GoogleTms,
location: { lat: -41.8899962, lng: 174.0492437, z: 5 },
tileSet: 'health',
style: undefined,
},
{
name: 'health-2193-z5',
tileMatrix: Nztm2000QuadTms,
location: { lat: -41.8899962, lng: 174.0492437, z: 1 },
tileSet: 'aerial',
},
{
name: 'topographic-3857-z5',
tileMatrix: GoogleTms,
location: { lat: -41.8899962, lng: 174.0492437, z: 5 },
tileSet: 'topographic',
style: 'topographic',
},
{
name: 'topolite-3857-z5',
tileMatrix: GoogleTms,
location: { lat: -41.8899962, lng: 174.0492437, z: 5 },
tileSet: 'topographic',
style: 'topolite',
},
{
name: 'topographic-3857-z14',
tileMatrix: GoogleTms,
location: { lat: -41.8899962, lng: 174.0492437, z: 14 },
tileSet: 'topographic',
style: 'topographic',
},
{
name: 'topolite-3857-z17',
tileMatrix: GoogleTms,
location: { lat: -43.8063936, lng: 172.9679876, z: 17 },
tileSet: 'topographic',
style: 'topolite',
},
];

export class CommandScreenShot extends CommandLineAction {
private host: CommandLineStringParameter;
private tag: CommandLineStringParameter;
private verbose?: CommandLineFlagParameter;

public constructor() {
super({
actionName: 'screenshot',
summary: 'dump screenshots of from LINZ Basemaps',
documentation: 'Dump screenshots with selected tile sets',
});
}

protected onDefineParameters(): void {
this.host = this.defineStringParameter({
argumentName: 'HOST',
parameterLongName: '--host',
description: 'Host to use',
defaultValue: 'basemaps.linz.govt.nz',
});

this.tag = this.defineStringParameter({
argumentName: 'TAG',
parameterShortName: '-t',
parameterLongName: '--tag',
description: 'PR tag(PR-number) or "production"',
defaultValue: 'production',
});

this.verbose = this.defineFlagParameter({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verbose is handled by being a part of BaseCommandLine

./bmc --verbose screenshot

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All fixed. Can you give another look?

parameterLongName: '--verbose',
description: 'Verbose logging',
required: false,
});
}

async onExecute(): Promise<void> {
const logger = LogConfig.get();
const verbose = this.verbose?.value ?? false;
if (verbose) logger.level = 'trace';

logger.info('Page:Launch');
const chrome = await chromium.launch();

try {
await this.takeScreenshots(chrome, logger);
} finally {
await chrome.close();
}
}

async takeScreenshots(chrome: Browser, logger: LogType): Promise<void> {
const host = this.host.value ?? this.host.defaultValue;
const tag = this.tag.value ?? this.tag.defaultValue;
if (host == null || tag == null) throw new Error('Missing host or tag.');

for (const test of TileTest) {
const page = await chrome.newPage();

const tileSetId = await this.getTileSetId(test.tileSet, tag);
const styleId = await this.getStyleId(test.style, tag);

const searchParam = new URLSearchParams();
searchParam.set('p', test.tileMatrix.identifier);
searchParam.set('i', tileSetId);
if (styleId) searchParam.set('s', styleId);

const loc = `@${test.location.lat},${test.location.lng},z${test.location.z}`;
const fileName = '.artifacts/visual-snapshots/' + host + '_' + test.name + '.png';

await mkdir(`.artifacts/visual-snapshots/`, { recursive: true });

const url = `https://${host}/?${searchParam.toString()}&debug=true&debug.screenshot=true#${loc}`;

logger.info({ url, expected: fileName }, 'Page:Load');

await page.goto(url);

try {
if (host.startsWith('dev')) {
await page.waitForSelector('div#map-loaded', { state: 'attached' });
await page.waitForTimeout(1000);
await page.waitForLoadState('networkidle');
} else {
throw new Error('Not supported on production yet');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is now in production we can take this logic out.

}
await page.screenshot({ path: fileName });
} catch (e) {
await page.screenshot({ path: fileName });
throw e;
}
logger.info({ url, expected: fileName }, 'Page:Load:Done');
await page.close();
}
}

async getTileSetId(tileSetId: string, tag: string): Promise<string> {
if (tag === 'production') return tileSetId;

const tileSetTagId = `${tileSetId}@${tag}`;
const dbId = Config.TileSet.id(tileSetTagId);
const tileSet = await Config.TileSet.get(dbId);

if (tileSet) return tileSetTagId;
return tileSetId;
}

async getStyleId(styleId: string | undefined, tag: string): Promise<string> {
if (styleId == null) return '';
if (tag === 'production') return styleId ?? '';

const styleIdTagId = `${styleId}@${tag}`;
const dbId = Config.Style.id(styleIdTagId);
const style = await Config.Style.get(dbId);
if (style) return styleIdTagId;
return styleId;
}
}
13 changes: 13 additions & 0 deletions packages/config-cli/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"target": "ES2020",
"lib": ["ES2020", "DOM"],
"module": "ES2020",
"moduleResolution": "node",
"rootDir": "./src",
"outDir": "./build"
},
"include": ["src/**/*"],
"references": [{ "path": "../__tests__" }]
}
1 change: 1 addition & 0 deletions tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
{ "path": "./packages/bathymetry" },
{ "path": "./packages/_infra" },
{ "path": "./packages/config" },
{ "path": "./packages/config-cli" },
{ "path": "./packages/shared" },
{ "path": "./packages/lambda-cog" },
{ "path": "./packages/lambda-tiler" },
Expand Down
12 changes: 12 additions & 0 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -5996,6 +5996,18 @@ pkg-dir@^4.2.0:
dependencies:
find-up "^4.0.0"

playwright-core@1.22.2:
version "1.22.2"
resolved "https://registry.yarnpkg.com/playwright-core/-/playwright-core-1.22.2.tgz#ed2963d79d71c2a18d5a6fd25b60b9f0a344661a"
integrity sha512-w/hc/Ld0RM4pmsNeE6aL/fPNWw8BWit2tg+TfqJ3+p59c6s3B6C8mXvXrIPmfQEobkcFDc+4KirNzOQ+uBSP1Q==

playwright@^1.22.0:
version "1.22.2"
resolved "https://registry.yarnpkg.com/playwright/-/playwright-1.22.2.tgz#353a7c29f89ca9600edc7a9a30aed790823c797d"
integrity sha512-hUTpg7LytIl3/O4t0AQJS1V6hWsaSY5uZ7w1oCC8r3a1AQN5d6otIdCkiB3cbzgQkcMaRxisinjMFMVqZkybdQ==
dependencies:
playwright-core "1.22.2"

pngjs@^4.0.1:
version "4.0.1"
resolved "https://registry.yarnpkg.com/pngjs/-/pngjs-4.0.1.tgz#f803869bb2fc1bfe1bf99aa4ec21c108117cfdbe"
Expand Down