-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(config-cli): New config-cli package includes dump basemaps screenshots command line tool #2231
Merged
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
0161017
New config-cli package include dump basemaps screenshots
Wentao-Kuang b802e83
Convert the test tiles into json file.
Wentao-Kuang 502fe0a
Remove unused imports
Wentao-Kuang e0ccde5
Move bmc into bin file
Wentao-Kuang a6e3544
Remove verbose as it is handdled by Basecommandline
Wentao-Kuang 653fb8d
Using Zod to parse test tiles
Wentao-Kuang b9bbdf9
Update yarn.locl
Wentao-Kuang 3f08b85
Some minor changes.
Wentao-Kuang File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
# Basemaps Config CLI | ||
|
||
This package is to control the configuration in the LINZ basemaps product. | ||
|
||
## Usage -- Screenshots | ||
|
||
Dump the screenshots from basemaps production | ||
|
||
```bash | ||
./bin/bmc.js screenshot | ||
``` | ||
|
||
Dump the screenshots from different host and tag | ||
|
||
```bash | ||
./bin/bmc.js screenshot --host HOST --tag PR-TAG | ||
|
||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
#!/usr/bin/env node | ||
Error.stackTraceLimit = 100; | ||
import { BasemapsConfig } from '../build/cli/screenshot/index.js'; | ||
|
||
new BasemapsConfig().run(); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
{ | ||
"name": "@basemaps/config-cli", | ||
"version": "6.27.0", | ||
"repository": { | ||
"type": "git", | ||
"url": "https://github.com/linz/basemaps.git", | ||
"directory": "packages/config" | ||
}, | ||
"author": { | ||
"name": "Land Information New Zealand", | ||
"url": "https://linz.govt.nz", | ||
"organization": true | ||
}, | ||
"type": "module", | ||
"engines": { | ||
"node": "^12.20.0 || ^14.13.1 || >=16.0.0" | ||
}, | ||
"license": "MIT", | ||
"main": "./build/index.js", | ||
"types": "./build/index.d.ts", | ||
"bin": { | ||
"bmc": "./bin/bmc.js" | ||
}, | ||
"scripts": { | ||
"test": "ospec --globs 'build/**/*.test.js'" | ||
}, | ||
"publishConfig": { | ||
"access": "public" | ||
}, | ||
"files": [ | ||
"build/", | ||
"bin/" | ||
], | ||
"dependencies": { | ||
"@basemaps/config-cli": "^6.27.0", | ||
"@basemaps/geo": "^6.26.0", | ||
"@basemaps/shared": "^6.27.0", | ||
"@rushstack/ts-command-line": "^4.3.13", | ||
"playwright": "^1.22.0", | ||
"zod": "^3.17.3" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
import { BaseCommandLine } from '@basemaps/shared/build/cli/base.js'; | ||
import 'source-map-support/register.js'; | ||
import { CommandScreenShot } from './screenshot.js'; | ||
|
||
export class BasemapsConfig extends BaseCommandLine { | ||
constructor() { | ||
super({ | ||
toolFilename: 'bmc', | ||
toolDescription: 'Basemaps config command tools', | ||
}); | ||
this.addAction(new CommandScreenShot()); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,142 @@ | ||
import { Config, fsa, LogConfig, LogType } from '@basemaps/shared'; | ||
import { mkdir } from 'fs/promises'; | ||
import { Browser, chromium } from 'playwright'; | ||
import { CommandLineAction, CommandLineStringParameter } from '@rushstack/ts-command-line'; | ||
import { z } from 'zod'; | ||
|
||
enum TileMatrixIdentifier { | ||
Nztm2000Quad = 'NZTM2000Quad', | ||
Google = 'WebMercatorQuad', | ||
} | ||
|
||
const zLocation = z.object({ | ||
lat: z.number().gte(-90).lte(90), | ||
lng: z.number().gte(-180).lte(180), | ||
z: z.number().gte(0).lte(32), | ||
}); | ||
|
||
const zTileTest = z.object({ | ||
name: z.string(), | ||
tileMatrix: z.nativeEnum(TileMatrixIdentifier), | ||
location: zLocation, | ||
tileSet: z.string(), | ||
style: z.string().optional(), | ||
}); | ||
|
||
export type TileTestSchema = z.infer<typeof zTileTest>; | ||
|
||
export class CommandScreenShot extends CommandLineAction { | ||
private host: CommandLineStringParameter; | ||
private tag: CommandLineStringParameter; | ||
private tiles: CommandLineStringParameter; | ||
|
||
public constructor() { | ||
super({ | ||
actionName: 'screenshot', | ||
summary: 'dump screenshots of from LINZ Basemaps', | ||
documentation: 'Dump screenshots with selected tile sets', | ||
}); | ||
} | ||
|
||
protected onDefineParameters(): void { | ||
this.host = this.defineStringParameter({ | ||
argumentName: 'HOST', | ||
parameterLongName: '--host', | ||
description: 'Host to use', | ||
defaultValue: 'basemaps.linz.govt.nz', | ||
}); | ||
|
||
this.tag = this.defineStringParameter({ | ||
argumentName: 'TAG', | ||
parameterShortName: '-t', | ||
parameterLongName: '--tag', | ||
description: 'PR tag(PR-number) or "production"', | ||
defaultValue: 'production', | ||
}); | ||
|
||
this.tiles = this.defineStringParameter({ | ||
argumentName: 'TILES', | ||
parameterLongName: '--tiles', | ||
description: 'JSON file path for the test tiles', | ||
defaultValue: './test-tiles/default.test.tiles.json', | ||
}); | ||
} | ||
|
||
async onExecute(): Promise<void> { | ||
const logger = LogConfig.get(); | ||
logger.info('Page:Launch'); | ||
const chrome = await chromium.launch(); | ||
|
||
try { | ||
await this.takeScreenshots(chrome, logger); | ||
} finally { | ||
await chrome.close(); | ||
} | ||
} | ||
|
||
async takeScreenshots(chrome: Browser, logger: LogType): Promise<void> { | ||
const host = this.host.value ?? this.host.defaultValue; | ||
const tag = this.tag.value ?? this.tag.defaultValue; | ||
const tiles = this.tiles.value ?? this.tiles.defaultValue; | ||
if (host == null || tag == null || tiles == null) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can this ever happen? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think so. But we need a checking in typescript to parse them as |
||
throw new Error('Missing essential parameter to run the process.'); | ||
|
||
const TestTiles = await fsa.readJson<TileTestSchema[]>(tiles); | ||
for (const test of TestTiles) { | ||
const page = await chrome.newPage(); | ||
|
||
const tileSetId = await this.getTileSetId(test.tileSet, tag); | ||
const styleId = await this.getStyleId(test.style, tag); | ||
|
||
const searchParam = new URLSearchParams(); | ||
searchParam.set('p', test.tileMatrix); | ||
searchParam.set('i', tileSetId); | ||
if (styleId) searchParam.set('s', styleId); | ||
|
||
const loc = `@${test.location.lat},${test.location.lng},z${test.location.z}`; | ||
const fileName = '.artifacts/visual-snapshots/' + host + '_' + test.name + '.png'; | ||
|
||
await mkdir(`.artifacts/visual-snapshots/`, { recursive: true }); | ||
|
||
const url = `https://${host}/?${searchParam.toString()}&debug=true&debug.screenshot=true#${loc}`; | ||
|
||
logger.info({ url, expected: fileName }, 'Page:Load'); | ||
|
||
await page.goto(url); | ||
|
||
try { | ||
await page.waitForSelector('div#map-loaded', { state: 'attached' }); | ||
await page.waitForTimeout(1000); | ||
await page.waitForLoadState('networkidle'); | ||
await page.screenshot({ path: fileName }); | ||
} catch (e) { | ||
await page.screenshot({ path: fileName }); | ||
throw e; | ||
} | ||
logger.info({ url, expected: fileName }, 'Page:Load:Done'); | ||
await page.close(); | ||
} | ||
} | ||
|
||
async getTileSetId(tileSetId: string, tag: string): Promise<string> { | ||
if (tag === 'production') return tileSetId; | ||
|
||
const tileSetTagId = `${tileSetId}@${tag}`; | ||
const dbId = Config.TileSet.id(tileSetTagId); | ||
const tileSet = await Config.TileSet.get(dbId); | ||
|
||
if (tileSet) return tileSetTagId; | ||
return tileSetId; | ||
} | ||
|
||
async getStyleId(styleId: string | undefined, tag: string): Promise<string> { | ||
if (styleId == null) return ''; | ||
if (tag === 'production') return styleId ?? ''; | ||
|
||
const styleIdTagId = `${styleId}@${tag}`; | ||
const dbId = Config.Style.id(styleIdTagId); | ||
const style = await Config.Style.get(dbId); | ||
if (style) return styleIdTagId; | ||
return styleId; | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
[ | ||
{ | ||
"name": "health-3857-z5", | ||
"tileMatrix": "WebMercatorQuad", | ||
"location": { "lat": -41.8899962, "lng": 174.0492437, "z": 5 }, | ||
"tileSet": "health" | ||
}, | ||
{ | ||
"name": "health-2193-z5", | ||
"tileMatrix": "NZTM2000Quad", | ||
"location": { "lat": -41.8899962, "lng": 174.0492437, "z": 1 }, | ||
"tileSet": "aerial" | ||
}, | ||
{ | ||
"name": "topographic-3857-z5", | ||
"tileMatrix": "WebMercatorQuad", | ||
"location": { "lat": -41.8899962, "lng": 174.0492437, "z": 5 }, | ||
"tileSet": "topographic", | ||
"style": "topographic" | ||
}, | ||
{ | ||
"name": "topolite-3857-z5", | ||
"tileMatrix": "WebMercatorQuad", | ||
"location": { "lat": -41.8899962, "lng": 174.0492437, "z": 5 }, | ||
"tileSet": "topographic", | ||
"style": "topolite" | ||
}, | ||
{ | ||
"name": "topographic-3857-z14", | ||
"tileMatrix": "WebMercatorQuad", | ||
"location": { "lat": -41.8899962, "lng": 174.0492437, "z": 14 }, | ||
"tileSet": "topographic", | ||
"style": "topographic" | ||
}, | ||
{ | ||
"name": "topolite-3857-z17", | ||
"tileMatrix": "WebMercatorQuad", | ||
"location": { "lat": -43.8063936, "lng": 172.9679876, "z": 17 }, | ||
"tileSet": "topographic", | ||
"style": "topolite" | ||
} | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
{ | ||
"extends": "../../tsconfig.base.json", | ||
"compilerOptions": { | ||
"target": "ES2020", | ||
"lib": ["ES2020", "DOM"], | ||
"module": "ES2020", | ||
"moduleResolution": "node", | ||
"rootDir": "./src", | ||
"outDir": "./build" | ||
}, | ||
"include": ["src/**/*"], | ||
"references": [{ "path": "../__tests__" }] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NIT: if its described as
bin : { "bmc": ... }
the actual command will bebmc
once you npm install it