Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support asconfig.json #1238

Merged
merged 69 commits into from
Jul 16, 2020
Merged
Show file tree
Hide file tree
Changes from 67 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
67d8013
implement naive asconfig
jtenner Jun 17, 2020
6a70013
switch to args, whoops
jtenner Jun 17, 2020
4cf1b2b
semicolon for lint
jtenner Jun 17, 2020
0115341
each merge returns an options object
jtenner Jun 17, 2020
cb6e9ac
try to follow coding style, maybe use for of loop?
jtenner Jun 17, 2020
f071ae8
Actually, ' Object' is not 'Object'
jtenner Jun 17, 2020
dd12c9a
add asconfig.entries to compilation, keep track of asconfigDir for la…
jtenner Jun 20, 2020
6f98121
take into account relative baseDirs
jtenner Jun 20, 2020
d637a1f
fix small linted error
jtenner Jun 20, 2020
ef481e5
Merge branch 'asconfig' of https://github.com/jtenner/assemblyscript …
jtenner Jun 29, 2020
483cc9d
Setup tests for asconfig
jtenner Jul 1, 2020
9f602b1
Merge branch 'master' of https://github.com/assemblyscript/assemblysc…
jtenner Jul 1, 2020
ca95430
add const test
jtenner Jul 1, 2020
3257573
test single level inheritence with targets
jtenner Jul 1, 2020
e05a71f
config extension test, switch getAsconfig params
jtenner Jul 2, 2020
a0700c6
Modify asinit
jtenner Jul 3, 2020
4dd142e
typos
jtenner Jul 3, 2020
3b5a8ce
switch code style to regular functions
jtenner Jul 3, 2020
2a4e351
leading newline in json
jtenner Jul 3, 2020
9e0d217
config changes
jtenner Jul 3, 2020
7155d50
switch to isObject helper, and use typeof for string validation
jtenner Jul 3, 2020
a7a3999
semicolon
jtenner Jul 3, 2020
4e3de7d
Update bin/asinit
jtenner Jul 3, 2020
3e39805
Merge branch 'master' of https://github.com/assemblyscript/assemblysc…
jtenner Jul 3, 2020
932961d
switch to more general unique function
jtenner Jul 3, 2020
a29f617
unique cleanups
jtenner Jul 3, 2020
2ead5ae
Update cli/asc.js
jtenner Jul 3, 2020
c6af0dd
ignore cyclical dependencies
jtenner Jul 5, 2020
ec7d44c
Update cli/asc.js
jtenner Jul 5, 2020
d3abe65
Merge branch 'asconfig' of https://github.com/jtenner/assemblyscript …
jtenner Jul 5, 2020
acd8725
suggested changes
jtenner Jul 5, 2020
8195764
Swap merge args, rename config.entries --> config.include
Jul 8, 2020
f30d456
Rename include back to entries
Jul 9, 2020
7e26838
Added entry-points test
Jul 10, 2020
d8bd07b
Swap back and fix transform
Jul 10, 2020
7827a93
Added more tests
Jul 10, 2020
3e0303d
Merge pull request #1 from willemneal/asconfig
jtenner Jul 11, 2020
bb4cdaf
Merge branch 'master' of https://github.com/assemblyscript/assemblysc…
jtenner Jul 11, 2020
f44491c
cleanup
jtenner Jul 11, 2020
ac37a14
add watfile output
jtenner Jul 11, 2020
a77826c
remove complicated test
jtenner Jul 13, 2020
7c4e5cf
remove options test matching, why export argv/args
jtenner Jul 13, 2020
2ef00d0
Merge branch 'master' of https://github.com/assemblyscript/assemblysc…
jtenner Jul 13, 2020
3ad6d88
Whoops
jtenner Jul 13, 2020
ca18d62
make tests pass. What's next?
jtenner Jul 13, 2020
7f25365
linting issues
jtenner Jul 13, 2020
dc55518
linting issues
jtenner Jul 13, 2020
cbcb88e
Add showConfig to print out config and exit
Jul 13, 2020
9e35892
fix linting issue with indentation? really?
jtenner Jul 14, 2020
42703a1
remove watfile output, it's a bug, add complicated test
jtenner Jul 14, 2020
0f1803d
add complicated test
jtenner Jul 14, 2020
b177e94
check memory size here, switch args merge order
jtenner Jul 14, 2020
e80cd8b
Merge remote-tracking branch 'jtenner/asconfig' into asconfig
Jul 15, 2020
d3a347d
Merge pull request #2 from willemneal/asconfig
jtenner Jul 15, 2020
2495578
Update cli/asc.js
jtenner Jul 15, 2020
ad7d24a
Update cli/asc.json
jtenner Jul 15, 2020
9e4aebf
Update bin/asinit
jtenner Jul 15, 2020
debc2fc
Update bin/asinit
jtenner Jul 15, 2020
00e9b27
Update cli/asc.json
jtenner Jul 15, 2020
d304b36
Update tests/asconfig/entry-points/nested/assembly/index.ts
jtenner Jul 15, 2020
16bfe42
Update bin/asinit
jtenner Jul 15, 2020
bc24391
Update cli/asc.js
jtenner Jul 15, 2020
27656af
remove flags
jtenner Jul 15, 2020
e2bf925
Merge branch 'asconfig' of https://github.com/jtenner/assemblyscript …
jtenner Jul 15, 2020
8f5e8fb
add suggested target description
jtenner Jul 15, 2020
55135bc
remove path.join because resolve performs concatenation already
jtenner Jul 15, 2020
1b1fc19
updated test to include 'optimize' flag
jtenner Jul 15, 2020
62caba3
rename to resolveBasedir
jtenner Jul 15, 2020
0d0e29e
Merge branch 'master' of https://github.com/assemblyscript/assemblysc…
jtenner Jul 15, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ out/
raw/
.history
*.backup
.vscode
38 changes: 36 additions & 2 deletions bin/asinit
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ const compilerDir = path.join(__dirname, "..");
const compilerVersion = require(path.join(compilerDir, "package.json")).version;
const assemblyDir = path.join(projectDir, "assembly");
const tsconfigFile = path.join(assemblyDir, "tsconfig.json");
const asconfigFile = path.join(projectDir, "asconfig.json");
let tsconfigBase = path.relative(assemblyDir, path.join(compilerDir, "std", "assembly.json"));
if (/^(\.\.[/\\])*node_modules[/\\]assemblyscript[/\\]/.test(tsconfigBase)) {
// Use node resolution if the compiler is a normal dependency
Expand Down Expand Up @@ -84,6 +85,9 @@ console.log([
colors.cyan(" ./tests/index.js"),
" Example test to check that your module is indeed working.",
"",
colors.cyan(" ./asconfig.json"),
" Configuration file defining both a 'debug' and a 'release' target.",
"",
colors.cyan(" ./package.json"),
" Package info containing the necessary commands to compile to WebAssembly.",
"",
Expand All @@ -108,6 +112,7 @@ function createProject(answer) {
ensureIndexJs();
ensureTestsDirectory();
ensureTestsIndexJs();
ensureAsconfigJson();
console.log([
colors.green("Done!"),
"",
Expand Down Expand Up @@ -207,6 +212,35 @@ function ensureTsconfigJson() {
console.log();
}

function ensureAsconfigJson() {
console.log("- Making sure that 'asconfig.json' is set up...");
if (!fs.existsSync(asconfigFile)) {
fs.writeFileSync(asconfigFile, JSON.stringify({
targets: {
debug: {
// -b build/untouched.wasm -t build/untouched.wat --sourceMap --debug
binaryFile: "build/untouched.wasm",
textFile: "build/untouched.wat",
sourceMap: true,
debug: true
},
release: {
jtenner marked this conversation as resolved.
Show resolved Hide resolved
// -b build/optimized.wasm -t build/optimized.wat --sourceMap --optimize
binaryFile: "build/optimized.wasm",
textFile: "build/optimized.wat",
sourceMap: true,
optimize: true
}
},
options: {}
}, null, 2));
console.log(colors.green(" Created: ") + asconfigFile);
} else {
console.log(colors.yellow(" Exists: ") + asconfigFile);
}
console.log();
}

function ensureEntryFile() {
console.log("- Making sure that 'assembly/index.ts' exists...");
if (!fs.existsSync(entryFile)) {
Expand Down Expand Up @@ -253,8 +287,8 @@ function ensureGitignore() {
function ensurePackageJson() {
console.log("- Making sure that 'package.json' contains the build commands...");
const entryPath = path.relative(projectDir, entryFile).replace(/\\/g, "/");
const buildUntouched = "asc " + entryPath + " -b build/untouched.wasm -t build/untouched.wat --sourceMap --debug";
const buildOptimized = "asc " + entryPath + " -b build/optimized.wasm -t build/optimized.wat --sourceMap --optimize";
const buildUntouched = "asc " + entryPath + " --target debug";
const buildOptimized = "asc " + entryPath + " --target release";
PerachBD marked this conversation as resolved.
Show resolved Hide resolved
const buildAll = "npm run asbuild:untouched && npm run asbuild:optimized";
if (!fs.existsSync(packageFile)) {
fs.writeFileSync(packageFile, JSON.stringify({
Expand Down
150 changes: 144 additions & 6 deletions cli/asc.js
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,8 @@ exports.main = function main(argv, options, callback) {
if (!stderr) throw Error("'options.stderr' must be specified");

const opts = optionsUtil.parse(argv, exports.options);
const args = opts.options;
let args = opts.options;

argv = opts.arguments;
if (args.noColors) {
colorsUtil.stdout.supported =
Expand Down Expand Up @@ -270,6 +271,91 @@ exports.main = function main(argv, options, callback) {

// Set up base directory
const baseDir = args.baseDir ? path.resolve(args.baseDir) : ".";
const target = args.target;

// Once the baseDir is calculated, we can resolve the config, and its extensions
let asconfig = getAsconfig(args.config, baseDir, readFile);
let asconfigDir = baseDir;

const seenAsconfig = new Set();
seenAsconfig.add(path.join(baseDir, args.config));

while (asconfig) {
// merge target first, then merge options, then merge extended asconfigs
if (asconfig.targets && asconfig.targets[target]) {
args = optionsUtil.merge(exports.options, asconfig.targets[target], args);
}
if (asconfig.options) {
if (asconfig.options.transform) {
// ensure that a transform's path is relative to the current config
asconfig.options.transform = asconfig.options.transform.map(p => {
if (!path.isAbsolute(p)) {
if (p.startsWith(".")) {
return path.join(asconfigDir, p);
}
return require.resolve(p);
}
return p;
});
}
args = optionsUtil.merge(exports.options, args, asconfig.options);
}

// entries are added to the compilation
if (asconfig.entries) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There could also be a workspace field, which could each have its own ascofig.json.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or should this been it's own higher level cli like cargo?

for (const entry of asconfig.entries) {
argv.push(
path.isAbsolute(entry)
? entry
// the entry is relative to the asconfig directory
: path.join(asconfigDir, entry)
);
}
}

// asconfig "extends" another config, merging options of it's parent
if (asconfig.extends) {
asconfigDir = path.isAbsolute(asconfig.extends)
// absolute extension path means we know the exact directory and location
? path.dirname(asconfig.extends)
// relative means we need to calculate a relative asconfigDir
: path.join(asconfigDir, path.dirname(asconfig.extends));
const fileName = path.basename(asconfig.extends);
const filePath = path.join(asconfigDir, fileName);
if (seenAsconfig.has(filePath)) {
asconfig = null;
} else {
seenAsconfig.add(filePath);
asconfig = getAsconfig(fileName, asconfigDir, readFile);
}
} else {
asconfig = null; // finished resolving the configuration chain
}
}

// If showConfig print args and exit
if (args.showConfig) {
stderr.write(JSON.stringify(args, null, 2));
return callback(null);
}

// This method resolves a path relative to the baseDir instead of process.cwd()
function resolve(arg) {
if (path.isAbsolute(arg)) return arg;
jtenner marked this conversation as resolved.
Show resolved Hide resolved
return path.resolve(baseDir, arg);
}

// create a unique set of values
function unique(values) {
return [...new Set(values)];
}

// returns a relative path from baseDir
function makeRelative(arg) {
return path.relative(baseDir, arg);
}
// postprocess we need to get absolute file locations argv
argv = unique(argv.map(resolve)).map(makeRelative);

// Set up options
const compilerOptions = assemblyscript.newOptions();
Expand Down Expand Up @@ -347,7 +433,7 @@ exports.main = function main(argv, options, callback) {
const transforms = [];
if (args.transform) {
let tsNodeRegistered = false;
let transformArgs = args.transform;
let transformArgs = unique(args.transform.map(resolve));
for (let i = 0, k = transformArgs.length; i < k; ++i) {
let filename = transformArgs[i].trim();
if (!tsNodeRegistered && filename.endsWith(".ts")) { // ts-node requires .ts specifically
Expand Down Expand Up @@ -376,6 +462,7 @@ exports.main = function main(argv, options, callback) {
}
}
}

function applyTransform(name, ...args) {
for (let i = 0, k = transforms.length; i < k; ++i) {
let transform = transforms[i];
Expand All @@ -400,11 +487,12 @@ exports.main = function main(argv, options, callback) {
assemblyscript.parse(program, exports.libraryFiles[libPath], exports.libraryPrefix + libPath + extension.ext, false);
});
});
const customLibDirs = [];
let customLibDirs = [];
if (args.lib) {
let lib = args.lib;
if (typeof lib === "string") lib = lib.split(",");
Array.prototype.push.apply(customLibDirs, lib.map(lib => lib.trim()));
if (typeof lib === "string") lib = lib.trim().split(/\s*,\s*/);
customLibDirs.push(...lib.map(resolve));
customLibDirs = unique(customLibDirs); // `lib` and `customLibDirs` may include duplicates
for (let i = 0, k = customLibDirs.length; i < k; ++i) { // custom
let libDir = customLibDirs[i];
let libFiles;
Expand Down Expand Up @@ -574,7 +662,7 @@ exports.main = function main(argv, options, callback) {
const filename = argv[i];

let sourcePath = String(filename).replace(/\\/g, "/").replace(extension.re, "").replace(/[\\/]$/, "");

// Setting the path to relative path
sourcePath = path.isAbsolute(sourcePath) ? path.relative(baseDir, sourcePath) : sourcePath;

Expand Down Expand Up @@ -925,6 +1013,56 @@ exports.main = function main(argv, options, callback) {
}
};

const toString = Object.prototype.toString;

function isObject(arg) {
return toString.call(arg) === "[object Object]";
}

function getAsconfig(file, baseDir, readFile) {
const contents = readFile(file, baseDir);
const location = path.join(baseDir, file);
if (!contents) return null;

// obtain the configuration
let config;
try {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

asconfig validation could probably look differently. Do you have any suggestions, or is this good?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking just the top-level entries used by asconfig itself seems good, just the typeOf helper feels a bit alien. Perhaps just helpers for isObject, isArray, isString?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Array.isArray() is a great tool for arrays. But JS isn't very nice about checking objects. I will do a small refactor to create helpers for isString() and isObject().

config = JSON.parse(contents);
} catch(ex) {
throw new Error("Asconfig is not valid json: " + location);
}

// validate asconfig shape
if (config.options && !isObject(config.options)) {
throw new Error("Asconfig.options is not an object: " + location);
}

if (config.include && !Array.isArray(config.include)) {
throw new Error("Asconfig.include is not an array: " + location);
}

if (config.targets) {
if (!isObject(config.targets)) {
throw new Error("Asconfig.targets is not an object: " + location);
}
const targets = Object.keys(config.targets);
for (let i = 0; i < targets.length; i++) {
const target = targets[i];
if (!isObject(config.targets[target])) {
throw new Error("Asconfig.targets." + target + " is not an object: " + location);
}
}
}

if (config.extends && typeof config.extends !== "string") {
throw new Error("Asconfig.extends is not a string: " + location);
}

return config;
}

exports.getAsconfig = getAsconfig;

/** Checks diagnostics emitted so far for errors. */
function checkDiagnostics(program, stderr) {
var diagnostic;
Expand Down
17 changes: 17 additions & 0 deletions cli/asc.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,18 @@
"type": "b",
"default": false
},
"config": {
"category": "General",
"description": "Configuration file to apply. CLI arguments take precedence.",
"type": "s",
"default": "asconfig.json"
},
"target": {
"category": "General",
"description": "Target configuration to use. Defaults to 'release'.",
"type": "s",
"default": "release"
},

"optimize": {
"category": "Optimization",
Expand Down Expand Up @@ -288,6 +300,11 @@
"type": "b",
"default": false
},
"showConfig": {
"description": "Print computed compiler options and exit.",
"type": "b",
"default": false
},
"measure": {
"description": "Prints measuring information on I/O and compile times.",
"type": "b",
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,12 @@
"check:config": "tsc --noEmit -p src --diagnostics --listFiles",
"check:require": "tsc --noEmit --target ESNEXT --module commonjs --experimentalDecorators tests/require/index",
"check:lint": "eslint --max-warnings 0 --ext js . && eslint --max-warnings 0 --ext ts .",
"test": "npm run test:parser && npm run test:compiler && npm run test:packages && npm run test:extension",
"test": "npm run test:parser && npm run test:compiler && npm run test:packages && npm run test:extension && npm run test:asconfig",
"test:parser": "node tests/parser",
"test:compiler": "node tests/compiler",
"test:packages": "cd tests/packages && npm run test",
"test:extension": "cd tests/extension && npm run test",
"test:asconfig": "cd tests/asconfig && npm run test",
"make": "npm run clean && npm test && npm run build && npm test",
"all": "npm run check && npm run make",
"docs": "typedoc --tsconfig tsconfig-docs.json --mode modules --name \"AssemblyScript Compiler API\" --out ./docs/api --ignoreCompilerErrors --excludeNotExported --excludePrivate --excludeExternals --exclude **/std/** --includeDeclarations --readme src/README.md",
Expand Down
16 changes: 16 additions & 0 deletions tests/asconfig/complicated/asconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"targets": {
"release": {
"optimize": true,
"runtime": "stub",
"initialMemory": 30,
"explicitStart": true,
"measure": true,
"pedantic": true
}
},
"options": {
"initialMemory": 100,
"enable": ["simd"]
}
}
6 changes: 6 additions & 0 deletions tests/asconfig/complicated/assembly/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
assert(ASC_OPTIMIZE_LEVEL == 3);
assert(ASC_SHRINK_LEVEL == 1);
assert(ASC_FEATURE_SIMD);
let size = memory.size();
trace("size", 1, size);
assert(size == 30);
6 changes: 6 additions & 0 deletions tests/asconfig/complicated/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"private": true,
"scripts": {
"test": "node ../index.js"
}
}
3 changes: 3 additions & 0 deletions tests/asconfig/cyclical/asconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"extends": "./extends.json"
}
1 change: 1 addition & 0 deletions tests/asconfig/cyclical/assembly/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
assert(true);
3 changes: 3 additions & 0 deletions tests/asconfig/cyclical/extends.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"extends": "./asconfig.json"
}
6 changes: 6 additions & 0 deletions tests/asconfig/cyclical/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"private": true,
"scripts": {
"test": "node ../index.js"
}
}
3 changes: 3 additions & 0 deletions tests/asconfig/entry-points/asconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"entries": ["assembly/globals.ts"]
}
3 changes: 3 additions & 0 deletions tests/asconfig/entry-points/assembly/globals.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
// @ts-ignore: decorator
@global const answerToLife = 42;
assert(answerToLife);
2 changes: 2 additions & 0 deletions tests/asconfig/entry-points/assembly/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@

assert(answerToLife == 42);
3 changes: 3 additions & 0 deletions tests/asconfig/entry-points/nested/asconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"extends": "../asconfig.json"
}
1 change: 1 addition & 0 deletions tests/asconfig/entry-points/nested/assembly/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
assert(answerToLife == 42);
6 changes: 6 additions & 0 deletions tests/asconfig/entry-points/nested/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"private": true,
"scripts": {
"test": "node ../../index.js"
}
}
Loading