Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add Configuration Schema #274

Merged
merged 44 commits into from
Mar 24, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
036b53e
[FEATURE] Add Configuration Schema
matz3 Mar 12, 2020
fb6f502
Add more schemas
matz3 Mar 12, 2020
e68ace7
Use tabs
matz3 Mar 12, 2020
234ad0a
Fix schema, add more tests + coverage
matz3 Mar 13, 2020
c95927a
Restructure schema files
matz3 Mar 13, 2020
7b21994
Integrate into ProjectPreprocessor / Add ValidationError
matz3 Mar 13, 2020
bbef4c6
Add framework schema
matz3 Mar 13, 2020
7c480ac
Refactor schema, validate, AjvCoverage
matz3 Mar 16, 2020
f546bd9
Improve error messages, fix tests, move AjvCoverage
matz3 Mar 16, 2020
a97dcda
Add validate test
matz3 Mar 16, 2020
cf00391
Add filterErrors test, refactor
matz3 Mar 16, 2020
c8d0ba2
validate.js -> validator.js, add yaml formatting functions
matz3 Mar 16, 2020
61de70f
Extract ValidationError from validator
matz3 Mar 16, 2020
f092963
ValidationError.analyzeYamlError
matz3 Mar 16, 2020
cd91e16
ValidationError.analyzeYamlError: Handle arrays
matz3 Mar 17, 2020
2b68bfa
Fix some special cases
matz3 Mar 17, 2020
ca03e2d
Refactor readConfigFile
matz3 Mar 17, 2020
197265a
Improve error messages
matz3 Mar 17, 2020
62cde6c
Don't use better-ajv-errors
matz3 Mar 17, 2020
e665a2b
Fix js-yaml filename, fix ui5Framework use-case (_transparentProject)
matz3 Mar 17, 2020
4e41c16
Adopt error messages
matz3 Mar 19, 2020
59a77c0
Restructure files
matz3 Mar 19, 2020
00c934d
Tests
matz3 Mar 19, 2020
21535d9
Fix Node 10
matz3 Mar 19, 2020
673d434
Schema refactoring / fixes
matz3 Mar 19, 2020
5e2c638
Add bundles schema definition
matz3 Mar 19, 2020
1724fd0
type module
matz3 Mar 20, 2020
60b2d99
Improve horizontal separator line
matz3 Mar 20, 2020
3473e94
Fix tests on Windows
matz3 Mar 20, 2020
584c500
Throw error for root project when ui5.yaml can't be read
matz3 Mar 20, 2020
558dceb
Also validate shim configuration
matz3 Mar 20, 2020
9e990fe
Remove test
matz3 Mar 20, 2020
08776eb
Code review changes part 1
matz3 Mar 20, 2020
1e72cc9
Fix ui5framework.integration.js
matz3 Mar 20, 2020
186a3a2
Code review changes part 2
matz3 Mar 23, 2020
7df1980
Code review changes part 3
matz3 Mar 24, 2020
bed154b
analyzeYamlError: support multiple documents
matz3 Mar 24, 2020
581e939
Add schema for builder/componentPreload
matz3 Mar 24, 2020
db5d4ec
Fix yaml documentIndex logic
matz3 Mar 24, 2020
8bf7f6d
Fix another test on windows
matz3 Mar 24, 2020
de84807
Support Windows Line-Endings
matz3 Mar 24, 2020
6f32895
projectPreprocessor readConfigFile: Add more tests and fix error hand…
matz3 Mar 24, 2020
3a4329e
Update coverage thresholds / Add comment
matz3 Mar 24, 2020
2341c79
Add latest changes after resolving conflicts...
matz3 Mar 24, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,15 @@ module.exports = {
Openui5Resolver: require("./lib/ui5Framework/Openui5Resolver"),
Sapui5Resolver: require("./lib/ui5Framework/Sapui5Resolver")
},
/**
* @public
* @see module:@ui5/project.validation
* @namespace
*/
validation: {
validator: require("./lib/validation/validator"),
ValidationError: require("./lib/validation/ValidationError")
},
/**
* @private
* @see module:@ui5/project.translators
Expand Down
133 changes: 110 additions & 23 deletions lib/projectPreprocessor.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ const fs = require("graceful-fs");
const path = require("path");
const {promisify} = require("util");
const readFile = promisify(fs.readFile);
const parseYaml = require("js-yaml").safeLoadAll;
const jsyaml = require("js-yaml");
const typeRepository = require("@ui5/builder").types.typeRepository;
const {validate} = require("./validation/validator");

class ProjectPreprocessor {
constructor({tree}) {
Expand Down Expand Up @@ -80,7 +81,7 @@ class ProjectPreprocessor {
return this.applyExtension(extProject);
}));
}
this.applyShims(project);
await this.applyShims(project);
if (this.isConfigValid(project)) {
// Do not apply transparent projects.
// Their only purpose might be to have their dependencies processed
Expand Down Expand Up @@ -194,26 +195,19 @@ class ProjectPreprocessor {
async loadProjectConfiguration(project) {
if (project.specVersion) { // Project might already be configured
// Currently, specVersion is the indicator for configured projects
this.normalizeConfig(project);
return {};
}

let configs;
if (project._transparentProject) {
// Assume that project is already processed
return {};
}

// A projects configPath property takes precedence over the default "<projectPath>/ui5.yaml" path
const configPath = project.configPath || path.join(project.path, "/ui5.yaml");
try {
configs = await this.readConfigFile(configPath);
} catch (err) {
const errorText = "Failed to read configuration for project " +
`${project.id} at "${configPath}". Error: ${err.message}`;
await this.validateAndNormalizeExistingProject(project);

if (err.code !== "ENOENT") { // Something else than "File or directory does not exist"
throw new Error(errorText);
}
log.verbose(errorText);
return {};
}

const configs = await this.readConfigFile(project);

if (!configs || !configs.length) {
return {};
}
Expand Down Expand Up @@ -384,11 +378,77 @@ class ProjectPreprocessor {
}
}

async readConfigFile(configPath) {
const configFile = await readFile(configPath);
return parseYaml(configFile, {
filename: path
});
async readConfigFile(project) {
// A projects configPath property takes precedence over the default "<projectPath>/ui5.yaml" path
const configPath = project.configPath || path.join(project.path, "ui5.yaml");
RandomByte marked this conversation as resolved.
Show resolved Hide resolved
let configFile;
try {
configFile = await readFile(configPath, {encoding: "utf8"});
} catch (err) {
const errorText = "Failed to read configuration for project " +
`${project.id} at "${configPath}". Error: ${err.message}`;

// Something else than "File or directory does not exist" or root project
if (err.code !== "ENOENT" || project._level === 0) {
throw new Error(errorText);
} else {
log.verbose(errorText);
return null;
}
}

let configs;

try {
// Using loadAll with DEFAULT_SAFE_SCHEMA instead of safeLoadAll to pass "filename".
// safeLoadAll doesn't handle its parameters properly.
// See https://github.com/nodeca/js-yaml/issues/456 and https://github.com/nodeca/js-yaml/pull/381
configs = jsyaml.loadAll(configFile, undefined, {
filename: configPath,
schema: jsyaml.DEFAULT_SAFE_SCHEMA
});
} catch (err) {
if (err.name === "YAMLException") {
throw new Error("Failed to parse configuration for project " +
`${project.id} at "${configPath}"\nError: ${err.message}`);
} else {
throw err;
}
}

if (!configs || !configs.length) {
return configs;
}

const validationResults = await Promise.all(
configs.map(async (config, documentIndex) => {
// Catch validation errors to ensure proper order of rejections within Promise.all
try {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should document that the try/catch block here is required to get the first error of the first document. And not the first error to occur in any document.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

await validate({
config,
project: {
id: project.id
},
yaml: {
path: configPath,
source: configFile,
documentIndex
}
});
} catch (error) {
return error;
}
})
);

const validationErrors = validationResults.filter(($) => $);

if (validationErrors.length > 0) {
// For now just throw the error of the first invalid document
throw validationErrors[0];
}

return configs;
}

handleShim(extension) {
Expand Down Expand Up @@ -451,7 +511,7 @@ class ProjectPreprocessor {
}
}

applyShims(project) {
async applyShims(project) {
const configShim = this.configShims[project.id];
// Apply configuration shims
if (configShim) {
Expand Down Expand Up @@ -482,6 +542,8 @@ class ProjectPreprocessor {

Object.assign(project, configShim);
delete project.shimDependenciesResolved; // Remove shim processing metadata from project

RandomByte marked this conversation as resolved.
Show resolved Hide resolved
await this.validateAndNormalizeExistingProject(project);
}

// Apply collections
Expand Down Expand Up @@ -539,6 +601,31 @@ class ProjectPreprocessor {
const middlewarePath = path.join(extension.path, extension.middleware.path);
middlewareRepository.addMiddleware(extension.metadata.name, middlewarePath);
}

async validateAndNormalizeExistingProject(project) {
// Validate project config, but exclude additional properties
const excludedProperties = [
"id",
"version",
"path",
"dependencies",
"_level"
];
const config = {};
for (const key in project) {
if (project.hasOwnProperty(key) && !excludedProperties.includes(key)) {
config[key] = project[key];
}
}
await validate({
config,
project: {
id: project.id
}
});

this.normalizeConfig(project);
}
}

/**
Expand Down
Loading