Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] depCache bundling mode #951

Merged
merged 30 commits into from
Dec 6, 2023
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
baf7223
Add depCache functionality to the builder
d3xter666 Nov 1, 2023
7d4a7fa
depCache for ComponentPreload task
d3xter666 Nov 1, 2023
b451870
depCache for LibraryPreload task
d3xter666 Nov 1, 2023
ba07977
Do not modify preloads for lib & comp default tasks
d3xter666 Nov 3, 2023
cbb22b4
(Re-)include filtered files for depCache mode
d3xter666 Nov 3, 2023
31a88ae
Do not duplicate bundleInfo's subModules
d3xter666 Nov 3, 2023
e3d4615
Add tests
d3xter666 Nov 3, 2023
08e61ff
Update lib/lbt/bundle/BundleDefinition.js
d3xter666 Nov 15, 2023
16c065f
Replace placeholders with string literals & refactor resource fetch e…
d3xter666 Nov 15, 2023
44396b0
Update JSDoc
d3xter666 Nov 15, 2023
52d1c3a
Add depCache to the AutoSplitter
d3xter666 Nov 15, 2023
acdfe6a
Do not write depCache data if there are no dependencies at all
d3xter666 Nov 15, 2023
a27f4d9
Add tests
d3xter666 Nov 15, 2023
798db53
Revert JSDoc
d3xter666 Nov 15, 2023
6def278
Remove obsolete code
d3xter666 Nov 15, 2023
daad21d
Use Symbols for moduleSizes map
d3xter666 Nov 16, 2023
10d570e
Add tests for the Splitter
d3xter666 Nov 16, 2023
ba48390
Enhance tests for AutoSplitter
d3xter666 Nov 16, 2023
3df4cef
Re-arrange tests
d3xter666 Nov 16, 2023
0ecfd7d
Enable auto split for depCache
d3xter666 Nov 17, 2023
7574c76
fix: Remove depCache duplicate definitions across bundles
d3xter666 Nov 17, 2023
981c6fc
Enhance tests for depCache in AutoSplitter
d3xter666 Nov 17, 2023
169d523
Refactor depCache autosplitter
d3xter666 Nov 17, 2023
8b58024
DepCache test for overlapping modules
d3xter666 Nov 17, 2023
c02ee04
Fix typo
d3xter666 Nov 20, 2023
822d6ab
Fix AutoSplitter dependencies resolution
d3xter666 Dec 4, 2023
3f2dfc4
Enhance tests for builder to support splitting and properly test spli…
d3xter666 Dec 4, 2023
31bb2c4
Provide reliable integration test for depCache with AutoSplit
d3xter666 Dec 5, 2023
5a19d47
Add tests for AutoSplitter
d3xter666 Dec 5, 2023
f66676f
Enable all tests for AutoSplitter
d3xter666 Dec 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions lib/lbt/bundle/AutoSplitter.js
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,25 @@ class AutoSplitter {
totalSize += "sap.ui.requireSync('');".length + toRequireJSName(module).length;
});
break;
case SectionType.DepCache:
moduleSizes["__depCacheSize"] = "sap.ui.loader.config({depCacheUI5:{}});".length;
d3xter666 marked this conversation as resolved.
Show resolved Hide resolved
d3xter666 marked this conversation as resolved.
Show resolved Hide resolved

section.modules.forEach( (module) => {
promises.push((async () => {
const resource = await this.pool.findResourceWithInfo(module);
const deps = resource.info.dependencies.filter(
(dep) =>
!resource.info.isConditionalDependency(dep) &&
!resource.info.isImplicitDependency(dep)
);
if (deps.length > 0) {
const depSize = `"${module}": [${deps.map((dep) => `"${dep}"`).join(",")}],`.length;
moduleSizes["__depCacheSize"] += depSize;
totalSize += depSize;
}
})());
});
break;
default:
break;
}
Expand Down Expand Up @@ -180,6 +199,15 @@ class AutoSplitter {
totalSize += 21 + toRequireJSName(module).length;
});
break;
case SectionType.DepCache:
currentSection = {
name: section.name,
mode: SectionType.DepCache,
filters: section.modules.slice()
};
currentModule.sections.push( currentSection );
totalSize += moduleSizes["__depCacheSize"];
d3xter666 marked this conversation as resolved.
Show resolved Hide resolved
break;
default:
break;
}
Expand Down
54 changes: 54 additions & 0 deletions lib/lbt/bundle/Builder.js
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,14 @@ const EVOBundleFormat = {
resolvedModule.executes(MODULE__UI5LOADER_AUTOCONFIG) ||
resolvedModule.executes(MODULE__JQUERY_SAP_GLOBAL) ||
resolvedModule.executes(MODULE__SAP_UI_CORE_CORE);
},

beforeDepCache(outW) {
outW.writeln(`sap.ui.loader.config({depCacheUI5:{`);
},

afterDepCache(outW) {
outW.writeln(`}});`);
}
};

Expand Down Expand Up @@ -220,6 +228,8 @@ class BundleBuilder {
return this.writeBundleInfos([section]);
case SectionType.Require:
return this.writeRequires(section);
case SectionType.DepCache:
d3xter666 marked this conversation as resolved.
Show resolved Hide resolved
return this.writeDepCache(section);
default:
throw new Error("unknown section mode " + section.mode);
}
Expand Down Expand Up @@ -549,6 +559,50 @@ class BundleBuilder {
});
}

async writeDepCache(section) {
const outW = this.outW;
let hasDepCache = false;

const sequence = section.modules.slice().sort();

if (sequence.length > 0) {
// let i = 0;
d3xter666 marked this conversation as resolved.
Show resolved Hide resolved
for (const module of sequence) {
let resource = null;
try {
resource = await this.pool.findResourceWithInfo(module);
} catch (e) {
log.error(` couldn't find ${module}`);
}

if (resource != null) {
const deps = resource.info.dependencies.filter(
(dep) =>
!resource.info.isConditionalDependency(dep) &&
!resource.info.isImplicitDependency(dep)
);
if (deps.length > 0) {
if (!hasDepCache) {
hasDepCache = true;
outW.ensureNewLine();
this.targetBundleFormat.beforeDepCache(outW, section);
}

outW.writeln(
`"${module}": [${deps.map((dep) => `"${dep}"`).join(",")}],`
);
} else {
log.verbose(` skipped ${module}, no dependencies`);
}
}
}

if (hasDepCache) {
this.targetBundleFormat.afterDepCache(outW, section);
}
}
}

async getSourceMapForModule({moduleName, moduleContent, resourcePath}) {
let moduleSourceMap = null;
let newModuleContent = moduleContent;
Expand Down
9 changes: 8 additions & 1 deletion lib/lbt/bundle/BundleDefinition.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,12 @@ export const SectionType = {
* Usually used as the last section in a merged module to enforce loading and
* execution of some specific module or modules.
*/
Require: "require"
Require: "require",

/**
* Dependency cache information that lists modules and their dependencies
* of all types: JS, declarative views/fragments.
* Only the dependencies of the modules are stored as 'depCache' configuration.
d3xter666 marked this conversation as resolved.
Show resolved Hide resolved
*/
DepCache: "depCache"
};
6 changes: 5 additions & 1 deletion lib/lbt/bundle/ResolvedBundleDefinition.js
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,11 @@ class ResolvedBundleDefinition {
return Promise.all(
modules.map( (submodule) => {
return pool.getModuleInfo(submodule).then(
(subinfo) => bundleInfo.addSubModule(subinfo)
(subinfo) => {
if (!bundleInfo.subModules.includes(subinfo.name)) {
bundleInfo.addSubModule(subinfo);
}
}
);
})
);
Expand Down
4 changes: 2 additions & 2 deletions lib/lbt/bundle/Resolver.js
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ class BundleResolver {
let oldIgnoredResources;
let oldSelectedResourcesSequence;

if ( section.mode == SectionType.Require ) {
if ( [SectionType.Require, SectionType.DepCache].includes(section.mode) ) {
oldSelectedResources = selectedResources;
oldIgnoredResources = visitedResources;
oldSelectedResourcesSequence = selectedResourcesSequence;
Expand Down Expand Up @@ -254,7 +254,7 @@ class BundleResolver {
});

return Promise.all(promises).then( function() {
if ( section.mode == SectionType.Require ) {
if ( [SectionType.Require, SectionType.DepCache].includes(section.mode) ) {
newKeys = selectedResourcesSequence;
selectedResources = oldSelectedResources;
visitedResources = oldIgnoredResources;
Expand Down
192 changes: 192 additions & 0 deletions test/lib/lbt/bundle/Builder.js
Original file line number Diff line number Diff line change
Expand Up @@ -817,6 +817,198 @@ ${SOURCE_MAPPING_URL}=library-preload.js.map
]);
});

test.serial("integration: createBundle with depCache", async (t) => {
d3xter666 marked this conversation as resolved.
Show resolved Hide resolved
d3xter666 marked this conversation as resolved.
Show resolved Hide resolved
const pool = new ResourcePool();
pool.addResource({
name: "a.js",
getPath: () => "a.js",
string: function() {
return this.buffer();
},
buffer: async () => "sap.ui.define([\"./b\", \"./c2\"],function(b, c){return {};});"
});
pool.addResource({
name: "b.js",
getPath: () => "b.js",
string: function() {
return this.buffer();
},
buffer: async () => "function Two(){return 2;}"
});
pool.addResource({
name: "c2.js",
getPath: () => "c2.js",
string: function() {
return this.buffer();
},
buffer: async () => "sap.ui.define([\"./c1\", \"./c3\"],function(c1, c3){return {};});"
});
pool.addResource({
name: "c1.js",
getPath: () => "c1.js",
string: function() {
return this.buffer();
},
buffer: async () => "function Three(){return 3.1;}"
});
pool.addResource({
name: "c3.js",
getPath: () => "c3.js",
string: function() {
return this.buffer();
},
buffer: async () => "function Three(){return 3.3;}"
});
pool.addResource({
name: "a.library",
getPath: () => "a.library",
string: function() {
return this.buffer();
},
buffer: async () => `<?xml version="1.0" encoding="UTF-8" ?>
<library xmlns="http://www.sap.com/sap.ui.library.xsd" >
<appData>
<packaging xmlns="http://www.sap.com/ui5/buildext/packaging" version="2.0" >
<module-infos>
<raw-module name="a.js"
requiresTopLevelScope="false" />
</module-infos>
</packaging>
</appData>
</library>`
});

const bundleDefinition = {
name: `library-depCache-preload.js`,
sections: [{
mode: "preload",
name: "preload-section",
filters: ["a.js"]
}, {
mode: "depCache",
filters: ["*.js"]
}]
};

const builder = new Builder(pool);
const oResult = await builder.createBundle(bundleDefinition, {});
t.is(oResult.name, "library-depCache-preload.js");
const expectedContent = `//@ui5-bundle library-depCache-preload.js
sap.ui.require.preload({
"a.js":function(){
sap.ui.define(["./b", "./c2"],function(b, c){return {};});
}
},"preload-section");
sap.ui.loader.config({depCacheUI5:{
"a.js": ["b.js","c2.js"],
"c2.js": ["c1.js","c3.js"],
}});
${SOURCE_MAPPING_URL}=library-depCache-preload.js.map
`;
t.deepEqual(oResult.content, expectedContent, "EVOBundleFormat " +
"should contain:" +
" preload part from a.js" +
" depCache part from a.js && c2.js");
t.is(oResult.bundleInfo.name, "library-depCache-preload.js", "bundle info name is correct");
t.deepEqual(oResult.bundleInfo.size, expectedContent.length, "bundle info size is correct");
t.deepEqual(oResult.bundleInfo.subModules, ["a.js", "b.js", "c2.js", "c1.js", "c3.js"],
"bundle info subModules are correct");
});

test.serial("integration: createBundle with depCache with NO dependencies", async (t) => {
matz3 marked this conversation as resolved.
Show resolved Hide resolved
const pool = new ResourcePool();
pool.addResource({
name: "a.js",
getPath: () => "a.js",
string: function() {
return this.buffer();
},
buffer: async () => "sap.ui.define([],function(){return {};});"
});
pool.addResource({
name: "b.js",
getPath: () => "b.js",
string: function() {
return this.buffer();
},
buffer: async () => "function Two(){return 2;}"
});
pool.addResource({
name: "c2.js",
getPath: () => "c2.js",
string: function() {
return this.buffer();
},
buffer: async () => "sap.ui.define([],function(){return {};});"
});
pool.addResource({
name: "c1.js",
getPath: () => "c1.js",
string: function() {
return this.buffer();
},
buffer: async () => "function Three(){return 3.1;}"
});
pool.addResource({
name: "c3.js",
getPath: () => "c3.js",
string: function() {
return this.buffer();
},
buffer: async () => "function Three(){return 3.3;}"
});
pool.addResource({
name: "a.library",
getPath: () => "a.library",
string: function() {
return this.buffer();
},
buffer: async () => `<?xml version="1.0" encoding="UTF-8" ?>
<library xmlns="http://www.sap.com/sap.ui.library.xsd" >
<appData>
<packaging xmlns="http://www.sap.com/ui5/buildext/packaging" version="2.0" >
<module-infos>
<raw-module name="a.js"
requiresTopLevelScope="false" />
</module-infos>
</packaging>
</appData>
</library>`
});

const bundleDefinition = {
name: `library-depCache-preload.js`,
sections: [{
mode: "preload",
name: "preload-section",
filters: ["a.js"]
}, {
mode: "depCache",
filters: ["*.js"]
}]
};

const builder = new Builder(pool);
const oResult = await builder.createBundle(bundleDefinition, {});
t.is(oResult.name, "library-depCache-preload.js");
const expectedContent = `//@ui5-bundle library-depCache-preload.js
sap.ui.require.preload({
"a.js":function(){
sap.ui.define([],function(){return {};});
}
},"preload-section");
${SOURCE_MAPPING_URL}=library-depCache-preload.js.map
`;
t.deepEqual(oResult.content, expectedContent, "EVOBundleFormat " +
"should contain:" +
" preload part from a.js" +
" depCache part from a.js && c2.js");
t.is(oResult.bundleInfo.name, "library-depCache-preload.js", "bundle info name is correct");
t.deepEqual(oResult.bundleInfo.size, expectedContent.length, "bundle info size is correct");
t.deepEqual(oResult.bundleInfo.subModules, ["a.js", "b.js", "c2.js", "c1.js", "c3.js"],
"bundle info subModules are correct");
});

test("integration: createBundle using predefine calls with source maps and a single, simple source", async (t) => {
const pool = new ResourcePool();

Expand Down
Loading