Build Tooling: Generate asset file in PHP format with the list depend…

…encies (#17298)

* Build Tooling: Generate asset file in PHP format with dependencies and stable version hash

* Use json2php to generate PHP version of assets file
Props to @aristath for sharing this npm package.

* Fix linting issues reported by phpcs

* Update packages/dependency-extraction-webpack-plugin/index.js

Co-Authored-By: Jon Surrell <jon.surrell@automattic.com>

* Fix linting issues reported by phpcs

* Update packages/dependency-extraction-webpack-plugin/index.js

Co-Authored-By: Jon Surrell <jon.surrell@automattic.com>

* Set php as the default output format for the Webpack plugin

* Docs: Update README file for Dependency Extraction Webpack Plugin

* Add CHANGELOG entry and update README file

* Make Dependency Extraction Webpack Plugin code exmaple less error-prone

* Changelog: Tweak "breaking changes" note

* Use require_once rather than include_once

* Promote require rather than include for the asset file

lib/client-assets.php

@@ -187,12 +187,13 @@ function gutenberg_register_packages_scripts() {
 		// For example, `…/build/a11y/index.js` becomes `wp-a11y`.
 		$handle = 'wp-' . basename( dirname( $path ) );
 
-		// Replace `.js` extension with `.deps.json` to find the generated dependencies file.
-		$dependencies_file = substr( $path, 0, -3 ) . '.deps.json';
-
-		$dependencies = is_readable( $dependencies_file )
-			? json_decode( file_get_contents( $dependencies_file ) )
-			: array();
+		// Replace `.js` extension with `.asset.php` to find the generated dependencies file.
+		$asset_file   = substr( $path, 0, -3 ) . '.asset.php';
+		$asset        = file_exists( $asset_file )
+			? require_once( $asset_file )
+			: null;
+		$dependencies = isset( $asset['dependencies'] ) ? $asset['dependencies'] : array();
+		$version      = isset( $asset['version'] ) ? $asset['version'] : filemtime( $path );
 
 		// Add dependencies that cannot be detected and generated by build tools.
 		switch ( $handle ) {
@@ -212,7 +213,7 @@ function gutenberg_register_packages_scripts() {
 			$handle,
 			gutenberg_url( $gutenberg_path ),
 			$dependencies,
-			filemtime( $path ),
+			$version,
 			true
 		);
 	}

packages/dependency-extraction-webpack-plugin/CHANGELOG.md

@@ -1,3 +1,9 @@
+## Master
+
+### Breaking Changes
+
+- The plugin now adds, for each entry point, an asset file saved by default in PHP format that declares an object with the list of WordPress script dependencies for the entry point ([#17298](https://github.com/WordPress/gutenberg/pull/17298)). There is also an option to use JSON as the output format. The shape of metadata is also different from the previous version. Read more in the [README](./README.md) file.
+
 ## 1.0.1 (2019-05-22)
 
 ### Bug Fixes

packages/dependency-extraction-webpack-plugin/README.md

@@ -3,8 +3,8 @@
 This webpack plugin serves two purposes:
 
 - Externalize dependencies that are available as script dependencies on modern WordPress sites.
-- Add a JSON file for each entrypoint that declares the WordPress script dependencies for the
-  entrypoint.
+- Add an asset file for each entry point that declares an object with the list of WordPress script dependencies for the
+  entry point. The asset file also contains the current version calculated for the current source code.
 
 This allows JavaScript bundles produced by webpack to leverage WordPress style dependency sharing
 without an error-prone process of manually maintaining a dependency list.
@@ -59,7 +59,7 @@ const config = {
 };
 ```
 
-Each entrypoint in the webpack bundle will include JSON file that declares the WordPress script dependencies that should be enqueued.
+Each entry point in the webpack bundle will include an asset file that declares the WordPress script dependencies that should be enqueued. Such file also contains the unique version hash calculated based on the file content.
 
 For example:
 
@@ -70,8 +70,8 @@ import { Component } from '@wordpress/element';
 // Webpack will produce the output output/entrypoint.js
 /* bundled JavaScript output */
 
-// Webpack will also produce output/entrypoint.deps.json declaring script dependencies
-['wp-element']
+// Webpack will also produce output/entrypoint.asset.php declaring script dependencies
+<?php return array('dependencies' => array('wp-element'), 'version' => 'dd4c2dc50d046ed9d4c063a7ca95702f');
 ```
 
 By default, the following module requests are handled:
@@ -109,6 +109,13 @@ module.exports = {
 }
 ```
 
+##### `outputFormat`
+
+- Type: string
+- Default: `php`
+
+The output format for the generated asset file. There are two options available: 'php' or 'json'.
+
 ##### `useDefaults`
 
 - Type: boolean
@@ -121,8 +128,8 @@ Pass `useDefaults: false` to disable the default request handling.
 - Type: boolean
 - Default: `false`
 
-Force `wp-polyfill` to be included in each entrypoint's dependency list. This would be the same as
-adding `import '@wordpress/polyfill';` to each entrypoint.
+Force `wp-polyfill` to be included in each entry point's dependency list. This would be the same as
+adding `import '@wordpress/polyfill';` to each entry point.
 
 ##### `requestToExternal`
 
@@ -200,20 +207,20 @@ module.exports = {
 The functions `requestToExternal` and `requestToHandle` allow this module to handle arbitrary
 modules. `requestToExternal` is necessary to handle any module and maps a module request to a global
 name. `requestToHandle` maps the same module request to a script handle, the strings that will be
-included in the `entrypoint.deps.json` files.
+included in the `entrypoint.asset.php` files.
 
 ### WordPress
 
 Enqueue your script as usual and read the script dependencies dynamically:
 
 ```php
-$script_path         = 'path/to/script.js';
-$script_deps_path    = 'path/to/script.deps.json';
-$script_dependencies = file_exists( $script_deps_path )
-	? json_decode( file_get_contents( $script_deps_path ) )
-	: array();
+$script_path       = 'path/to/script.js';
+$script_asset_path = 'path/to/script.asset.php';
+$script_asset      = file_exists( $script_asset_path )
+	? require( $script_asset_path ) )
+	: array( 'dependencies' => array(), 'version' => filemtime( $script_path ) );
 $script_url = plugins_url( $script_path, __FILE__ );
-wp_enqueue_script( 'script', $script_url, $script_dependencies );
+wp_enqueue_script( 'script', $script_url, $script_asset['dependencies'], $script_asset['version'] );
 ```
 
 <br/><br/><p align="center"><img src="https://s.w.org/style/images/codeispoetry.png?1" alt="Code is Poetry." /></p>

packages/dependency-extraction-webpack-plugin/index.js

@@ -2,6 +2,7 @@
  * External dependencies
  */
 const { createHash } = require( 'crypto' );
+const json2php = require( 'json2php' );
 const { ExternalsPlugin } = require( 'webpack' );
 const { RawSource } = require( 'webpack-sources' );
 
@@ -15,6 +16,7 @@ class DependencyExtractionWebpackPlugin {
 		this.options = Object.assign(
 			{
 				injectPolyfill: false,
+				outputFormat: 'php',
 				useDefaults: true,
 			},
 			options
@@ -74,17 +76,27 @@ class DependencyExtractionWebpackPlugin {
 		return request;
 	}
 
+	stringify( asset ) {
+		if ( this.options.outputFormat === 'php' ) {
+			return `<?php return ${ json2php( JSON.parse( JSON.stringify( asset ) ) ) };`;
+		}
+
+		return JSON.stringify( asset );
+	}
+
 	apply( compiler ) {
 		this.externalsPlugin.apply( compiler );
 
 		const { output } = compiler.options;
 		const { filename: outputFilename } = output;
 
 		compiler.hooks.emit.tap( this.constructor.name, ( compilation ) => {
-			// Process each entrypoint independently.
+			const { injectPolyfill, outputFormat } = this.options;
+
+			// Process each entry point independently.
 			for ( const [ entrypointName, entrypoint ] of compilation.entrypoints.entries() ) {
 				const entrypointExternalizedWpDeps = new Set();
-				if ( this.options.injectPolyfill ) {
+				if ( injectPolyfill ) {
 					entrypointExternalizedWpDeps.add( 'wp-polyfill' );
 				}
 
@@ -98,27 +110,31 @@ class DependencyExtractionWebpackPlugin {
 					}
 				}
 
-				// Build a stable JSON string that declares the WordPress script dependencies.
-				const sortedDepsArray = Array.from( entrypointExternalizedWpDeps ).sort();
-				const depsString = JSON.stringify( sortedDepsArray );
+				const runtimeChunk = entrypoint.getRuntimeChunk();
+
+				// Get a stable, stringified representation of the WordPress script asset.
+				const assetString = this.stringify( {
+					dependencies: Array.from( entrypointExternalizedWpDeps ).sort(),
+					version: runtimeChunk.hash,
+				} );
 
-				// Determine a filename for the `[entrypoint].deps.json` file.
+				// Determine a filename for the asset file.
 				const [ filename, query ] = entrypointName.split( '?', 2 );
-				const depsFilename = compilation
+				const assetFilename = compilation
 					.getPath( outputFilename, {
-						chunk: entrypoint.getRuntimeChunk(),
+						chunk: runtimeChunk,
 						filename,
 						query,
 						basename: basename( filename ),
 						contentHash: createHash( 'md4' )
-							.update( depsString )
+							.update( assetString )
 							.digest( 'hex' ),
 					} )
-					.replace( /\.js$/i, '.deps.json' );
+					.replace( /\.js$/i, '.asset.' + ( outputFormat === 'php' ? 'php' : 'json' ) );
 
 				// Add source and file into compilation for webpack to output.
-				compilation.assets[ depsFilename ] = new RawSource( depsString );
-				entrypoint.getRuntimeChunk().files.push( depsFilename );
+				compilation.assets[ assetFilename ] = new RawSource( assetString );
+				runtimeChunk.files.push( assetFilename );
 			}
 		} );
 	}

packages/dependency-extraction-webpack-plugin/package.json

@@ -24,6 +24,7 @@
 	],
 	"main": "index.js",
 	"dependencies": {
+		"json2php": "^0.0.4",
 		"webpack": "^4.8.3",
 		"webpack-sources": "^1.3.0"
 	},

packages/dependency-extraction-webpack-plugin/test/__snapshots__/build.js.snap

@@ -1,11 +1,6 @@
 // Jest Snapshot v1, https://goo.gl/fbAQLP
 
-exports[`Webpack \`dynamic-import\` should produce expected output: Dependencies JSON should match snapshot 1`] = `
-Array [
-  "lodash",
-  "wp-blob",
-]
-`;
+exports[`Webpack \`dynamic-import\` should produce expected output: Asset file should match snapshot 1`] = `"<?php return array('dependencies' => array('lodash', 'wp-blob'), 'version' => '3bafd7358340331fd0c5c21f3f44f76e');"`;
 
 exports[`Webpack \`dynamic-import\` should produce expected output: External modules should match snapshot 1`] = `
 Array [
@@ -29,12 +24,7 @@ Array [
 ]
 `;
 
-exports[`Webpack \`function-output-filename\` should produce expected output: Dependencies JSON should match snapshot 1`] = `
-Array [
-  "lodash",
-  "wp-blob",
-]
-`;
+exports[`Webpack \`function-output-filename\` should produce expected output: Asset file should match snapshot 1`] = `"<?php return array('dependencies' => array('lodash', 'wp-blob'), 'version' => '63cfdf72e22844c4e01642544c75a571');"`;
 
 exports[`Webpack \`function-output-filename\` should produce expected output: External modules should match snapshot 1`] = `
 Array [
@@ -58,22 +48,30 @@ Array [
 ]
 `;
 
-exports[`Webpack \`no-default\` should produce expected output: Dependencies JSON should match snapshot 1`] = `Array []`;
+exports[`Webpack \`no-default\` should produce expected output: Asset file should match snapshot 1`] = `"<?php return array('dependencies' => array(), 'version' => '4def21ff1ab3d47e29e1aacf8bd2a1f4');"`;
 
 exports[`Webpack \`no-default\` should produce expected output: External modules should match snapshot 1`] = `Array []`;
 
-exports[`Webpack \`no-deps\` should produce expected output: Dependencies JSON should match snapshot 1`] = `Array []`;
+exports[`Webpack \`no-deps\` should produce expected output: Asset file should match snapshot 1`] = `"<?php return array('dependencies' => array(), 'version' => '1ca5da751210847d1fe4bd5831ffd29a');"`;
 
 exports[`Webpack \`no-deps\` should produce expected output: External modules should match snapshot 1`] = `Array []`;
 
-exports[`Webpack \`overrides\` should produce expected output: Dependencies JSON should match snapshot 1`] = `
+exports[`Webpack \`output-format-json\` should produce expected output: Asset file should match snapshot 1`] = `"{\\"dependencies\\":[\\"lodash\\"],\\"version\\":\\"24a5cf4a92b121c1709114ecf54db6b6\\"}"`;
+
+exports[`Webpack \`output-format-json\` should produce expected output: External modules should match snapshot 1`] = `
 Array [
-  "wp-blob",
-  "wp-script-handle-for-rxjs",
-  "wp-url",
+  Object {
+    "externalType": "this",
+    "request": Object {
+      "this": "lodash",
+    },
+    "userRequest": "lodash",
+  },
 ]
 `;
 
+exports[`Webpack \`overrides\` should produce expected output: Asset file should match snapshot 1`] = `"<?php return array('dependencies' => array('wp-blob', 'wp-script-handle-for-rxjs', 'wp-url'), 'version' => '8db429b7f1e39c290ed16684ca6819c6');"`;
+
 exports[`Webpack \`overrides\` should produce expected output: External modules should match snapshot 1`] = `
 Array [
   Object {
@@ -116,11 +114,7 @@ Array [
 ]
 `;
 
-exports[`Webpack \`with-externs\` should produce expected output: Dependencies JSON should match snapshot 1`] = `
-Array [
-  "wp-url",
-]
-`;
+exports[`Webpack \`with-externs\` should produce expected output: Asset file should match snapshot 1`] = `"<?php return array('dependencies' => array('wp-url'), 'version' => 'dd4c2dc50d046ed9d4c063a7ca95702f');"`;
 
 exports[`Webpack \`with-externs\` should produce expected output: External modules should match snapshot 1`] = `
 Array [
@@ -152,12 +146,7 @@ Array [
 ]
 `;
 
-exports[`Webpack \`wordpress\` should produce expected output: Dependencies JSON should match snapshot 1`] = `
-Array [
-  "lodash",
-  "wp-blob",
-]
-`;
+exports[`Webpack \`wordpress\` should produce expected output: Asset file should match snapshot 1`] = `"<?php return array('dependencies' => array('lodash', 'wp-blob'), 'version' => '14aca98920139a73ab66800dda496e79');"`;
 
 exports[`Webpack \`wordpress\` should produce expected output: External modules should match snapshot 1`] = `
 Array [
@@ -181,12 +170,7 @@ Array [
 ]
 `;
 
-exports[`Webpack \`wordpress-require\` should produce expected output: Dependencies JSON should match snapshot 1`] = `
-Array [
-  "lodash",
-  "wp-blob",
-]
-`;
+exports[`Webpack \`wordpress-require\` should produce expected output: Asset file should match snapshot 1`] = `"<?php return array('dependencies' => array('lodash', 'wp-blob'), 'version' => '34be485f04528be9581e8a782e8ceb8c');"`;
 
 exports[`Webpack \`wordpress-require\` should produce expected output: External modules should match snapshot 1`] = `
 Array [

packages/dependency-extraction-webpack-plugin/test/build.js

@@ -46,19 +46,19 @@ describe.each( configFixtures )( 'Webpack `%s`', ( configCase ) => {
 			webpack( options, ( err, stats ) => {
 				expect( err ).toBeNull();
 
-				const depsFiles = glob( `${ outputDirectory }/*.deps.json` );
+				const assetFiles = glob( `${ outputDirectory }/*.asset.@(json|php)` );
 				const expectedLength =
 					typeof options.entry === 'object' ? Object.keys( options.entry ).length : 1;
-				expect( depsFiles ).toHaveLength( expectedLength );
+				expect( assetFiles ).toHaveLength( expectedLength );
 
-				// Deps files should match
-				depsFiles.forEach( ( depsFile ) => {
-					expect( require( depsFile ) ).toMatchSnapshot(
-						'Dependencies JSON should match snapshot'
+				// Asset files should match.
+				assetFiles.forEach( ( assetFile ) => {
+					expect( fs.readFileSync( assetFile, 'utf-8' ) ).toMatchSnapshot(
+						'Asset file should match snapshot'
 					);
 				} );
 
-				// Webpack stats external modules should match
+				// Webpack stats external modules should match.
 				const externalModules = stats.compilation.modules
 					.filter( ( { external } ) => external )
 					.sort()

packages/dependency-extraction-webpack-plugin/test/fixtures/output-format-json/index.js

@@ -0,0 +1,6 @@
+/**
+ * External dependencies
+ */
+import _ from 'lodash';
+
+_.map( [], _.identity );

packages/dependency-extraction-webpack-plugin/test/fixtures/output-format-json/webpack.config.js

@@ -0,0 +1,5 @@
+const DependencyExtractionWebpackPlugin = require( '../../..' );
+
+module.exports = {
+	plugins: [ new DependencyExtractionWebpackPlugin( { outputFormat: 'json' } ) ],
+};

packages/scripts/CHANGELOG.md

@@ -1,5 +1,9 @@
 ## Master
 
+### Breaking Changes
+
+- The bundled `@wordpress/dependency-extraction-webpack-plugin` dependency has been updated to the next major version `^2.0.0`. `start` and `build` scripts save now the generated asset file for each entry point in the new PHP output format.
+
 ## 4.1.0 (2019-09-03)
 
 ### New Features