SDK Modernization (#585)

# Building
Webpack 5 support is here!

 - the node package is built using Babel via `yarn build:node`
 - the browser bundle is built using Webpack via `yarn build:browser`
   * this is configured in `cfg/webpack.config.browser.js`
   * its accessible after including the script via the `StellarBase` variable
   * it adds a handful of important polyfills
   * it does linting via ESLint

# Linting
ESLint, Prettier, and DTSLint are all updated:

 - ESLint works with Babel via the `@babel/eslint-parser` in each `.eslintrc.js` file and is implicitly part of building via Webpack (see above)
 - DTSLint is part of the `@definitelytyped` project and has additional configuration implicitly as part of the `types/.eslintrc.js` file which uses `@typescript-eslint/parser`, instead
 - Prettier is configured in `cfg/prettier.config.js` (and `cfg/.prettierignore`) and runs on all files now. It's also (still) a pre-commit hook.

Just run `yarn pretty && yarn lint` to format your files and check that everything is up to par.

# Testing
**Istanbul**: Istanbul is [unmaintained](https://www.npmjs.com/package/istanbul) and superceded by the `nyc` package.

 - the `istanbul` plugin in `.babelrc` tells Babel about it
 - `cfg/.nycrc` extends a Babel-compatible configuration file ([docs](https://github.com/istanbuljs/nyc#babel-projects))
 - the `nyc` key in `package.json` configures instrumentation behavior
 - the coverage is *actually* provided by `babel-plugin-istanbul` ([docs](https://github.com/istanbuljs/babel-plugin-istanbul)), which is why `"instrument": false` in the above

**Node Testing**: Mocha (and its helper friends Sinon and Chai) has been updated.

 - the `mocha` key in `package.json` configures Mocha (replacing `mocha.opts`)
 - this does Babel transpiling and hooks in the helper script
 - code coverage is provided by `nyc` (see above)

**Browser Testing**: Karma has been updated.

 - the SauceLabs Karma configuration has been removed entirely because it wasn't being used
 - Karma has been updated with a code coverage integration with `nyc` ([docs](https://github.com/istanbuljs/babel-plugin-istanbul#karma), see above)
 - It relies on the Webpack configuration but makes some minor changes

Run `yarn test` to run both suites.

# Scripts
Development has a slightly different workflow now:

 - build+test locally with `yarn test:node`
 - this will only run node tests (which are faster/simpler)
 - you can `yarn build && yarn test:all` to get the both test suites (note that building first is *required*)
 - run `yarn preversion` when you want a thorough, clean test of the whole system in "production" mode

.babelrc

@@ -1,9 +1,12 @@
 {
   "presets": [
-    ["env", {
-      "targets": {
-        "ie": "11"
+    "@babel/preset-env"
+  ],
+  "env": {
+      "development": {
+          "plugins": [
+              "istanbul"
+          ]
       }
-    }]
-  ]
+  }
 }

.github/workflows/gh_pages.yml

@@ -20,8 +20,8 @@ jobs:
       - name: Install Dependencies
         run: yarn install
 
-      - name: Build
-        run: gulp
+      - name: Build & Test
+        run: yarn preversion
 
       - name: Checkout GH pages
         uses: actions/checkout@v2

.github/workflows/npm_publish.yml

@@ -19,8 +19,8 @@ jobs:
       - name: Install Depencencies
         run: yarn
 
-      - name: Build
-        run: gulp
+      - name: Test & Build
+        run: yarn preversion
 
       - name: Publish npm package
         run: yarn publish

.github/workflows/tests.yml

@@ -13,7 +13,7 @@ jobs:
       fail-fast: false
       max-parallel: 2
       matrix:
-        node-version: [14, 16]
+        node-version: [14, 16, 18]
 
     steps:
     - uses: actions/checkout@v2
@@ -26,14 +26,14 @@ jobs:
     - name: Install Dependencies
       run: yarn install
 
-    - name: Build
-      run: gulp
+    - name: Build All
+      run: yarn build:prod
 
     - name: Run Node Tests
-      run: yarn test
+      run: yarn test:node
 
     - name: Run Browser Tests
-      run: gulp test:browser
+      run: yarn build:browser:prod && yarn test:browser
 
     - name: Run Linter
-      run: yarn dtslint
+      run: yarn lint

.gitignore

@@ -1,7 +1,9 @@
+.nvmrc
+.nyc_output/
 /node_modules/
 /tmp/
-/lib/
 /dist/
+/config/dist/
 /coverage/
 /jsdoc/
 .DS_Store

CONTRIBUTING.md

@@ -7,7 +7,7 @@ Then please [sign the Contributor License Agreement](https://docs.google.com/for
 
 
 # Releasing
-SDK maintainers should follow [semantic versioning](https://semver.org/) best practices for releasing the SDK. 
+SDK maintainers should follow [semantic versioning](https://semver.org/) best practices for releasing the SDK.
 
 Use your best judgement when deciding on when to craft a release: maybe enough changes have accumulated to warrant a release, maybe there's a high-urgency fix that needs to be published immediately, or maybe you can put it off for a particular feature. It's all dependent on what else is going on.
 
@@ -22,9 +22,9 @@ Once all of the PRs for a particular release are in, it's time to actually publi
 
  - [ ] Ensure that all of the PRs in this delta are accurately reflected in the [CHANGELOG](./CHANGELOG.md), broken down by impact and linking to the corresponding PRs. Update the file if necessary.
 
- - [ ] Update the top-level `"version"` field in the [package.json](./package.json) file to reflect the new version.
+ - [ ] Run `yarn preversion` to build and test an optimized bundle and ensure Typescript compatibility (one of the most common sources of bugs, since this library is written purely in JS but must be usable from TS).
 
- - [ ] Run the final sanity check to ensure the builds pass: `yarn dtslint && yarn test && yarn preversion`. The first command checks that you have Typescript compatibility (one of the most common sources of bugs, since this library is written purely in JS but must be usable from TS).
+ - [ ] Run `yarn version` to update the version number in the package.json (or modify the `"version"` field manually).
 
  - [ ] Commit & push your branch, then [create a PR](https://github.com/stellar/js-stellar-base/compare).
 

Makefile

@@ -1,4 +1,4 @@
-XDR_BASE_URL_CURR=https://github.com/stellar/stellar-core/raw/master/src/protocol-curr/xdr
+XDR_BASE_URL_CURR=https://github.com/stellar/stellar-xdr/raw/curr
 XDR_BASE_LOCAL_CURR=xdr/curr
 XDR_FILES_CURR= \
 	Stellar-SCP.x \
@@ -9,7 +9,7 @@ XDR_FILES_CURR= \
 	Stellar-types.x
 XDR_FILES_LOCAL_CURR=$(addprefix xdr/curr/,$(XDR_FILES_CURR))
 
-XDR_BASE_URL_NEXT=https://github.com/stellar/stellar-xdr-next/raw/main
+XDR_BASE_URL_NEXT=https://github.com/stellar/stellar-xdr/raw/next
 XDR_BASE_LOCAL_NEXT=xdr/next
 XDR_FILES_NEXT= \
 	Stellar-SCP.x \

README.md

@@ -14,9 +14,9 @@ implementation in JavaScript that can be used on either Node.js or web browsers.
 
 > **Warning!** The Node version of this package uses the [`sodium-native`](https://www.npmjs.com/package/sodium-native) package, a native implementation of [Ed25519](https://ed25519.cr.yp.to/) in Node.js, as an [optional dependency](https://docs.npmjs.com/files/package.json#optionaldependencies).
 > This means that if for any reason installation of this package fails, `stellar-base` will fallback to the much slower implementation contained in [`tweetnacl`](https://www.npmjs.com/package/tweetnacl).
-> 
+>
 > If you'd explicitly prefer **not** to install the `sodium-native` package, pass the appropriate flag to skip optional dependencies when installing this package (e.g. `--no-optional` if using `npm install` or `--without-optional` using `yarn install`).
-> 
+>
 > If you are using `stellar-base` in a browser you can ignore this. However, for production backend deployments you should most likely be using `sodium-native`.
 > If `sodium-native` is successfully installed and working,
 > `StellarBase.FastSigning` variable will be equal `true`. Otherwise it will be
@@ -102,11 +102,11 @@ Make sure that you are using the latest version number. They can be found on the
 
 We support the oldest LTS release of Node, which is [currently 14.x](https://nodejs.org/en/about/releases/). Please likewise install and develop on Node 14 so you don't get surprised when your code works locally but breaks in CI.
 
-If you work on several projects that use different Node versions, you might find helpful to install a nodejs version manager.
+If you work on several projects that use different Node versions, you might find helpful to install a NodeJS version manager:
 
-- https://github.com/creationix/nvm
-- https://github.com/wbyoung/avn
-- https://github.com/asdf-vm/asdf
+  - https://github.com/creationix/nvm
+  - https://github.com/wbyoung/avn
+  - https://github.com/asdf-vm/asdf
 
 2. Install Yarn
 
@@ -127,17 +127,22 @@ yarn
 
 5. Observe the project's code style
 
-While you're making changes, make sure to run the linter-watcher to catch any
+While you're making changes, make sure to regularly run the linter to catch any
 linting errors (in addition to making sure your text editor supports ESLint)
 
 ```shell
-node_modules/.bin/gulp watch
+yarn lint
+```
+
+as well as fixing any formatting errors with
+
+```shell
+yarn fmt
 ```
 
 If you're working on a file not in `src`, limit your code to Node 6.16 ES! See
-what's supported here: https://node.green/ (The reason is that our npm library
-must support earlier versions of Node, so the tests need to run on those
-versions.)
+what's supported here: https://node.green/. (Our npm library must support
+earlier versions of Node, so the tests need to run on those versions.)
 
 #### Updating XDR definitions
 
@@ -154,18 +159,16 @@ For information on how to use js-stellar-base, take a look at the docs in the
 To run all tests:
 
 ```shell
-gulp test
+yarn test
 ```
 
 To run a specific set of tests:
 
 ```shell
-gulp test:node
-gulp test:browser
+yarn test:node
+yarn test:browser
 ```
 
-You can also run `yarn test` for a simpler subset of the test cases.
-
 Tests are also run automatically in Github Actions for every master commit and
 pull request.
 

src/.eslintrc.js → config/.eslintrc.js

@@ -3,7 +3,8 @@ module.exports = {
     es6: true
   },
   extends: ['airbnb-base', 'prettier'],
-  plugins: ['prettier', 'prefer-import'],
+  plugins: ['@babel', 'prettier', 'prefer-import'],
+  parser: '@babel/eslint-parser',
   rules: {
     // OFF
     'import/prefer-default-export': 0,
@@ -37,6 +38,5 @@ module.exports = {
 
     // ERROR
     'no-unused-expressions': [2, { allowTaggedTemplates: true }]
-  },
-  parser: 'babel-eslint'
+  }
 };

.jsdoc.json → config/.jsdoc.json

@@ -7,9 +7,7 @@
     "destination": "./jsdoc/",
     "recurse": true,
     "template": "node_modules/minami",
-    "readme": "./README.md"
+    "readme": "README.md"
   },
-  "plugins": [
-    "plugins/markdown"
-  ]
+  "plugins": ["plugins/markdown"]
 }

config/.nycrc

@@ -0,0 +1,3 @@
+{
+    "extends": "@istanbuljs/nyc-config-babel"
+}

config/.prettierignore

@@ -0,0 +1,7 @@
+../package.json
+../node_modules
+../lib
+../dist
+../src/generated
+../src/vendor
+../docs

config/karma.conf.js

@@ -0,0 +1,37 @@
+const webpackConfig = require('./webpack.config.browser.js');
+
+delete webpackConfig.output;
+webpackConfig.entry = {}; // karma fills these in
+webpackConfig.plugins.shift(); // drop eslinter plugin
+
+module.exports = function (config) {
+  config.set({
+    frameworks: ['mocha', 'sinon-chai'],
+    browsers: ['FirefoxHeadless', 'ChromeHeadless'],
+
+    files: [
+      '../dist/stellar-base.js', // webpack should build this first
+      '../test/unit/**/*.js'
+    ],
+
+    preprocessors: {
+      '../test/unit/**/*.js': ['webpack']
+    },
+
+    webpack: webpackConfig,
+    webpackMiddleware: {
+      noInfo: true
+    },
+
+    colors: true,
+    singleRun: true,
+
+    reporters: ['dots', 'coverage'],
+    coverageReporter: {
+      type: 'text-summary',
+      instrumenterOptions: {
+        istanbul: { noCompact: true }
+      }
+    }
+  });
+};

prettier.config.js → config/prettier.config.js

@@ -1,12 +1,13 @@
 module.exports = {
   arrowParens: 'always',
   bracketSpacing: true,
-  jsxBracketSameLine: false,
+  bracketSameLine: false,
   printWidth: 80,
   proseWrap: 'always',
   semi: true,
   singleQuote: true,
   tabWidth: 2,
+  parser: 'babel',
   trailingComma: 'none',
   useTabs: false
 };

config/webpack.config.browser.js

@@ -0,0 +1,75 @@
+var path = require('path');
+var webpack = require('webpack');
+
+var ESLintPlugin = require('eslint-webpack-plugin');
+var TerserPlugin = require('terser-webpack-plugin');
+var NodePolyfillPlugin = require('node-polyfill-webpack-plugin');
+
+const config = {
+  target: 'web',
+  // https://stackoverflow.com/a/34018909
+  entry: {
+    'stellar-base': path.resolve(__dirname, '../src/index.js'),
+    'stellar-base.min': path.resolve(__dirname, '../src/index.js')
+  },
+  resolve: {
+    fallback: {
+      crypto: require.resolve('crypto-browserify'),
+      stream: require.resolve('stream-browserify'),
+      buffer: require.resolve('buffer')
+    },
+    extensions: ['.ts', '.js']
+  },
+  output: {
+    clean: true,
+    library: 'StellarBase',
+    compareBeforeEmit: true,
+    path: path.resolve(__dirname, '../dist')
+  },
+  mode: process.env.NODE_ENV ?? 'development',
+  devtool: process.env.NODE_ENV === 'production' ? false : 'inline-source-map',
+  module: {
+    rules: [
+      {
+        test: /\.m?js$/,
+        exclude: /node_modules/,
+        use: {
+          loader: 'babel-loader',
+          options: {
+            cacheDirectory: true
+          }
+        }
+      }
+    ]
+  },
+  optimization: {
+    minimize: true,
+    minimizer: [
+      new TerserPlugin({
+        include: /\.min\.js$/,
+        terserOptions: {
+          format: {
+            ascii_only: true
+          }
+        }
+      })
+    ]
+  },
+  plugins: [
+    // this must be first for karma to work (see line 5 of karma.conf.js)
+    new ESLintPlugin({
+      overrideConfigFile: path.resolve(__dirname, './.eslintrc.js')
+    }),
+    // Ignore native modules (sodium-native)
+    new webpack.IgnorePlugin({ resourceRegExp: /sodium-native/ }),
+    new NodePolyfillPlugin(),
+    new webpack.ProvidePlugin({
+      Buffer: ['buffer', 'Buffer']
+    })
+  ],
+  watchOptions: {
+    ignored: /(node_modules|coverage|lib|dist)/
+  }
+};
+
+module.exports = config;