Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Tinymath with math.js #4492

Merged
merged 15 commits into from
Sep 12, 2023
Merged
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- [Console] Migrate `/lib/autocomplete/` module to TypeScript ([#4148](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/4148))
- [Dashboard] Restructure the `Dashboard` plugin folder to be more cohesive with the project ([#4575](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/4575))
- [Chrome] Remove breadcrumb style overrrides ([#4621](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/4621))
- Replace tinymath with math.js ([#4492](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/4492))

### 🔩 Tests

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,7 @@
"json-stringify-safe": "5.0.1",
"lodash": "^4.17.21",
"lru-cache": "^4.1.5",
"mathjs": "^11.8.2",
"minimatch": "^3.0.4",
"moment": "^2.24.0",
"moment-timezone": "^0.5.27",
Expand All @@ -219,7 +220,6 @@
"symbol-observable": "^1.2.0",
"tar": "^6.1.11",
"tinygradient": "^1.1.5",
"tinymath": "1.2.1",
"tslib": "^2.0.0",
"type-detect": "^4.0.8",
"uuid": "3.3.2",
Expand Down
1 change: 0 additions & 1 deletion packages/osd-optimizer/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@
"rxjs": "^6.5.5",
"source-map-support": "^0.5.19",
"terser-webpack-plugin": "^2.1.2",
"tinymath": "1.2.1",
"watchpack": "^2.1.1",
"webpack-merge": "^4.2.2"
},
Expand Down
1 change: 0 additions & 1 deletion packages/osd-optimizer/src/worker/webpack.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,6 @@ export function getWebpackConfig(bundle: Bundle, bundleRefs: BundleRefs, worker:
extensions: ['.js', '.ts', '.tsx', '.json'],
mainFields: ['browser', 'main'],
alias: {
tinymath: require.resolve('tinymath/lib/tinymath.es5.js'),
core_app_image_assets: Path.resolve(worker.repoRoot, 'src/core/public/core_app/images'),
},
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,12 +134,12 @@ export function MathAgg(props) {
values={{
link: (
<EuiLink
href="https://github.com/elastic/tinymath/blob/master/docs/functions.md"
href="https://mathjs.org/docs/reference/functions.html"
target="_blank"
>
<FormattedMessage
id="visTypeTimeseries.math.expressionDescription.tinyMathLinkText"
defaultMessage="TinyMath"
id="visTypeTimeseries.math.expressionDescription.linkText"
defaultMessage="Math.js"
/>
</EuiLink>
),
Expand Down
Copy link
Member

@abbyhu2000 abbyhu2000 Jul 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we disable the following math.js functions due to security issues? How do you know these are the only functions that we need to disable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These functions are marked as unsafe by the mathjs documentation https://mathjs.org/docs/expressions/security.html

A user could try to inject malicious JavaScript code via the expression parser. The expression parser of mathjs offers a sandboxed environment to execute expressions which should make this impossible. It’s possible though that there are unknown security vulnerabilities, so it’s important to be careful, especially when allowing server side execution of arbitrary expressions.

The expression parser of mathjs parses the input in a controlled way into an expression tree or abstract syntax tree (AST). In a “compile” step, it does as much as possible preprocessing on the static parts of the expression, and creates a fast performing function which can be used to evaluate the expression repeatedly using a dynamically passed scope.

The parser actively prevents access to JavaScripts internal eval and new Function which are the main cause of security attacks. Mathjs versions 4 and newer does not use JavaScript’s eval under the hood.

Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

import { create, all } from 'mathjs';

const math = create(all);
export const evaluate = math.evaluate;
export const MATH_THROW_MSG = 'Math.js function is disabled';

math.import(
{
import() {
throw new Error(MATH_THROW_MSG);
},
createUnit() {
throw new Error(MATH_THROW_MSG);
},
evaluate() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this evaluate() function different than math.evaluate() from line 9?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is the same evaluate function. The idea is to disable risky functions on the user side. We can find a list of all risky functions in the mathjs documentation: https://mathjs.org/docs/expressions/security.html.

So it is possible to do: mathjs.evaluate('add(2, 2)').
At the same time, it is impossible to evaluate inside the evaluate function itself: mathjs.evaluate('evaluate("add(2,2)")').

throw new Error(MATH_THROW_MSG);
},
parse() {
throw new Error(MATH_THROW_MSG);
},
simplify() {
throw new Error(MATH_THROW_MSG);
},
derivative() {
throw new Error(MATH_THROW_MSG);
},
},
{ override: true }
);
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ import { getDefaultDecoration } from '../../helpers/get_default_decoration';
import { getSiblingAggValue } from '../../helpers/get_sibling_agg_value';
import { getSplits } from '../../helpers/get_splits';
import { mapBucket } from '../../helpers/map_bucket';
import { evaluate } from 'tinymath';
import { evaluate } from './evaluate';

export function mathAgg(resp, panel, series, meta) {
return (next) => (results) => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@

import { mathAgg } from './math';
import { stdMetric } from './std_metric';
import { MATH_THROW_MSG } from './evaluate';

describe('math(resp, panel, series)', () => {
let panel;
Expand Down Expand Up @@ -145,7 +146,7 @@ describe('math(resp, panel, series)', () => {
);
});

test('throws on actual tinymath expression errors', () => {
test('throws on actual math.js expression errors', () => {
series.metrics[2].script = 'notExistingFn(params.a)';
expect(() =>
stdMetric(resp, panel, series)(mathAgg(resp, panel, series)((results) => results))([])
Expand All @@ -156,4 +157,52 @@ describe('math(resp, panel, series)', () => {
stdMetric(resp, panel, series)(mathAgg(resp, panel, series)((results) => results))([])
).toThrow();
});

test('throws on math.js disabled function: import', () => {
series.metrics[2].script = 'import({ pi: 3.14 }, { override: true })';

expect(() =>
stdMetric(resp, panel, series)(mathAgg(resp, panel, series)((results) => results))([])
).toThrow(MATH_THROW_MSG);
});

test('throws on math.js disabled function: createUnit', () => {
series.metrics[2].script = 'createUnit("foo")';

expect(() =>
stdMetric(resp, panel, series)(mathAgg(resp, panel, series)((results) => results))([])
).toThrow(MATH_THROW_MSG);
});

test('throws on math.js disabled function: evaluate', () => {
series.metrics[2].script = 'evaluate("(2+3)/4") ';

expect(() =>
stdMetric(resp, panel, series)(mathAgg(resp, panel, series)((results) => results))([])
).toThrow(MATH_THROW_MSG);
});

test('throws on math.js disabled function: parse', () => {
series.metrics[2].script = '[h = parse("x^2 + x")]';

expect(() =>
stdMetric(resp, panel, series)(mathAgg(resp, panel, series)((results) => results))([])
).toThrow(MATH_THROW_MSG);
});

test('throws on math.js disabled function: simplify', () => {
series.metrics[2].script = 'simplify("2 * 1 * x ^ (2 - 1)")';

expect(() =>
stdMetric(resp, panel, series)(mathAgg(resp, panel, series)((results) => results))([])
).toThrow(MATH_THROW_MSG);
});

test('throws on math.js disabled function: derivative', () => {
series.metrics[2].script = 'derivative("x^2", "x")';

expect(() =>
stdMetric(resp, panel, series)(mathAgg(resp, panel, series)((results) => results))([])
).toThrow(MATH_THROW_MSG);
});
});
67 changes: 62 additions & 5 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -1154,6 +1154,13 @@
dependencies:
regenerator-runtime "^0.13.4"

"@babel/runtime@^7.22.5":
version "7.22.5"
resolved "https://registry.yarnpkg.com/@babel/runtime/-/runtime-7.22.5.tgz#8564dd588182ce0047d55d7a75e93921107b57ec"
integrity sha512-ecjvYlnAaZ/KVneE/OdKYBYfgXV3Ptu6zQWmgEF7vwKhQnvVS6bjMD2XYgj+SNvQ1GfK/pjgokfPkC/2CO8CuA==
dependencies:
regenerator-runtime "^0.13.11"

"@babel/template@^7.16.7", "@babel/template@^7.3.3":
version "7.16.7"
resolved "https://registry.yarnpkg.com/@babel/template/-/template-7.16.7.tgz#8d126c8701fde4d66b264b3eba3d96f07666d155"
Expand Down Expand Up @@ -6145,6 +6152,11 @@ compare-versions@^5.0.1:
resolved "https://registry.yarnpkg.com/compare-versions/-/compare-versions-5.0.1.tgz#14c6008436d994c3787aba38d4087fabe858555e"
integrity sha512-v8Au3l0b+Nwkp4G142JcgJFh1/TUhdxut7wzD1Nq1dyp5oa3tXaqb03EXOAB6jS4gMlalkjAUPZBMiAfKUixHQ==

complex.js@^2.1.1:
version "2.1.1"
resolved "https://registry.yarnpkg.com/complex.js/-/complex.js-2.1.1.tgz#0675dac8e464ec431fb2ab7d30f41d889fb25c31"
integrity sha512-8njCHOTtFFLtegk6zQo0kkVX1rngygb/KQI6z1qZxlFI3scluC+LVTCFbrkWjBv4vvLlbQ9t88IPMC6k95VTTg==

component-emitter@^1.2.1, component-emitter@^1.3.0:
version "1.3.0"
resolved "https://registry.yarnpkg.com/component-emitter/-/component-emitter-1.3.0.tgz#16e4070fba8ae29b679f2215853ee181ab2eabc0"
Expand Down Expand Up @@ -6908,6 +6920,11 @@ decimal.js@^10.2.1:
resolved "https://registry.yarnpkg.com/decimal.js/-/decimal.js-10.3.1.tgz#d8c3a444a9c6774ba60ca6ad7261c3a94fd5e783"
integrity sha512-V0pfhfr8suzyPGOx3nmq4aHqabehUZn6Ch9kyFpV79TGDTWFmHqUqXdabR7QHqxzrYolF4+tVmJhUG4OURg5dQ==

decimal.js@^10.4.3:
version "10.4.3"
resolved "https://registry.yarnpkg.com/decimal.js/-/decimal.js-10.4.3.tgz#1044092884d245d1b7f65725fa4ad4c6f781cc23"
integrity sha512-VBBaLc1MgL5XpzgIP7ny5Z6Nx3UrRkIViUkPUdtl9aya5amy3De1gsUUSB1g3+3sExYNjCAsAznmukyxCb1GRA==

decode-uri-component@^0.2.0:
version "0.2.2"
resolved "https://registry.yarnpkg.com/decode-uri-component/-/decode-uri-component-0.2.2.tgz#e69dbe25d37941171dd540e024c444cd5188e1e9"
Expand Down Expand Up @@ -7857,6 +7874,11 @@ escalade@^3.1.1:
resolved "https://registry.yarnpkg.com/escalade/-/escalade-3.1.1.tgz#d8cfdc7000965c5a0174b4a82eaa5c0552742e40"
integrity sha512-k0er2gUkLf8O0zKJiAhmkTnJlTvINGv7ygDNPbeIsX/TJjGJZHuh9B2UxbsaEkmlEo9MfhrSzmhIlhRlI2GXnw==

escape-latex@^1.2.0:
version "1.2.0"
resolved "https://registry.yarnpkg.com/escape-latex/-/escape-latex-1.2.0.tgz#07c03818cf7dac250cce517f4fda1b001ef2bca1"
integrity sha512-nV5aVWW1K0wEiUIEdZ4erkGGH8mDxGyxSeqPzRNtWP7ataw+/olFObw7hujFWlVjNsaDFw5VZ5NzVSIqRgfTiw==

escape-string-regexp@4.0.0, escape-string-regexp@^4.0.0:
version "4.0.0"
resolved "https://registry.yarnpkg.com/escape-string-regexp/-/escape-string-regexp-4.0.0.tgz#14ba83a5d373e3d311e5afca29cf5bfad965bf34"
Expand Down Expand Up @@ -10952,6 +10974,11 @@ jake@^10.8.5:
filelist "^1.0.1"
minimatch "^3.0.4"

javascript-natural-sort@^0.7.1:
version "0.7.1"
resolved "https://registry.yarnpkg.com/javascript-natural-sort/-/javascript-natural-sort-0.7.1.tgz#f9e2303d4507f6d74355a73664d1440fb5a0ef59"
integrity sha512-nO6jcEfZWQXDhOiBtG2KvKyEptz7RVbpGP4vTD2hLBdmNQSsCiicO2Ioinv6UI4y9ukqnBpy+XZ9H6uLNgJTlw==

jest-canvas-mock@^2.5.1:
version "2.5.1"
resolved "https://registry.yarnpkg.com/jest-canvas-mock/-/jest-canvas-mock-2.5.1.tgz#81509af658ef485e9a1bf39c64e06761517bdbcb"
Expand Down Expand Up @@ -12328,6 +12355,21 @@ markdown-it@^12.3.2:
mdurl "^1.0.1"
uc.micro "^1.0.5"

mathjs@^11.8.2:
version "11.8.2"
resolved "https://registry.yarnpkg.com/mathjs/-/mathjs-11.8.2.tgz#085d5cae70cd5c9702c0916979d6f55c74cb87e6"
integrity sha512-ZePu0oDbM0vuFExikIMY/9syjo/jbgNbX6ti+iMdaALDuxciMCsXIslGDBEn7QCpCWYBiVCYmc0lsmk5bwHBdQ==
dependencies:
"@babel/runtime" "^7.22.5"
complex.js "^2.1.1"
decimal.js "^10.4.3"
escape-latex "^1.2.0"
fraction.js "^4.2.0"
javascript-natural-sort "^0.7.1"
seedrandom "^3.0.5"
tiny-emitter "^2.1.0"
typed-function "^4.1.0"

mathml-tag-names@^2.1.3:
version "2.1.3"
resolved "https://registry.yarnpkg.com/mathml-tag-names/-/mathml-tag-names-2.1.3.tgz#4ddadd67308e780cf16a47685878ee27b736a0a3"
Expand Down Expand Up @@ -15059,6 +15101,11 @@ regenerator-runtime@^0.11.0:
resolved "https://registry.yarnpkg.com/regenerator-runtime/-/regenerator-runtime-0.11.1.tgz#be05ad7f9bf7d22e056f9726cee5017fbf19e2e9"
integrity sha512-MguG95oij0fC3QV3URf4V2SDYGJhJnJGqvIIgdECeODCT98wSWDAJ94SSuVpYQUoTcGUIL6L4yNB7j1DFFHSBg==

regenerator-runtime@^0.13.11:
version "0.13.11"
resolved "https://registry.yarnpkg.com/regenerator-runtime/-/regenerator-runtime-0.13.11.tgz#f6dca3e7ceec20590d07ada785636a90cdca17f9"
integrity sha512-kY1AZVr2Ra+t+piVaJ4gxaFaReZVH40AKNo7UCX6W+dEwBo/2oZJzqfuN1qLq1oL45o56cPaTXELwrTh8Fpggg==

regenerator-runtime@^0.13.3, regenerator-runtime@^0.13.4:
version "0.13.9"
resolved "https://registry.yarnpkg.com/regenerator-runtime/-/regenerator-runtime-0.13.9.tgz#8925742a98ffd90814988d7566ad30ca3b263b52"
Expand Down Expand Up @@ -15677,6 +15724,11 @@ secure-json-parse@^2.4.0:
resolved "https://registry.yarnpkg.com/secure-json-parse/-/secure-json-parse-2.4.0.tgz#5aaeaaef85c7a417f76271a4f5b0cc3315ddca85"
integrity sha512-Q5Z/97nbON5t/L/sH6mY2EacfjVGwrCcSi5D3btRO2GZ8pf1K1UN7Z9H5J57hjVU2Qzxr1xO+FmBhOvEkzCMmg==

seedrandom@^3.0.5:
version "3.0.5"
resolved "https://registry.yarnpkg.com/seedrandom/-/seedrandom-3.0.5.tgz#54edc85c95222525b0c7a6f6b3543d8e0b3aa0a7"
integrity sha512-8OwmbklUNzwezjGInmZ+2clQmExQPvomqjL7LFqOYqtmuxRgQYqOD3mHaU+MvZn5FLUeVxVfQjwLZW/n/JFuqg==

selenium-webdriver@^4.0.0-alpha.7:
version "4.0.0-alpha.7"
resolved "https://registry.yarnpkg.com/selenium-webdriver/-/selenium-webdriver-4.0.0-alpha.7.tgz#e3879d8457fd7ad8e4424094b7dc0540d99e6797"
Expand Down Expand Up @@ -17046,6 +17098,11 @@ timsort@~0.3.0:
resolved "https://registry.yarnpkg.com/timsort/-/timsort-0.3.0.tgz#405411a8e7e6339fe64db9a234de11dc31e02bd4"
integrity sha1-QFQRqOfmM5/mTbmiNN4R3DHgK9Q=

tiny-emitter@^2.1.0:
version "2.1.0"
resolved "https://registry.yarnpkg.com/tiny-emitter/-/tiny-emitter-2.1.0.tgz#1d1a56edfc51c43e863cbb5382a72330e3555423"
integrity sha512-NB6Dk1A9xgQPMoGqC5CVXn123gWyte215ONT5Pp5a0yt4nlEoO1ZWeCwpncaekPHXO60i47ihFnZPiRPjRMq4Q==

tiny-invariant@^1.0.2, tiny-invariant@^1.0.6:
version "1.2.0"
resolved "https://registry.yarnpkg.com/tiny-invariant/-/tiny-invariant-1.2.0.tgz#a1141f86b672a9148c72e978a19a73b9b94a15a9"
Expand All @@ -17069,11 +17126,6 @@ tinygradient@^1.1.5:
"@types/tinycolor2" "^1.4.0"
tinycolor2 "^1.0.0"

tinymath@1.2.1:
version "1.2.1"
resolved "https://registry.yarnpkg.com/tinymath/-/tinymath-1.2.1.tgz#f97ed66c588cdbf3c19dfba2ae266ee323db7e47"
integrity sha512-8CYutfuHR3ywAJus/3JUhaJogZap1mrUQGzNxdBiQDhP3H0uFdQenvaXvqI8lMehX4RsanRZzxVfjMBREFdQaA==

tmp@0.0.30:
version "0.0.30"
resolved "https://registry.yarnpkg.com/tmp/-/tmp-0.0.30.tgz#72419d4a8be7d6ce75148fd8b324e593a711c2ed"
Expand Down Expand Up @@ -17388,6 +17440,11 @@ typechecker@^6.2.0:
resolved "https://registry.yarnpkg.com/typechecker/-/typechecker-6.4.0.tgz#c087dc744c5a0f17524d58a17eb31a9660ab7324"
integrity sha512-EbOu+9szY13mhl0EsvLXnR+pTCa3gTHQQPLdce72ujcC9fRHXlVFBNXtHeRhgzLxLlKUh4zA9C0tezLDgshf+A==

typed-function@^4.1.0:
version "4.1.0"
resolved "https://registry.yarnpkg.com/typed-function/-/typed-function-4.1.0.tgz#da4bdd8a6d19a89e22732f75e4a410860aaf9712"
integrity sha512-DGwUl6cioBW5gw2L+6SMupGwH/kZOqivy17E4nsh1JI9fKF87orMmlQx3KISQPmg3sfnOUGlwVkroosvgddrlg==

typedarray-to-buffer@^3.1.5:
version "3.1.5"
resolved "https://registry.yarnpkg.com/typedarray-to-buffer/-/typedarray-to-buffer-3.1.5.tgz#a97ee7a9ff42691b9f783ff1bc5112fe3fca9080"
Expand Down
Loading