Add JSON storage limit utilities #14

david0xd · 2022-06-24T16:16:42Z

This PR will add utility functions to handle limits and validation of JSON storage used by Snaps.

This is still work in progress.

The main goal of this PR is to implement the following function:

export function validateJsonAndGetSize(
  jsObject: unknown,
  skipSizingProcess = false,
): [isValid: boolean, plainTextSizeInBytes: number]

This function has two capabilities:

Validate JavaScript object serialization capabilities (validate if JavaScript object can be serialized to JSON by using JSON.stringify(...))
Calculate the potential size in Bytes after JavaScript object is serialized to plain text in JSON format.

Size calculation can be skipped by setting skipSizingProcess=true in which case only the validation process will be done and 0 will be returned for a size.

Fixes https://app.zenhub.com/workspaces/snaps-platform-615b3a7c08d2b20015eb6c4e/issues/metamask/snaps-skunkworks/571
Fixes MetaMask/snaps#571

Related to: MetaMask/snaps#209

FrederikBolding

Great start! Left some comments

src/misc.ts

src/json.ts

src/misc.ts

src/test.data.ts

src/misc.ts

src/json.ts

src/misc.ts

src/json.ts

src/test.data.ts

src/misc.ts

src/json.ts

src/misc.ts

src/test.data.ts

Mrtenz

I think we should prefer avoiding any whenever possible.

src/json.test.ts

src/json.ts

Mrtenz

LGTM 👍

src/json.ts

… tests

FrederikBolding

LGTM - Good work on this!

FrederikBolding requested changes Jul 1, 2022

View reviewed changes

Mrtenz requested changes Jul 4, 2022

View reviewed changes

FrederikBolding reviewed Jul 5, 2022

View reviewed changes

src/json.ts Outdated Show resolved Hide resolved

src/test.data.ts Show resolved Hide resolved

src/misc.ts Show resolved Hide resolved

src/json.ts Outdated Show resolved Hide resolved

src/misc.ts Outdated Show resolved Hide resolved

src/test.data.ts Show resolved Hide resolved

Mrtenz reviewed Jul 7, 2022

View reviewed changes

src/json.test.ts Outdated Show resolved Hide resolved

src/json.test.ts Outdated Show resolved Hide resolved

FrederikBolding requested changes Jul 11, 2022

View reviewed changes

src/json.ts Outdated Show resolved Hide resolved

src/json.ts Outdated Show resolved Hide resolved

src/json.ts Outdated Show resolved Hide resolved

Mrtenz reviewed Jul 12, 2022

View reviewed changes

FrederikBolding reviewed Jul 12, 2022

View reviewed changes

src/json.ts Show resolved Hide resolved

src/json.ts Outdated Show resolved Hide resolved

david0xd added 23 commits July 12, 2022 15:29

Add JSON storage limit utilities

887021c

Add handling for string length depending on encoding

95da527

Add handling for number sizes

5a38ae7

Refactor code and add more tests

b3d204b

Add functional code improvements and tests

f4f38e4

Add edge case handling and more complex tests

a354cf9

Refactor code, change approaches and improve tests

6655545

Minor code refactoring and additional test

80fdac7

Add one more unit test

0634e04

Refactor tests and test data

0b5444f

Add skip feature for JSON size calculation

e884151

Add algorithm improvements and new test data

9fc439b

Minor refactoring of calculateStringSize function

a1c2265

Minor refactoring of isASCII function

7fd7f44

Minor refactoring of skip sizing feature

72fe7a3

Add unit tests for circular structures

0abc592

Add test improvement with escape characters

c8db2f6

Code refactoring

87484b2

Code refactoring & improvements

ee6d28f

Small code refactoring (review answer)

a573df7

Another minor code refactoring (review answer)

537e236

Remove useless comment

a053094

Add more tests for serialization checks of functions and symbols

e16f51e

david0xd added 10 commits July 12, 2022 15:29

Add specific approach for handling Date objects

2237edd

Add ECMA TC39 (test262) test scenarios and improve algorithm

b0d8b6a

Add another batch of ECMA TC39 (test262) test scenarios

b87f459

Fix types in tests

1c1586b

Add optimized check for circular structures

6913a2c

Change approach for circular object detection

31c7da6

Improve circular object detection

484a376

Improve Date object sizing

50f41c2

Change type-check approach (if-else)

eb20c53

Add explanation comments for circular structure detection and improve…

b6482a7

… tests

david0xd force-pushed the dd/snaps-storage-limit branch from d1e06bb to b6482a7 Compare July 12, 2022 13:30

FrederikBolding approved these changes Jul 12, 2022

View reviewed changes

david0xd marked this pull request as ready for review July 12, 2022 14:53

david0xd requested a review from a team as a code owner July 12, 2022 14:53

Mrtenz approved these changes Jul 12, 2022

View reviewed changes

david0xd merged commit aeb62e7 into main Jul 12, 2022

david0xd deleted the dd/snaps-storage-limit branch July 12, 2022 15:16

david0xd mentioned this pull request Jul 13, 2022

Add new validation and limit for storage MetaMask/snaps#621

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add JSON storage limit utilities #14

Add JSON storage limit utilities #14

david0xd commented Jun 24, 2022 •

edited

Loading

FrederikBolding left a comment

Mrtenz left a comment

Mrtenz left a comment

FrederikBolding left a comment

Add JSON storage limit utilities #14

Add JSON storage limit utilities #14

Conversation

david0xd commented Jun 24, 2022 • edited Loading

FrederikBolding left a comment

Choose a reason for hiding this comment

Mrtenz left a comment

Choose a reason for hiding this comment

Mrtenz left a comment

Choose a reason for hiding this comment

FrederikBolding left a comment

Choose a reason for hiding this comment

david0xd commented Jun 24, 2022 •

edited

Loading