Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stdlibs): add package strconv #1464

Merged
merged 22 commits into from
Oct 17, 2024
Merged

feat(stdlibs): add package strconv #1464

merged 22 commits into from
Oct 17, 2024

Conversation

thehowl
Copy link
Member

@thehowl thehowl commented Dec 19, 2023

This PR adds the full strconv package, implemented as pure Gno code. It removes the native functions Itoa (and others), and adds support for new functions such as FormatFloat.

Summary of changes

  • Standard libraries
    • Changes of strconv from Go stdlib: https://gist.github.com/thehowl/904b42b1ea53fef9b8a0486155c02b73 -- tldr:
      • removed everything related to complex types.
      • avoid using reflection and dot imports.
      • instead of loading testfp.txt, embed the file directly into the source.
      • define min/max explicitly as they're not built-in yet.
      • remove go:build tags.
      • (all of these mostly involve test files.)
    • unicode
      • Update tables, so that strconv tests succeed.
    • unicode/utf8
      • Update to latest go version. Mostly, use fallback (as we now half-support it) and use AppendString.
  • GnoVM
    • PackageInjector is no longer necessary (hallelujah), see [META] improved native bindings #814 for context. This justifies the changes in store.go, store_test.go, nodes.go, tests/imports.go.
    • gonative.go and machine.go changes improve some error messages.
    • preprocess.go changes fix a bug which can be seen in the for20.gno test. If a for loop is labeled, then a bare break (ie. without a label to break to) would panic, as it wouldn't find any for loop without a label (in findBranchLabel). I added a regression test as well as a couple test showing the error message for when we misplace continue/break statements.
  • Tests.
    • strconv.Itoa now uses more gas than its existing native implementation. This is to be expected; we can consider moving it back to a native implementation if we deem it useful for performance, but I think it's good for us to work on having as much code implemented directly in gno before moving it back to Go for performance.

@thehowl thehowl added 🌱 feature New update to Gno 📦 🤖 gnovm Issues or PRs gnovm related labels Dec 19, 2023
@thehowl thehowl added this to the 🌟 main.gno.land (wanted) milestone Dec 19, 2023
@thehowl thehowl self-assigned this Dec 19, 2023
Copy link

codecov bot commented Dec 19, 2023

Codecov Report

Attention: Patch coverage is 83.07692% with 11 lines in your changes missing coverage. Please review.

Project coverage is 63.47%. Comparing base (6d986bf) to head (f8618e8).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
gnovm/pkg/gnolang/machine.go 70.37% 6 Missing and 2 partials ⚠️
gnovm/pkg/gnolang/preprocess.go 92.85% 1 Missing and 1 partial ⚠️
gnovm/pkg/gnolang/gonative.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1464      +/-   ##
==========================================
+ Coverage   61.11%   63.47%   +2.36%     
==========================================
  Files         565      552      -13     
  Lines       75396    86833   +11437     
==========================================
+ Hits        46077    55117    +9040     
- Misses      25950    28135    +2185     
- Partials     3369     3581     +212     
Flag Coverage Δ
contribs/gnodev 60.62% <ø> (-0.84%) ⬇️
contribs/gnofaucet 15.77% <ø> (+0.45%) ⬆️
contribs/gnokeykc ?
contribs/gnomd ?
gno.land 67.56% <ø> (-0.38%) ⬇️
gnovm 67.24% <85.71%> (+1.06%) ⬆️
go-1.22.x ?
misc/autocounterd ?
misc/genproto ?
misc/genstd 79.72% <ø> (-0.83%) ⬇️
misc/goscan ?
misc/logos 19.95% <ø> (+0.07%) ⬆️
misc/loop ?
tm2 62.25% <ø> (+0.23%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@thehowl
Copy link
Member Author

thehowl commented Jan 16, 2024

So, tests are currently failing. A bunch of tests I've fixed after realising that the underlying problem is with how we do comparisons when we have a bitshift with an untyped integer on the lhs: #1462 (comment)

I suspect there may be more bugs involved.

In any case, I attempted to make a separate branch from this which merged in the changes from #1426. Turns out, that the tests are completely passing with that code applied to Gno.

So, I'll be continuing work on this once #1426 is merged. For the meantime, I'll update the OP to mark what remaining work is to be done to complete this PR so I can easily pick it up later.

@ltzmaxwell
Copy link
Contributor

ltzmaxwell commented Jan 19, 2024

In any case, I attempted to make a separate branch from this which merged in the changes from #1426. Turns out, that the tests are completely passing with that code applied to Gno.

I did this as an experiment, it passes all the test after making a small adjustment on your code, atof_test/errEqual(). https://github.com/ltzmaxwell/gno/blob/4e6a40db2922bcd5ea8c100ad028e1dbb08cd2a7/gnovm/stdlibs/strconv/atof_test.gno#L523

go test tests/*.go -run "TestPackages/(strconv)" -v -p 1 -timeout=30m
=== RUN   TestPackages
=== RUN   TestPackages/strconv
DEBUG DISABLED (FOR TEST DEPENDENCIES INIT)
=== RUN   TestPackages/strconv/TestParseBool
=== RUN   TestPackages/strconv/TestFormatBool
=== RUN   TestPackages/strconv/TestAppendBool
=== RUN   TestPackages/strconv/TestParseFloatPrefix
=== RUN   TestPackages/strconv/TestAtof
=== RUN   TestPackages/strconv/TestAtofSlow
=== RUN   TestPackages/strconv/TestAtofRandom
    machine.go:448: tested 100 random numbers
=== RUN   TestPackages/strconv/TestRoundTrip
=== RUN   TestPackages/strconv/TestRoundTrip32
    machine.go:448: tested 21393 float32's
=== RUN   TestPackages/strconv/TestParseFloatIncorrectBitSize
=== RUN   TestPackages/strconv/TestParseUint32
=== RUN   TestPackages/strconv/TestParseUint64
=== RUN   TestPackages/strconv/TestParseUint64Base
=== RUN   TestPackages/strconv/TestParseInt32
=== RUN   TestPackages/strconv/TestParseInt64
=== RUN   TestPackages/strconv/TestParseInt64Base
=== RUN   TestPackages/strconv/TestParseUint
=== RUN   TestPackages/strconv/TestParseInt
=== RUN   TestPackages/strconv/TestAtoi
=== RUN   TestPackages/strconv/TestParseIntBitSize
=== RUN   TestPackages/strconv/TestParseUintBitSize
=== RUN   TestPackages/strconv/TestParseIntBase
=== RUN   TestPackages/strconv/TestParseUintBase
=== RUN   TestPackages/strconv/TestNumError
=== RUN   TestPackages/strconv/TestNumErrorUnwrap
=== RUN   TestPackages/strconv/TestDecimalShift
=== RUN   TestPackages/strconv/TestDecimalRound
=== RUN   TestPackages/strconv/TestDecimalRoundedInteger
=== RUN   TestPackages/strconv/TestFtoa
=== RUN   TestPackages/strconv/TestFtoaPowersOfTwo
=== RUN   TestPackages/strconv/TestFtoaRandom
    machine.go:448: testing 100 random numbers with fast and slow FormatFloat
=== RUN   TestPackages/strconv/TestFormatFloatInvalidBitSize
=== RUN   TestPackages/strconv/TestMulByLog2Log10
=== RUN   TestPackages/strconv/TestMulByLog10Log2
=== RUN   TestPackages/strconv/TestItoa
=== RUN   TestPackages/strconv/TestUitoa
=== RUN   TestPackages/strconv/TestFormatUintVarlen
=== RUN   TestPackages/strconv/TestIsPrint
=== RUN   TestPackages/strconv/TestIsGraphic
=== RUN   TestPackages/strconv/TestQuote
=== RUN   TestPackages/strconv/TestQuoteToASCII
=== RUN   TestPackages/strconv/TestQuoteToGraphic
=== RUN   TestPackages/strconv/TestQuoteRune
=== RUN   TestPackages/strconv/TestQuoteRuneToASCII
=== RUN   TestPackages/strconv/TestQuoteRuneToGraphic
=== RUN   TestPackages/strconv/TestCanBackquote
=== RUN   TestPackages/strconv/TestUnquote
=== RUN   TestPackages/strconv/TestUnquoteInvalidUTF8
=== RUN   TestPackages/strconv/TestErrorPrefixes
DEBUG ENABLED
=== RUN   TestPackages/strconv/TestFp
--- PASS: TestPackages (232.93s)
    --- PASS: TestPackages/strconv (232.92s)
        --- PASS: TestPackages/strconv/TestParseBool (0.00s)
        --- PASS: TestPackages/strconv/TestFormatBool (0.00s)
        --- PASS: TestPackages/strconv/TestAppendBool (0.00s)
        --- PASS: TestPackages/strconv/TestParseFloatPrefix (4.19s)
        --- PASS: TestPackages/strconv/TestAtof (1.09s)
        --- PASS: TestPackages/strconv/TestAtofSlow (3.16s)
        --- PASS: TestPackages/strconv/TestAtofRandom (0.03s)
        --- PASS: TestPackages/strconv/TestRoundTrip (0.01s)
        --- PASS: TestPackages/strconv/TestRoundTrip32 (3.84s)
        --- PASS: TestPackages/strconv/TestParseFloatIncorrectBitSize (0.00s)
        --- PASS: TestPackages/strconv/TestParseUint32 (0.00s)
        --- PASS: TestPackages/strconv/TestParseUint64 (0.00s)
        --- PASS: TestPackages/strconv/TestParseUint64Base (0.01s)
        --- PASS: TestPackages/strconv/TestParseInt32 (0.00s)
        --- PASS: TestPackages/strconv/TestParseInt64 (0.00s)
        --- PASS: TestPackages/strconv/TestParseInt64Base (0.01s)
        --- PASS: TestPackages/strconv/TestParseUint (0.00s)
        --- PASS: TestPackages/strconv/TestParseInt (0.00s)
        --- PASS: TestPackages/strconv/TestAtoi (0.00s)
        --- PASS: TestPackages/strconv/TestParseIntBitSize (0.00s)
        --- PASS: TestPackages/strconv/TestParseUintBitSize (0.00s)
        --- PASS: TestPackages/strconv/TestParseIntBase (0.00s)
        --- PASS: TestPackages/strconv/TestParseUintBase (0.00s)
        --- PASS: TestPackages/strconv/TestNumError (0.00s)
        --- PASS: TestPackages/strconv/TestNumErrorUnwrap (0.00s)
        --- PASS: TestPackages/strconv/TestDecimalShift (0.00s)
        --- PASS: TestPackages/strconv/TestDecimalRound (0.00s)
        --- PASS: TestPackages/strconv/TestDecimalRoundedInteger (0.00s)
        --- PASS: TestPackages/strconv/TestFtoa (0.03s)
        --- PASS: TestPackages/strconv/TestFtoaPowersOfTwo (1.77s)
        --- PASS: TestPackages/strconv/TestFtoaRandom (1.54s)
        --- PASS: TestPackages/strconv/TestFormatFloatInvalidBitSize (0.00s)
        --- PASS: TestPackages/strconv/TestMulByLog2Log10 (0.03s)
        --- PASS: TestPackages/strconv/TestMulByLog10Log2 (0.01s)
        --- PASS: TestPackages/strconv/TestItoa (0.00s)
        --- PASS: TestPackages/strconv/TestUitoa (0.00s)
        --- PASS: TestPackages/strconv/TestFormatUintVarlen (0.00s)
        --- PASS: TestPackages/strconv/TestIsPrint (106.06s)
        --- PASS: TestPackages/strconv/TestIsGraphic (109.88s)
        --- PASS: TestPackages/strconv/TestQuote (0.00s)
        --- PASS: TestPackages/strconv/TestQuoteToASCII (0.00s)
        --- PASS: TestPackages/strconv/TestQuoteToGraphic (0.00s)
        --- PASS: TestPackages/strconv/TestQuoteRune (0.00s)
        --- PASS: TestPackages/strconv/TestQuoteRuneToASCII (0.00s)
        --- PASS: TestPackages/strconv/TestQuoteRuneToGraphic (0.00s)
        --- PASS: TestPackages/strconv/TestCanBackquote (0.00s)
        --- PASS: TestPackages/strconv/TestUnquote (0.01s)
        --- PASS: TestPackages/strconv/TestUnquoteInvalidUTF8 (0.00s)
        --- PASS: TestPackages/strconv/TestErrorPrefixes (0.00s)
        --- PASS: TestPackages/strconv/TestFp (0.19s)
PASS
ok  	command-line-arguments	233.372s

@thehowl thehowl mentioned this pull request Feb 27, 2024
8 tasks
@thehowl
Copy link
Member Author

thehowl commented Mar 26, 2024

Current state: the issues arising porting this standard library are a result of #1462. These are fixed in #1775, which depends on #1426, which at the time of writing are pending reviews and merge.

@thehowl thehowl marked this pull request as ready for review October 15, 2024 19:26
@github-actions github-actions bot added the 📦 ⛰️ gno.land Issues or PRs gno.land package related label Oct 15, 2024
@thehowl
Copy link
Member Author

thehowl commented Oct 15, 2024

This PR is now ready for review. I updated the OP with a summary of the changes involved.

@moul
Copy link
Member

moul commented Oct 16, 2024

but I think it's good for us to work on having as much code implemented directly in gno before moving it back to Go for performance.

Sounds reasonable, let's do it.

gnovm/pkg/gnolang/machine.go Outdated Show resolved Hide resolved
Copy link
Member

@moul moul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a nice one!

@thehowl thehowl merged commit 05cd4f5 into master Oct 17, 2024
122 checks passed
@thehowl thehowl deleted the dev/morgan/strconv branch October 17, 2024 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📦 ⛰️ gno.land Issues or PRs gno.land package related 📦 🤖 gnovm Issues or PRs gnovm related 🌱 feature New update to Gno
Projects
Status: Done
Status: Done
Development

Successfully merging this pull request may close these issues.

5 participants