Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

with recent NeoJSON work, should probably take a different approach to handling Unicode String encoding #6

Closed
dalehenrich opened this issue Nov 26, 2014 · 7 comments
Closed

with recent NeoJSON work, should probably take a different approach to handling Unicode String encoding #6

dalehenrich opened this issue Nov 26, 2014 · 7 comments

Comments

@dalehenrich
Copy link
Member

NeoJSON work ...
@dalehenrich
Copy link
Member Author

needs to be done pre-3.3 release and use encodeAsUTF8intoString instead of encodeAsUtf8 as String

@dalehenrich
Copy link
Member Author

Looks like the method STONReaderTests>>testJsonString was changed for GemStone ... so this one is likely to be the canary in the gold mine ... Also this test is familiar from NeoJSON where I added additional tests to cover the Unicode16 and Unicode32 examples .... needed for STON tests as well ... and critical for getting unicode handling right.

This was referenced Jun 19, 2015
Merged
Closed
dalehenrich added a commit that referenced this issue Jun 24, 2015
@dalehenrich
Issue #6: isolate the methods that will be involved in the fix for th…
d910c70 
…is issue so that the changes can be cherry-picked into the common branch
dalehenrich added a commit that referenced this issue Jun 24, 2015
@dalehenrich
Issue #6: cherry-pick the methods that will be involved in bugfix to …
b36748b 
…common branch
dalehenrich added a commit that referenced this issue Jun 24, 2015
@dalehenrich
Issue #6: remove String extenstion methods for decode/encode ... not …
d44fd8e 
…a common implementation
dalehenrich added a commit that referenced this issue Jun 24, 2015
@dalehenrich
Issue #6: picked up missing methods for STONReader and STONWriter
cb3aa48
dalehenrich added a commit that referenced this issue Jun 24, 2015
@dalehenrich
Issue #6 add STON class>>fromUTF8: and STON class>>toUTF8:
2285e27
@dalehenrich
Copy link
Member Author

As I have evolved the STON object serialization for tODE, I am now interested in refactoring the STON work into three segments:

  • old-style STON (v0.5 and v0.9) supports which means handicapped handling of multi-byte characters
  • old-style GemStone/tODE support (v0.5.1) where UTF8 encoding is supplied at individual string level and codePoints over 126 are escaped
  • new-style GemStone/TODE support (v0.9.*) where UTF8 encoding is done for the entire STON string, no character escaping is necessary (UTF8uses 8-bit ASCII characters), if escaped characters are encountered, assumption is made that STON was encoded using old-style GemStone/tODE (v0.5.1)
  • new-style STON (utf8) support where UTF8 encoding is done for the entire STON string, no character escaping is used and on decode escaped codePoints signal an error ...
  • support both pharo and gemstone on the gemstone branch ...

dalehenrich added a commit that referenced this issue Jun 26, 2015
@dalehenrich
Issue #6: laying the groundwork for supporting the 4 STON serializati…
b27abeb 
…on options:

  - classic (utf8 challenged)
  - utf8-friendly
  - GemStone/tODE v0.5.1
  - GemStone/tODE v0.9.*

Consolidating the Pharo/GemStone packages in the gemstone branch.
dalehenrich added a commit that referenced this issue Jun 26, 2015
@dalehenrich
Issue #6: split out platform-specific methods for pharo and gemstone
f3c8735
dalehenrich added a commit that referenced this issue Jun 26, 2015
@dalehenrich
Issue #6: push a few more methods around
a9a7bb5
dalehenrich added a commit that referenced this issue Jun 26, 2015
@dalehenrich
Issue #6: split out platform-specific tests
753cdcb
dalehenrich added a commit that referenced this issue Jun 26, 2015
@dalehenrich
Issue #6: update travis artifacts for combined pharo/gemstone branch
3e5828b
dalehenrich added a commit that referenced this issue Jun 26, 2015
@dalehenrich
Issue #6: fix syntax error
69c2e48
dalehenrich added a commit to dalehenrich/tode that referenced this issue Jun 26, 2015
@dalehenrich
With recent Ston work (GsDevKit/ston#6) switch up baseline to same br…
e53205e 
…anch/tag for Ston ...
dalehenrich added a commit that referenced this issue Jun 26, 2015
@dalehenrich
Issue #6: STONReader and STONWriter tests are now passing for all 4 s…
eb9280c 
…erialization schemes
dalehenrich added a commit that referenced this issue Jun 27, 2015
@dalehenrich
Issue #6: pretty much flesh out the test suite to cover expanded Unic…
4c565b8 
…ode strings (Unicode16 and Unicode32 examples) and a bit of cross materialization (051 serializer -> 091 marterializer)
@dalehenrich
Copy link
Member Author

Implementation covering the 4 different serialization schemes .. now need to revisit STON api to access the different schemes ... (don't break tODE:) .... convert object serializers first)

@dalehenrich
Copy link
Member Author

get test results: https://travis-ci.org/GsDevKit/ston/builds/68563919

dalehenrich added a commit that referenced this issue Jun 29, 2015
@dalehenrich
Issue #6: Cannot embed Unicode16 or Unicode32 strings in 2.x methods
154dd73
@dalehenrich
Copy link
Member Author

get test results (again) https://travis-ci.org/GsDevKit/ston/builds/68845525

@dalehenrich dalehenrich mentioned this issue Jun 29, 2015
Closed
dalehenrich added a commit that referenced this issue Jun 29, 2015
@dalehenrich
Issue #6: return STONWriter and STONReader implementations to origina…
bcdafe6 
…l implementation ... GemStone-specific mods are now present in STON051*, STON091*, and STONutf8* classes ... Modify tests to accomodate svenvc#11
dalehenrich added a commit that referenced this issue Jun 29, 2015
@dalehenrich
Issue #6: remote the (now) unused protocol in STON for using UTF8 enc…
12f7eae 
…oding
@dalehenrich dalehenrich mentioned this issue Jun 29, 2015
Merged
@dalehenrich
Copy link
Member Author

tests green

dalehenrich added a commit that referenced this issue Jun 30, 2015
@dalehenrich
Merge pull request #10 from GsDevKit/gemstone_dev
25aace1 
Issue #6 merge: v0.9.4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@dalehenrich