Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fast ANSI string operations in C #316

Merged
merged 54 commits into from
Aug 4, 2021
Merged

Fast ANSI string operations in C #316

merged 54 commits into from
Aug 4, 2021

Conversation

gaborcsardi
Copy link
Member

@gaborcsardi gaborcsardi commented Jul 17, 2021 •
edited
Loading

Implemented:

  • ansi_simplify()
  • ansi_substr()
  • ansi_has_any()
  • ansi_strip()
  • ansi_html()
  • utf8_nchar()

TODO:

  • Support ANSI hyperlinks in the C ANSI iterator.
  • Better UTF-8 display width, consider combining characters.
  • Make new ANSI functions work for NAs.

Might also implement other ansi_*() functions in C, to make them faster.

Closes #316.

@gaborcsardi
Beginning of the ANSI iterator
cf9a840
@gaborcsardi
Improve ANSI operations
dc45041 
* Omit empty tags now.
* Better color representation.
@gaborcsardi
ANSI: handle colors properly
bd5cfd5
@gaborcsardi
Refactor ANSI buffer to be self-contained
8bb3cec
@gaborcsardi
Generic ANSI manipulation iterator
ef0dced
@gaborcsardi
UTF-8 character width calculation
4515742
@gaborcsardi
Improve detection of incomplete UTF-8 characters
ca4faff
@gaborcsardi
Make UTF-8 funcs available in other files
9011045
@gaborcsardi
Speed up ansi_substr()
65ab59d
@gaborcsardi
Add minimal docs to ansi_simplify()
3e2c2b3 
Plus the output classes as well.
@gaborcsardi
Do not drop classes in ansi_simplify()
493deed
@gaborcsardi
Improve parsing ANSI colors
787b37f 
Especially broken color sequences.
@gaborcsardi
Further improve ANSI color parsing
f9f6f56
@gaborcsardi
ANSI manipulation: always close all tags
b860504
@gaborcsardi
Handle SGR off tag better
7ee0ad9 
We only emit it if there was an unkown SGR tag
in the input. Otherwise we use the more specific
closing tags.
@gaborcsardi
Better handling of ANSI 0m, well formed color tags
f793146 
* Now we always close a color tag, before we
  start another one.
* 0m is now only emitted when there was an
  unknown SGR tag before.
@gaborcsardi
More ANSI test cases
91ff798
@gaborcsardi
ANSI manipulation: support bright colors
a8b9b5b 
Also simplify data structure of colors, just store
the color number directly, we are using 254 and 255
for 256 and RGB color, anyway.
@gaborcsardi
ANSI manipulation: emit well-formed tags.
71b8806 
I.e. <t1><t2>...</t2></t1> instead of
<t1><t2>...</t1></t2>.
@gaborcsardi
Add ansi_html()
9eb32aa 
Plus helper function for CSS and also ansi_simplify().
@gaborcsardi
Handle CSI seqs in ansi_simplify() and ansi_html()
4d9be35
@gaborcsardi
Slight ANSI iterator speedup
cecb043 
When there are no tags.
@gaborcsardi
Rewrite ansi_has_any() with iterator
2511461 
It does not currently find hyperlink seqs...
@gaborcsardi
Remove some unused ANSI data tags
323ab81
@gaborcsardi
Rewrite ansi_strip() with ANSI iterator
92c8574
@gaborcsardi
Merge branch 'master' into fix/fast-ansi
9026196
@gaborcsardi
Fix ansi_strtrim() for vectors
a842da1
@gaborcsardi
Handle NAs in ANSI operations
d13aca8
@gaborcsardi
Proper UTF-8 graphemes in width measurement
daeab9c 
Coming to the other functions as well...
@gaborcsardi gaborcsardi mentioned this pull request Jul 20, 2021
Closed
15 tasks
@gaborcsardi
ANSI functions force UTF-8 now
a6ed5db 
They convert to UTF-8 on input, always return UTF-8.
@gaborcsardi
Fix ansi_columns(), blame copy-paste
274a02d
@gaborcsardi
Fix compilation on Linux and Windows
7fe6360
@gaborcsardi
Fox grapheme iterator crash
a96155c
@gaborcsardi
Fix ANSI code on older R versions
cfe4fee 
`LENGTH(NULL)` does not seems to work well on
R 3.4 and before.
@gaborcsardi
Use graphemes in ansi_nchar()
5e92e5e 
Also add `utf8_nchar()`.

Towards #317.
@gaborcsardi
ansi_strsplit() converts argument to character now
f962704
@gaborcsardi
Fix ansi_strip() if sgr = FALSE or csi = FALSE
d8474d6
@gaborcsardi
Some ANSI test cases
3a883ae
@gaborcsardi
Do not try to kill the thread twice
18cb66a 
The second might happen after the DLL was unloaded,
e.g. in testthatlabs, I am not sure why.
@gaborcsardi
Message when flushing coverage info
5bfac89 
It is nice to see that it is happening.
@gaborcsardi
Rename capture_messages() test helper
ca6f5e5 
To avoid name clash with testthat.
@gaborcsardi
Fix rlang error tests in dark iterm themes
98d93f8
@gaborcsardi
Add grapheme support to ansi_nchar()
30d9231
@gaborcsardi
Test ansi_align with graphemes
db5134b
@gaborcsardi
Test ansi_columns() with graphemes
53398ed
@gaborcsardi
Substrings using grapheme clusters
757c9cf 
`ansi_substr()` and `ansi_substring()` uses this now,
and the new `utf8_substr()` function as well.
The latter is slightly faster, but it does not work
for ANSI styled strings.
@gaborcsardi
Merge branch 'master' into fix/fast-ansi
3d6c121
@gaborcsardi
Add unnamed-chunk-1.svg
b2ebe86 
Some chunks in the docs create this, and
it is very small, so it is simpler to just
include it.
@gaborcsardi
Update snapshots
33296c1 
For simplified ANSI sequences.
@gaborcsardi
Add ANSI benchmark vignette
99c564c
@gaborcsardi
Add utf8_* functions to the pkgdown reference
84d09e1 
IDK how and when they were removed...
@gaborcsardi
Note that style_hyperlink() is experimental.
1c9ccd9 
And that ansi_* functions do not handle it.
@gaborcsardi
Add utf8_graphemes()
a169b5e
@gaborcsardi
Fix broken example
a9e458d
@gaborcsardi gaborcsardi mentioned this pull request Aug 4, 2021
Closed
@gaborcsardi gaborcsardi merged commit 84dedca into master Aug 4, 2021
@gaborcsardi gaborcsardi deleted the fix/fast-ansi branch September 20, 2021 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@gaborcsardi