[lexical-markdown] Feature: add ability to control finding the end of a node matched by TextMatchTransformer

[AlessioGr]

Description

Currently, for a TextMatchTransformer, the importRegExp is used to find a match in a given text. The end of the matched node is determined by the length of the match.

However, in some scenarios, you want to use importRegExp to only match the start of the node. You might then want to determine the end manually, for example via a different regex, or some custom logic.

Use-Case example

A good example is matching inline JSX, e.g. "Hello <InlineBlock prop1="hey">some text</InlineBlock> there!".

In this example, we use importRegExp solely to match <InlineBlock. We use a custom algorithm to then find the proper end of this node (</InlineBlock>). You might think this is possible solely with regex, but it's not.

Example that showcases why:

"Hello <InlineBlock prop1="hey">some <InlineBlock>Sub Block</InlineBlock> text</InlineBlock> there!"

We need custom logic that keeps track of the opening and closing JSX tags in order to find the correct end of the node: the second </InlineBlock>.

This PR adds a getEndIndex property to TextMatchTransformer that allows us to do these kinds of things, by providing our custom algorithm to find the end of the node (Example).

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
lexical	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 5, 2024 7:08am
lexical-playground	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 5, 2024 7:08am

[github-actions]

size-limit report 📦

Path	Size
lexical - cjs	29.92 KB (0%)
lexical - esm	29.78 KB (0%)
@lexical/rich-text - cjs	38.57 KB (0%)
@lexical/rich-text - esm	31.63 KB (0%)
@lexical/plain-text - cjs	37.22 KB (0%)
@lexical/plain-text - esm	28.94 KB (0%)
@lexical/react - cjs	40.31 KB (0%)
@lexical/react - esm	33.04 KB (0%)

[etrepum]

Test coverage for this new functionality would be nice. It seems like you might also want to be able to let the transform fail if the end doesn't match correctly, maybe a more general solution (e.g. the option to use a function instead of regex for matching altogether) would be more appropriate? There's currently no "compilation" to merge regexes so there wouldn't be much of a disadvantage to having that sort of option.

[AlessioGr]

Test coverage for this new functionality would be nice

Will add one after #6682 is merged!

It seems like you might also want to be able to let the transform fail if the end doesn't match correctly

Done! Can now return false as well

maybe a more general solution (e.g. the option to use a function instead of regex for matching altogether) would be more appropriate? There's currently no "compilation" to merge regexes so there wouldn't be much of a disadvantage to having that sort of option.

I think adding a general option would be a good idea - however, I don't always wanna have to handle all that code related to text splitting myself (this:

)

Maybe it's better to add that in a separate PR in addition to getEndIndex?

[etrepum]

@AlessioGr were you planning to still add the test case here now that #6682 is merged?

[AlessioGr]

Yep, I am! Will be done this week

[etrepum]

Just following up here, looks like we are still waiting on test cases but otherwise this is ready to go

[AlessioGr]

@etrepum sorry for the delay - just added a test, so this should be ready now.

Pretty tricky finding a good, simple use-case for a test here, as the actual function we use to match JSX is huge. For this test, I just added a very basic text match converter where getEndIndex is used just for simple JSX opening/closing tag counting. Similar to the other CODE_TAG_COUNTER_EXAMPLE example, but simpler and for text match transformers