You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When I try to parse a markdown that is one big list of ordered and unordered list, mistune runs into Recursion Error. I suspect it might be taking it as one big nested list.
If this markdown continued like this to have more lines than the sys.getrecursionlimit(), parsing this markdown would result in a recursion error.
Here is some code that reproduces the error:
importmistuneimportsyslist_to_repeat="""- Fruits1. Apple2. Orange"""breaking_md=""# Get number of list items such that we hit the recursion limitrepeat_times=sys.getrecursionlimit() //len(list_to_repeat.splitlines()) +10print(f"Recursion Limit: {sys.getrecursionlimit()}")
print(f"Repeat times: {repeat_times}")
foridxinrange(repeat_times):
breaking_md+=list_to_repeatprint("\nTesting breaking markdown...")
print(f"Number of lines in breaking markdown: {len(breaking_md.splitlines())}")
try:
mistune.create_markdown(renderer="ast")(breaking_md)
exceptRecursionErrorasexc:
print(f"RecursionError: {exc}")
non_breaking_md=""foridxinrange(repeat_times):
non_breaking_md+=list_to_repeatnon_breaking_md+="\n---\n"print("\nTesting non-breaking markdown...")
print(f"Number of lines in non-breaking markdown: {len(non_breaking_md.splitlines())}")
mistune.create_markdown(renderer="ast")(non_breaking_md)
print("No error")
Output:
Recursion Limit: 1000
Repeat times: 260
Testing breaking markdown...
Number of lines in breaking markdown: 1040
RecursionError: maximum recursion depth exceeded while calling a Python object
Testing non-breaking markdown...
Number of lines in non-breaking markdown: 1560
No error
Here is the stacktrace when the recursion error is hit
I set the recursion limit to 100 so I can get a smaller full stacktrace. The stacktrace is similar with limit 1000.
Traceback (most recent call last):
File "/Users/ketanbhatt/Library/Application Support/JetBrains/PyCharm2023.1/scratches/scratch_3.py", line 24, in <module>
mistune.create_markdown(renderer="ast")(breaking_md)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/markdown.py", line 110, in __call__
return self.parse(s)[0]
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/markdown.py", line 85, in parse
self.block.parse(state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 446, in parse
end_pos = self.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 150, in _parse_list_item
end_pos = block.parse_method(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/core.py", line 168, in parse_method
return func(m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/block_parser.py", line 379, in parse_list
return parse_list(self, m, state)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 63, in parse_list
groups = _parse_list_item(block, bullet, groups, token, state, rules)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 91, in _parse_list_item
text, continue_width = _compile_continue_width(text, leading_width)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/list_parser.py", line 205, in _compile_continue_width
text = expand_tab(text)
File "/Users/ketanbhatt/.venv/lib/python3.8/site-packages/mistune/util.py", line 18, in expand_tab
return _expand_tab_re.sub(repl, text)
RecursionError: maximum recursion depth exceeded while calling a Python object
Thank you for your work : )
The text was updated successfully, but these errors were encountered:
Hello 👋🏼
I am using
mistune==3.0.1
.When I try to parse a markdown that is one big list of ordered and unordered list, mistune runs into Recursion Error. I suspect it might be taking it as one big nested list.
Here is a sample markdown:
If this markdown continued like this to have more lines than the
sys.getrecursionlimit()
, parsing this markdown would result in a recursion error.Here is some code that reproduces the error:
Output:
Here is the stacktrace when the recursion error is hit
I set the recursion limit to
100
so I can get a smaller full stacktrace. The stacktrace is similar with limit 1000.Thank you for your work : )
The text was updated successfully, but these errors were encountered: