Parsing fails with BadRoute exception

[marciof]

For these 3 Wikipedia articles: https://gist.github.com/marciof/a456f9c2df404db8b1f8

Traceback (most recent call last):
  (...)
  File "/home/marciof/py2.6/lib/python2.6/site-packages/mwparserfromhell/parser/__init__.py", line 62, in parse
    tokens = self._tokenizer.tokenize(text, context, skip_style_tags)
  File "/home/marciof/py2.6/lib/python2.6/site-packages/mwparserfromhell/parser/tokenizer.py", line 1149, in tokenize
    return self._parse(context)
  File "/home/marciof/py2.6/lib/python2.6/site-packages/mwparserfromhell/parser/tokenizer.py", line 1125, in _parse
    return self._handle_tag_close_close()
  File "/home/marciof/py2.6/lib/python2.6/site-packages/mwparserfromhell/parser/tokenizer.py", line 724, in _handle_tag_close_close
    self._fail_route()
  File "/home/marciof/py2.6/lib/python2.6/site-packages/mwparserfromhell/parser/tokenizer.py", line 141, in _fail_route
    raise BadRoute(context)
mwparserfromhell.parser.tokenizer.BadRoute

[earwig]

Looking into it...

[earwig]

Ugh, okay; this wikicode is a complete mess. The issue seems to be a result of a few factors coming together that appear to be mentioned in #40 and #42 already. It ultimately boils down to hitting the recursion limit and screwing up with handling the wikicode past that point. #42 will make the parser a lot faster, which would allow me to raise the recursion limits, so I'm hoping this sort of issue can be avoided through that.

[earwig]

We really need to come up with another system for handling HTML tags. They're very problematic.

[earwig]

Hopefully this should be fixed by the final solution to #40, but I'm leaving it open to verify that it is indeed fixed.

[lahwaacz]

To avoid duplicate reports, it might be a good idea to temporarily change the error message from This is a bug and should be reported. Info: C tokenizer exited with BAD_ROUTE. to something like

This is a known bug (#40 or #65). As a workaround, try to pass `skip_style_tags=True` in the meantime.
Info: C tokenizer exited with BAD_ROUTE.

As a bonus, you might check if skip_style_tags is already true.