links fail for identifiers wrapped in backticks with pymdownx.inlinehilite enabled

[tlambert03]

This took me a while to figure out, and it's possible this is a "won't fix", but curious to hear your thoughts on this issue.

I use a number of autorefs in the following format:

[`some.identifier`][]

I'm not sure if that's officially supported, but it has always worked well for me, making properly hyperlinked text wrapped in <code>. I recently installed mkdocs-gallery which broke this behavior, and I eventually tracked it down to the addition of pymdownx.inlinehilite to the config. So, with the following config, the above autoref will fail:

site_name: My Docs
markdown_extensions:
  - pymdownx.inlinehilite
plugins:
  - mkdocstrings

I tracked that down to this line:

autorefs/src/mkdocs_autorefs/references.py

Lines 56 to 61 in a6e373b

	if re.search(r"[/ \x00-\x1f]", identifier):
	# Do nothing if the matched reference contains:
	# - a space, slash or control character (considered unintended);
	# - specifically \x01 is used by Python-Markdown HTML stash when there's inline formatting,
	# but references with Markdown formatting are not possible anyway.
	return None, m.start(0), end

if pymdownx.inlinehilite is not included in the config, identifier will equal 'some.identifier' at that line, re.search will not match, and the link will be created. If inlinehilight is included though, identifier will look something like '\x02wzxhzdk:1\x03', and the search will hit preventing the link.

Is this something that you can imagine a fix for? Or is the [`some.identifier`][] syntax just not supported?

thanks!

[oprypin]

[`some.identifier`][] syntax is supported, we'll try to find a fix for this. Thanks for the detailed report.

[oprypin]

Sadly the fix isn't something simple priority-based

pymdownx anyway uses the same priority as the standard one and that's not it
https://github.com/facelessuser/pymdown-extensions/blob/e6474b38703b45e3dc431d3c1a0b1f24d80ee7fa/pymdownx/inlinehilite.py#L7
https://github.com/Python-Markdown/markdown/blob/93054dd9f7e6e2f555537873a3ec76d99e82326a/markdown/inlinepatterns.py#L76

and we need the priority to be lower than that

autorefs/src/mkdocs_autorefs/references.py

Line 213 in a6e373b

priority=168, # Right after markdown.inlinepatterns.ReferenceInlineProcessor

the difference is that pymdownx stashes the html and the standard one doesn't

[oprypin]

Ah actually the built-in one also stashes things, and we are able to detect that one but not this one somehow

autorefs/src/mkdocs_autorefs/references.py

Lines 88 to 89 in a6e373b

	if INLINE_PLACEHOLDER_RE.fullmatch(identifier):
	identifier = self.unescape(identifier)

[pawamoy]

It seems to be because the first unescape reveals a second stashed item, this time stored in the HTML stash. We can retrieve the item from the stash, but it was stashed as a string, so we can't use .itertext() on it to easily get the text.

if INLINE_PLACEHOLDER_RE.fullmatch(identifier):
    identifier = self.unescape(identifier)
if match := HTML_PLACEHOLDER_RE.fullmatch(identifier):
    identifier = self.md.htmlStash.rawHtmlBlocks[int(match.group(1))]  # no unstash function that does this?
    ...  # how to get text?

For [pathlib.Path][], identifier becomes:

<span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span>

Should we load that into an Element tree again? Or use a regex to pick up what's between > and <?

identifier = "".join(re.findall(r">([^<>]+)<", identifier))

And finally we have to unescape HTML characters:

identifier = html.unescape(identifier)

[pawamoy]

OK that works well, except that our fix_refs post-processing regular expression stops at the first </span>, and messes up the HTML.

[oprypin]

Should we load that into an Element tree again? Or use a regex to pick up what's between > and <?

markupsafe striptags would fit perfectly for this task. It even unescapes as well

[oprypin]

Created #40 accordingly (sorry for the snipe)

[pawamoy]

No worries, I can add myself as a co-author when squashing (I've spent quite some time investigating and debugging 😅)