Fix code block highlighting

[2bndy5]

resolves #123

As per #123 (comment)

The highlight directive and highlight_language config option are still respected (in that order) when the doxygen \code cmd (or MD fenced blocks) do not specify the optional arg.

@code{.py}
# uses pygments' lexer for python
@endcode

@code
// uses pygments defaults for language highlighting (kinda like an auto-detect)
@endcode

[2bndy5]

I've checked and verified that using the the following still works to fix the issue

block = nodes.literal("", "", *nodelist)
if node.domain:
    block["language"] = node.domain

This might be more backwards compatible if the code-block had hyperlinked syntax.

EDIT: It seems that references within the code-blocks are rendered as strings despite the presence of a ref tag. I should raise this as an issue.

[michaeljones]

I've made some opinionated adjustments @2bndy5. I hope they're alright.

I would like us to have a basic example of this somewhere. Is there something in the test suite that already demonstrates this? I'm quite out of touch.

[michaeljones]

I can try to make one tomorrow if there isn't one. It'll probably help me understand it better.

[2bndy5]

I'm not sure. If any test/example includes a code block, then it should be demonstrated accordingly. I'll have to check because I can't remember all the examples in the test folders.

If I don't find one, then I'll likely just modify an existing one or add a new one.

[2bndy5]

well there isn't much in the doxygen test suite with code snippets, but I found this:
(compared to the live docs currently hosted on RTD)

BTW, I'm not a big fan of the theme used here for the docs because, among various other reasons, the code blocks' differently colored parts are hard to visually distinguish. The RTD theme has this problems as well. Browser extensions like "dark reader" can only help so much.

Discovered a new issue

There's also an instance where pygments fails to guess what syntax is used. This instance exists under the doxygen test suite for the python docstrings (ironic - I know) named pyexample.
link to live hosted docs on RTD for comparison
The fact that these docstrings are interpreted as literal code from XML seems to be a problem with doxygen. Locally with v1.9.3, doxygen's XML produces no literal code blocks from the pyexample test. I know the CI uses v1.8.17, but RTD might be using something older (possibly the same version if they updated their docker image to use Ubuntu 20.04).

---

I don't see anything else in the docs that is making use of these changes. All the snippets in the TinyXml lib's example docs seems to remain unaffected. Meaning, pygments is still trying to automatically guess the code blocks' syntax just as before.

The real feature of this PR makes use of code blocks that look like so (to doxygen parser):

@code{.cmake}
set(user_list A B C)
for(element in ${user_list})
    message(STATUS "Element is ${element}")
endfor()
@endcode

where pygment is passed whatever is in the curly brackets when doxygen correctly outputs that as part of the XML.

[2bndy5]

@michaeljones If you want me to add a demonstration of this PR's changes, would you prefer a new page explaining how to properly specify a code-block's syntax (before running doxygen)? Or would you just want a new (or modified) test example?

These changes are also applied to MD fenced code-blocks.
This

```Dockerfile
RUN sudo apt-get install doxygen
```

is equivalent to

@code(Dockerfile)
RUN sudo apt-get install doxygen
@endcode

[michaeljones]

Thanks for the replies @2bndy5. I dislike burdening you when you've already contributed a lot but ideally we'd have some kind of demonstration page in the docs so that we can at least track breaking changes to it in some vague way. I used to diff the html output of master vs a PR in order to double check any differences. Bit out of the habit now but always good to have some version of it in the docs or test pages.

So something either here: https://breathe.readthedocs.io/en/latest/specific.html

Or on a new page in the test pages section? https://breathe.readthedocs.io/en/latest/testpages.html

[michaeljones]

Ideally the latter actually, I think the 'Specific Examples Test Suite' is becoming a bit too much of a dumping ground.

[michaeljones]

@2bndy5 - I can have a look at doing it myself. It'll do me good. Give me 30 mins to try.

[2bndy5]

No problem. As an avid documentation reader myself, I can appreciate when a specific feature has a devoted doc to describe how it works or doesn't work.

In brainstorming this doc's addition, I was imagining a MD file used to generate the new page (via autodoxygenpage).

[2bndy5]

It might be worth mentioning that inter-referenced hyperlinked text within code-blocks (as seen using doxygen's HTML output) is impossible because sphinx uses pygments for syntax highlighting. I could go on about the highlighting limitations in doxygen's HTML output due to doxygen's home-brewed mechanism (instead of jshighlight or pygments)...

[2bndy5]

Oops

I double checked my cmake code block for correctness. The cmake block should be written as

set(user_list A B C)
foreach(element ${user_list})
    message(STATUS "Element is ${element}")
endforeach()

[michaeljones]

I've taken a crack at it. I'm happy for you to add anything you'd like. I realise my understanding is not complete. Does it try to highlight all code blocks as C/C++ unless told otherwise?

[2bndy5]

Does it try to highlight all code blocks as C/C++ unless told otherwise?

I'm fuzzy on the XML's terminology here. The language of the documented compounddef (if provided by doxygen) is used as the domain passed to pygments. otherwise pygments should be told to guess the syntax. As I stated before, if an arg is passed to doxygen's \code command (this is meant to be a filename according to doxygen's manual), then that arg is first used by pygments' API to get the proper alias of the associated syntax. This arg can be used to supersede the syntax used for non C++ code blocks embedded within a C++ structure's docs.

/**
 * This is a C++ member's docs.
 * @code
 * // This is a C++ snippet that is highlighted automatically
 * @endcode
 * 
 * @code{.py}
 * # This is a python snippet that is highlighted appropriately
 * @endcode
 *
 * @code{.txt}
 * This snippet specifically has no syntax highlighting. It could be mistaken by pygments as
 * a CMakeLists.txt if the syntax contained within looks correct.
 * @endcode
 */
void someFunc();

[michaeljones]

I think I'm happy to merge now if you are. Maybe I should check it more closely. We can merge tomorrow unless anything comes up.

[2bndy5]

I'm looking it over now. I wouldn't mind if you want to let this incubate for another day. I'm going try and make this new doc as explicit as possible without confusing people (lead by example approach).

[2bndy5]

I'm happy with the new doc.

In the future, I plan to submit a PR that will significantly reduce the errors/warnings output during docs build (which is currently up to 501). I just don't want to flood you guys with any more PRs right now.

[michaeljones]

Great additions to the documentation. Thank you for taking the time.

And thanks for your patience and work to begin with!

[gmarull]

FYI #805