ReStructuredText Language Server

NOTE: This project is still in development

This LSP implementation is based on palantir/python-language-server.

Example Usage

The VS Code extension can be tried by cloning the repository, opening it in VS Code, then inside the editor, press F5. This will compile and run the extension in a new Extension Development Host window.

Outlines and Diagnostics

Auto-Completions

References

Code Blocks

Resources

• docutils
  • Overview of the Docutils Architecture
  • Docutils Element Reference
  • Docutils Transforms
  • Parsing Tutorial
• sphinx
  • Syntax Cheat Sheet
  • Sphinx build phases
• doc8 RST linter
  • very basic, only accepts docutils directives, doesn't check roles, etc
• vscode-restructuredtext
  • no intellisense
  • decent HTML viewer, but calls sphinx in separate process for each build (not cancelling the last!) and very quickly overloading CPU
  • used doc8 for linting (calling each time in a separate subprocess)
  • Issue for switching to LSP (2017!)
• python-jsonrpc-server
  • Used to implement LSP server protocol
• RST Parer for JS (incomplete)
• It would be ideal to store path/line/character mappings to links, references, etc in a lightweight, pure-python, database, for fast querying.
  • Note sphinx stores a pickled file of each doctree, with a mapping of document names to last updated time, stored in the app.env.all_docs, that is pickled to environment.pickle. I doubt this contains enough info for such querying though.
  • tinydb (last commit 4 days ago, 3.2K stars, no deps)
  • pickledb (last commit 4 days ago, 427 stars, no deps)
  • zodb (last commit month ago, 358 stars, 8 dependencies)
  • Flata (last commit 2017, 12 stars)
  • Comparison of pickledb/tinydb/zodb
• TexMate Grammar
  • Guide
  • VSCode Python
  • Regex Tutorial
  • Writing TexMate Grammar Blog
  • Multiline captures (not possible?)
    • microsoft/vscode-textmate#41
    • microsoft/vscode-textmate#57
  • LSP Semantic highlighting feature request
• Noted issues in the LSP repo
  • Get all Workspace documents
  • Progress indicators or coming in LSP version 3.15.0: Work Done Progress
    • Discussion: LSP-server readiness indicator

Docutils Transforms

Standard transforms from readers.standalone (with default_priority)

• transforms.references.Substitutions (220) Replace substitution references by their corresponding definition
• transforms.references.PropagateTargets (260) Propagate empty internal targets to the next element.
• transforms.frontmatter.DocTitle (320) Converts an initial section title to a document title
• transforms.frontmatter.DocInfo (340) Converts an initial field_list to docinfo
• transforms.frontmatter.SectionSubTitle (350) Converts children sections to subtitles
• transforms.references.AnonymousHyperlinks (440) Link anonymous references to targets
• transforms.references.IndirectHyperlinks (460) For targets that refer to other targets (via refname), replace with the final refuri (for external links) or a refid to the final target (for internal links)
• transforms.references.Footnotes (620) Assign numbers to autonumbered footnotes, and resolve links to footnotes, citations, and their references.
• transforms.references.ExternalTargets (640) Replace refname by refuri, for references to external targets
• transforms.references.InternalTargets (660) Replace refname by refid, for references to internal targets
• transforms.universal.StripComments (740) (from readers.Reader) Remove comment elements from the document tree
• transforms.universal.Decorations (820) (from readers.Reader) Populate a document's decoration element (header, footer)
• transforms.misc.Transitions (830) Move transitions (denoted by ----) at the end of sections up the tree. Send reporter.error if transition after a title, at the beginning or end of the document, and after another transition.
• transforms.universal.ExposeInternals (840) (from readers.Reader)
• transforms.references.DanglingReferences (850) Send reporter.info for any dangling references (including footnote and citation), and for unreferenced targets.

transforms.references.Substitutions

Replace substitution references by their corresponding definition

<paragraph>
    The
    <substitution_reference refname="biohazard">
        biohazard
        symbol is deservedly scary-looking.
<substitution_definition name="biohazard">
    <image alt="biohazard" uri="biohazard.png">

<paragraph>
    The
    <image alt="biohazard" uri="biohazard.png">
        symbol is deservedly scary-looking.
<substitution_definition name="biohazard">
    <image alt="biohazard" uri="biohazard.png">

transforms.references.PropagateTargets

Propagate empty internal targets to the next element.

<target ids="internal1" names="internal1">
<target anonymous="1" ids="id1">
<target ids="internal2" names="internal2">
<paragraph>
    This is a test.

<target refid="internal1">
<target anonymous="1" refid="id1">
<target refid="internal2">
<paragraph ids="internal2 id1 internal1" names="internal2 internal1">
    This is a test.

transforms.references.AnonymousHyperlinks

Link anonymous references to targets.

<paragraph>
    <reference anonymous="1">
        internal
    <reference anonymous="1">
        external
<target anonymous="1" ids="id1">
<target anonymous="1" ids="id2" refuri="http://external">

<paragraph>
    <reference anonymous="1" refid="id1">
        text
    <reference anonymous="1" refuri="http://external">
        external
<target anonymous="1" ids="id1">
<target anonymous="1" ids="id2" refuri="http://external">

transforms.references.IndirectHyperlinks

For targets that refer to other targets (via refname), replace with the final refuri (for external links) or a refid to the final target (for internal links).

External links:

<paragraph>
    <reference refname="indirect external">
        indirect external
<target id="id1" name="direct external" refuri="http://indirect">
<target id="id2" name="indirect external" refname="direct external">

<paragraph>
    <reference refname="indirect external">
        indirect external
<target id="id1" name="direct external" refuri="http://indirect">
<target id="id2" name="indirect external" refuri="http://indirect">

Internal links:

<target id="id1" name="final target">
<paragraph>
    <reference refname="indirect internal">
        indirect internal
<target id="id2" name="indirect internal 2" refname="final target">
<target id="id3" name="indirect internal" refname="indirect internal 2">

<target id="id1" name="final target">
<paragraph>
    <reference refid="id1">
        indirect internal
<target id="id2" name="indirect internal 2" refid="id1">
<target id="id3" name="indirect internal" refid="id1">

transforms.references.Footnotes

Assign numbers to autonumbered footnotes, and resolve links to footnotes, citations, and their references.

<paragraph>
    A labeled autonumbered footnote referece:
    <footnote_reference auto="1" id="id1" refname="footnote">
<paragraph>
    An unlabeled autonumbered footnote referece:
    <footnote_reference auto="1" id="id2">
<footnote auto="1" id="id3">
    <paragraph>
        Unlabeled autonumbered footnote.
<footnote auto="1" id="footnote" name="footnote">
    <paragraph>
        Labeled autonumbered footnote.

<paragraph>
    A labeled autonumbered footnote referece:
    <footnote_reference auto="1" id="id1" refid="footnote">
        2
<paragraph>
    An unlabeled autonumbered footnote referece:
    <footnote_reference auto="1" id="id2" refid="id3">
        1
<footnote auto="1" id="id3" backrefs="id2">
    <label>
        1
    <paragraph>
        Unlabeled autonumbered footnote.
<footnote auto="1" id="footnote" name="footnote" backrefs="id1">
    <label>
        2
    <paragraph>
        Labeled autonumbered footnote.

transforms.references.ExternalTargets

Replace refname by refuri, for references to external targets.

<paragraph>
    <reference refname="direct external">
        direct external
<target id="id1" name="direct external" refuri="http://direct">

<paragraph>
    <reference refuri="http://direct">
        direct external
<target id="id1" name="direct external" refuri="http://direct">

transforms.references.InternalTargets

Replace refname by refid, for references to internal targets.

<paragraph>
    <reference refname="direct internal">
        direct internal
<target id="id1" name="direct internal">

<paragraph>
    <reference refid="id1">
        direct internal
<target id="id1" name="direct internal">

transforms.misc.Transitions

Move transitions (denoted by --------) at the end of sections up the tree. Send reporter.error if transition after a title, at the beginning or end of the document, and after another transition.

<section>
    ...
    <transition>
<section>
    ...

<section>
    ...
<transition>
<section>
    ...

Sphinx Process

The Sphinx build phases and Sphinx core eventscan be summarised as:

config-inited(app,config)
builder-inited(app)
env-get-outdated(app, env, added, changed, removed)
env-before-read-docs(app, env, docnames)

for docname in docnames:
    env-purge-doc(app, env, docname)
    if not removed:
        source-read(app, docname, source)

        run parser (by default docutils.Parser, with sphinx configured roles/directives/nodes)
            parse to doctree
            apply transforms (by priority)
        doctree-read(app, doctree)

env-updated(app, env)
env-check-consistency(app, env)

for docname in docnames:
    apply post-transform (by priority)
    doctree-resolved(app, doctree, docname)

call builder

build-finished(app, exception)