fix RST parsing when no indent after enum.item (fix #17249) #17257

a-mr · 2021-03-04T21:22:50Z

According to RST spec this fragment

1. Nim is a statically typed compiled systems
programming language.

should be parsed as normal paragraph, not an enum.list, because there is no necessary indentation on the second line.

This PR makes rst.nim handle it in compliance with the spec and also adds a warning with suggestions how to fix it. Original python rst does not emit any warning but I think it's necessary since this behavior confuses people that are used to Markdown.

also change docs in 2 other places appropriately
fix that rst2html started numeration of lines from 0 - now it will be from 1 as in all text editors. It should have been done long ago. In this PR it's essential since human interpretation of the warning depends on proper line numbers

cc @narimiran

EDIT(timotheecour) fix #17249 (needed to put in body otherwise this PR won't show up in the ref'd issue: the link in title isn't enough) /cc @a-mr

compiler/docgen.nim

a-mr · 2021-03-05T19:07:00Z

Digging into line numbers reporting I observed that columns are also wrong but this time they are shifted by +1 (it was not clear because columns are much more often reported completely incorrectly because of more p.idx movement sensitivity).

It's because Nim global error/warning reporting adds ColOffset=1, while there is no such addition for lines. So before rst was parsed with initial line=0, column=1 but it should be line=1, column=0.

timotheecour · 2021-03-05T19:09:51Z

let's followup in timotheecour#638 for how to solve this aspect cleanly throughout nim compiler in followup work

lib/packages/docutils/rst.nim

lib/pure/strutils.nim

lib/packages/docutils/rstgen.nim

lib/packages/docutils/rst.nim

a-mr · 2021-03-10T16:11:10Z

CI failure is unrelated.

Test improvements will be done in a follow-up PR.

lib/packages/docutils/rst.nim

lib/packages/docutils/rstgen.nim

timotheecour · 2021-03-10T22:05:55Z

lib/packages/docutils/rst.nim

  let col = currentTok(p).col
  var w = 0
  while w < wildcards.len:
    if match(p, p.idx, wildcards[w]): break
    inc w
  assert w < wildcards.len
+  proc checkAfterNewline(p: RstParser, report: bool): bool =
+    let j = tokenAfterNewline(p, start=p.idx+1)
+    if p.tok[j].kind notin {tkIndent, tkEof} and


getting there, it's almost perfect...

another regression:

1. hello 1 1. hello 2 foo 2 1. hello 3 foo 3 1. sub1 1. sub2 1. sub3 zook5 1. hello 4 1. hello 5 1. hello 6 zook6 1. hello 7 1. hello 8 1. hello 9 1. hello 10 separator 1. hello 11 1. hello 12

before PR

after PR

sub3 gets displaced

there shouldn't be an interruption in numbering form 1. hello 5 to 1. hello 6

but this could potentially be addressed in future PR, wdyt ?

Everything is working as expected here.

sub3 is not displaced. It's just parsed as normal paragraph according to RST specification because there is zook5 without indent after it

the same here: "1. hello 6" was parsed as a normal paragraph and zook6 became a block quote because it's indented by 1

And the warning is issued in both cases, so it looks ok to me.

rst2html.py does the same thing if you change 1s to #s.

Also note that presence of auto-numerator 1 does not mean that it's a real "markdown mode".

I think it's possible to implement also markdown-style enumeration list parsing because it's basically just relaxation of the RST kind. But it would be another task.

timotheecour · 2021-03-10T22:38:15Z

nim-lang.Nim (packages Linux_i386) Failing after 58s — packages Linux_i386 failed
failed so i restarted it but it failed again (quickly, after 1mn)

can you rebase against devel maybe?

not sure if it relates to your PR

https://dev.azure.com/nim-lang/255dfe86-e590-40bb-a8a2-3c0295ebdeb1/_apis/build/builds/14075/logs/38

2021-03-10T21:10:24.0269335Z CC: stdlib_assertions.nim
2021-03-10T21:10:24.0288424Z CC: stdlib_dollars.nim
2021-03-10T21:10:24.1410956Z Error: execution of an external compiler program 'gcc -c  -w -fmax-errors=3  -I/home/vsts/work/1/s/lib -I/home/vsts/work/1/s -o /home/vsts/.cache/nim/koch_d/stdlib_assertions.nim.c.o /home/vsts/.cache/nim/koch_d/stdlib_assertions.nim.c' failed with exit code: 1
2021-03-10T21:10:24.1412125Z 
2021-03-10T21:10:24.1412666Z In file included from /usr/include/limits.h:25:0,
2021-03-10T21:10:24.1413573Z                  from /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed/limits.h:168,
2021-03-10T21:10:24.1414497Z                  from /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed/syslimits.h:7,
2021-03-10T21:10:24.1415407Z                  from /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed/limits.h:34,
2021-03-10T21:10:24.1416047Z                  from /home/vsts/work/1/s/lib/nimbase.h:268,
2021-03-10T21:10:24.1416680Z                  from /home/vsts/.cache/nim/koch_d/stdlib_assertions.nim.c:9:
2021-03-10T21:10:24.1417570Z /usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or directory
2021-03-10T21:10:24.1418154Z compilation terminated.
2021-03-10T21:10:24.1472789Z ##[error]Bash exited with code '1'.
2021-03-10T21:10:24.1576588Z ##[section]Finishing: Build koch

(refs #3123 (comment))
refs:

sudo apt-get install g++-multilib https://stackoverflow.com/questions/51765069/fatal-error-sys-cdefs-h-no-such-file-or-directory-while-compiling-openssl-1-1
libc6-dev-i386 https://itectec.com/ubuntu/ubuntu-fatal-error-sys-cdefs-h-no-such-file-or-directory/

timotheecour · 2021-03-10T23:03:04Z

good news, i have same error in another PR i just rebased (#17304) so this isn't related to your PR (=> issue #17325)

timotheecour

LGTM, remaining points can be addressed in followup work

timotheecour · 2021-03-12T05:04:22Z

(closed/re-opened so that bugfix #17327 is included)

…im-lang#17257)

…17895) * revive #16627 now that csources_v1 was merged * use dedent in rst.nim, refs #17257 (comment) * fix test and improve rendering of a rst warning

…ces_v1 (nim-lang#17895) * revive nim-lang#16627 now that csources_v1 was merged * use dedent in rst.nim, refs nim-lang#17257 (comment) * fix test and improve rendering of a rst warning

timotheecour reviewed Mar 5, 2021

View reviewed changes

compiler/docgen.nim Outdated Show resolved Hide resolved

timotheecour mentioned this pull request Mar 5, 2021

introduce Location type for line/col (maybe also type OneBasedIndex = distinct int for line numbers etc that start at 1) timotheecour/Nim#638

Open

timotheecour reviewed Mar 5, 2021

View reviewed changes

lib/packages/docutils/rst.nim Outdated Show resolved Hide resolved

This was referenced Mar 6, 2021

PR/issue review queue timotheecour/Nim#641

Open

rst.whichMsgClass is buggy: relies on $enum instead of $ast(enum) #17280

Closed

timotheecour reviewed Mar 8, 2021

View reviewed changes

lib/pure/strutils.nim Outdated Show resolved Hide resolved

timotheecour reviewed Mar 10, 2021

View reviewed changes

lib/packages/docutils/rstgen.nim Outdated Show resolved Hide resolved

timotheecour reviewed Mar 10, 2021

View reviewed changes

lib/packages/docutils/rst.nim Outdated Show resolved Hide resolved

timotheecour reviewed Mar 10, 2021

View reviewed changes

lib/packages/docutils/rst.nim Show resolved Hide resolved

a-mr added 7 commits March 10, 2021 23:29

fix RST parsing when no indent after enum.item

e82bb85

add workaround

1b60ab7

fix column reporting; fix directive line reporting

87b2633

use """ """

d866d71

add dedent + a workaround for it

eb7b5be

apply suggestion (delete unnecessary code)

6b4d476

revert dedent workarounds, address other comments

b0e20bc

timotheecour mentioned this pull request Mar 10, 2021

nim doc/rst2html output should show parsing warnings in generated html #17324

Open

timotheecour reviewed Mar 10, 2021

View reviewed changes

lib/packages/docutils/rstgen.nim Show resolved Hide resolved

fix regression

e13174c

a-mr force-pushed the rst-enumlist-continuation-after-nl branch from aa74473 to e13174c Compare March 10, 2021 21:08

timotheecour reviewed Mar 10, 2021

View reviewed changes

Trigger CI again

6e11cb8

timotheecour approved these changes Mar 10, 2021

View reviewed changes

timotheecour mentioned this pull request Mar 10, 2021

CI fails on azure 32 bit linux Linux_i386: sudo apt-fast install fails #17325

Open

a-mr mentioned this pull request Mar 11, 2021

Rst test check messages (fix #17280) #17338

Merged

timotheecour closed this Mar 12, 2021

timotheecour reopened this Mar 12, 2021

narimiran merged commit d97bf4f into nim-lang:devel Mar 12, 2021

timotheecour mentioned this pull request Mar 17, 2021

RST minor bugs #17340

Open

28 tasks

ringabout pushed a commit to ringabout/Nim that referenced this pull request Mar 22, 2021

fix RST parsing when no indent after enum.item (fix nim-lang#17249) (n…

aaef1b7

…im-lang#17257)

ardek66 pushed a commit to ardek66/Nim that referenced this pull request Mar 26, 2021

fix RST parsing when no indent after enum.item (fix nim-lang#17249) (n…

519a091

…im-lang#17257)

timotheecour added a commit to timotheecour/Nim that referenced this pull request Apr 29, 2021

use dedent in rst.nim, refs nim-lang#17257 (comment)

8732fb9

timotheecour mentioned this pull request Apr 29, 2021

close #16646; since now works with bootstrap nim post csources_v1 #17895

Merged

2 tasks

ringabout pushed a commit that referenced this pull request Dec 1, 2022

fix RST parsing when no indent after enum.item (fix #17249) (#17257)

b2d26fa

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix RST parsing when no indent after enum.item (fix #17249) #17257

fix RST parsing when no indent after enum.item (fix #17249) #17257

a-mr commented Mar 4, 2021 •

edited by timotheecour

Loading

a-mr commented Mar 5, 2021

timotheecour commented Mar 5, 2021

a-mr commented Mar 10, 2021

timotheecour Mar 10, 2021

a-mr Mar 10, 2021

a-mr Mar 10, 2021

timotheecour commented Mar 10, 2021 •

edited

Loading

timotheecour commented Mar 10, 2021 •

edited

Loading

timotheecour left a comment

timotheecour commented Mar 12, 2021

fix RST parsing when no indent after enum.item (fix #17249) #17257

fix RST parsing when no indent after enum.item (fix #17249) #17257

Conversation

a-mr commented Mar 4, 2021 • edited by timotheecour Loading

a-mr commented Mar 5, 2021

timotheecour commented Mar 5, 2021

a-mr commented Mar 10, 2021

timotheecour Mar 10, 2021

Choose a reason for hiding this comment

before PR

after PR

a-mr Mar 10, 2021

Choose a reason for hiding this comment

a-mr Mar 10, 2021

Choose a reason for hiding this comment

timotheecour commented Mar 10, 2021 • edited Loading

timotheecour commented Mar 10, 2021 • edited Loading

timotheecour left a comment

Choose a reason for hiding this comment

timotheecour commented Mar 12, 2021

a-mr commented Mar 4, 2021 •

edited by timotheecour

Loading

timotheecour commented Mar 10, 2021 •

edited

Loading

timotheecour commented Mar 10, 2021 •

edited

Loading