fix: should only fallback on parse error

[JounQin]

close #334

[codesandbox-ci]

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

[codecov-commenter]

Codecov Report

Merging #335 (a219965) into master (442cc02) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master      #335   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           21        21           
  Lines          510       510           
  Branches       106       110    +4     
=========================================
  Hits           510       510           

Impacted Files	Coverage Δ
packages/eslint-mdx/src/traverse.ts	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 442cc02...a219965. Read the comment docs.

[wooorm]

Seems to solve the reported issue so 👍 from me