Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow lexer to recover from some homoglyphs #62963

Merged
merged 2 commits into from
Jul 26, 2019
Merged

Allow lexer to recover from some homoglyphs #62963

merged 2 commits into from
Jul 26, 2019

Conversation

estebank
Copy link
Contributor

No description provided.

@rust-highfive
Copy link
Collaborator

r? @eddyb

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 24, 2019
@Centril
Copy link
Contributor

Centril commented Jul 25, 2019

r? @Centril

r=me rollup with nits fixed.

@rust-highfive rust-highfive assigned Centril and unassigned eddyb Jul 25, 2019
eddyb
eddyb reviewed Jul 25, 2019
View reviewed changes
src/test/ui/did_you_mean/issue-49746-unicode-confusable-in-float-literal-expt.stderr
@@ -14,5 +14,14 @@ help: Unicode character '−' (Minus Sign) looks like '-' (Minus/Hyphen), but it
LL | const UNIVERSAL_GRAVITATIONAL_CONSTANT: f64 = 6.674e-11; // m³⋅kg⁻¹⋅s⁻²
| ^

error: aborting due to 2 previous errors
error[E0277]: cannot subtract `{integer}` from `{float}`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I think we should do the substitution earlier, so it works in literals too, not just full tokens.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Unicode stuff is legal inside string literals, so I'm not sure there's a single earlier point where this can be done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's something I want to do (precisely for string literals) if and only if there is one unambiguous way of parsing it (like somebody copy-pasted code from a PDF and ends up with println!(‟Hello, world!”);, but doing so would require a more aggressive implementation with backtracking written in such a way that it doesn't affect the common case, and I don't see it would ever handling anything more complex than println!(‟It’s 💖”);. All of that I think should be done, but I'd want it to be part of a separate PR.

@Centril
Copy link
Contributor

Centril commented Jul 25, 2019

r? @eddyb

@rust-highfive rust-highfive assigned eddyb and unassigned Centril Jul 25, 2019
@eddyb
Copy link
Member

eddyb commented Jul 25, 2019

r? @petrochenkov

@rust-highfive rust-highfive assigned petrochenkov and unassigned eddyb Jul 25, 2019
@petrochenkov petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 25, 2019
@rust-highfive

This comment has been minimized.

Sign in to view
@estebank estebank force-pushed the homoglyph-recovery branch from cc5f7ab to 6844976 Compare July 25, 2019 19:37
@petrochenkov
Copy link
Contributor

@bors r+

@bors
Copy link
Contributor

bors commented Jul 25, 2019

📌 Commit 6844976 has been approved by petrochenkov

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 25, 2019
Centril added a commit to Centril/rust that referenced this pull request Jul 26, 2019
@Centril
Rollup merge of rust-lang#62963 - estebank:homoglyph-recovery, r=petr…
1aae896 
…ochenkov

Allow lexer to recover from some homoglyphs
@Centril Centril mentioned this pull request Jul 26, 2019
Closed
bors added a commit that referenced this pull request Jul 26, 2019
@bors
Auto merge of #63001 - Centril:rollup-up9baiv, r=Centril
f907629 
Rollup of 14 pull requests

Successful merges:

 - #62084 (allow clippy::unreadable_literal in unicode tables)
 - #62421 (Introduce `as_deref` to Option)
 - #62692 (rustc: precompute the largest Niche and store it in LayoutDetails.)
 - #62801 (Remove support for -Zlower-128bit-ops)
 - #62828 (Remove vector fadd/fmul reduction workarounds)
 - #62862 (code cleanup)
 - #62897 (Attempt to fix backtrace tests on i686-msvc)
 - #62904 (Disable d32 on armv6 hf targets)
 - #62907 (Initialize the MSP430 AsmParser)
 - #62956 (Implement slow-path for FirstSets::first)
 - #62963 (Allow lexer to recover from some homoglyphs)
 - #62970 (ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar)
 - #62983 (Remove needless indirection through Rc)
 - #62985 (librustc_errors: Support ui-testing flag in annotate-snippet emitter)

Failed merges:

r? @ghost
Centril added a commit to Centril/rust that referenced this pull request Jul 26, 2019
@Centril
Rollup merge of rust-lang#62963 - estebank:homoglyph-recovery, r=petr…
b8dadb0 
…ochenkov

Allow lexer to recover from some homoglyphs
@Centril Centril mentioned this pull request Jul 26, 2019
Closed
Centril added a commit to Centril/rust that referenced this pull request Jul 26, 2019
@Centril
Rollup merge of rust-lang#62963 - estebank:homoglyph-recovery, r=petr…
1893ac6 
…ochenkov

Allow lexer to recover from some homoglyphs
@Centril Centril mentioned this pull request Jul 26, 2019
Merged
bors added a commit that referenced this pull request Jul 26, 2019
@bors
Auto merge of #63015 - Centril:rollup-ydhpcas, r=Centril
c43753f 
Rollup of 22 pull requests

Successful merges:

 - #62084 (allow clippy::unreadable_literal in unicode tables)
 - #62120 (Add missing type links in documentation)
 - #62310 (Add missing doc links in boxed module)
 - #62421 (Introduce `as_deref` to Option)
 - #62583 (Implement Unpin for all raw pointers)
 - #62692 (rustc: precompute the largest Niche and store it in LayoutDetails.)
 - #62801 (Remove support for -Zlower-128bit-ops)
 - #62828 (Remove vector fadd/fmul reduction workarounds)
 - #62862 (code cleanup)
 - #62904 (Disable d32 on armv6 hf targets)
 - #62907 (Initialize the MSP430 AsmParser)
 - #62956 (Implement slow-path for FirstSets::first)
 - #62963 (Allow lexer to recover from some homoglyphs)
 - #62964 (clarify and unify some type test names)
 - #62970 (ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar)
 - #62980 (std: Add more accessors for `Metadata` on Windows)
 - #62983 (Remove needless indirection through Rc)
 - #62985 (librustc_errors: Support ui-testing flag in annotate-snippet emitter)
 - #63002 (error_index_generator should output stdout/stderr when it panics.)
 - #63004 (Add test for issue-54062)
 - #63007 (ci: debug network failures while downloading awscli from PyPI)
 - #63009 (Remove redundant `mut` from variable declaration.)

Failed merges:

r? @ghost
@bors bors merged commit 6844976 into rust-lang:master Jul 26, 2019
@petrochenkov petrochenkov mentioned this pull request Jul 28, 2019
Merged
@estebank estebank deleted the homoglyph-recovery branch November 9, 2023 05:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@petrochenkov petrochenkov petrochenkov left review comments

@matklad matklad matklad left review comments

@Centril Centril Centril left review comments

@eddyb eddyb eddyb left review comments

Assignees

@petrochenkov petrochenkov

Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@estebank @rust-highfive @Centril @eddyb @petrochenkov @bors @matklad