Bad span computations with unicode characters, should be handling them as graphemes

[huonw]

use std::io;

fn main() {
    let s = ~"ZͨA͑ͦ͒͋ͤ͑̚L̄͑͋Ĝͨͥ̿͒̽̈́Oͥ͛ͭ!̏"; while true { break; }
    io::println(s);
}

main.rs:4:46: 4:67 warning: denote infinite loops with loop { ... }, #[warn(while_true)] on by default
main.rs:4     let s = ~"ZͨA͑ͦ͒͋ͤ̚L̄͑͋Ĝͨͥ̿͒̽̈́Oͥ͛ͭ!̏"; while true { break; }
                                                       ^~~~~~~~~~~~~~~~~~~~~

The proper fix for this requires grapheme handling (#7043), e.g. some graphemes are double width.

[huonw]

If #12489 lands, the compiler handles these slightly better, but it's just operating on codepoint counts and assuming they're all single width: i.e. still needs to be changed to work with graphemes.

[pzol]

Visiting for triage, still requires #7043

[alexcrichton]

#7043 has been fixed, so this can actually make progress now!

[ftxqxd]

Because of #24428, this issue should be reopened.

[pnkfelix]

I agree with @huonw's comment on PR #24428: namely, we should revert the portions of that PR that regressed rustc

[bltavares]

Triaging:

Updating the code to rustc 1.8.0-nightly (57c357d89 2016-02-16)

fn main() {
    let s = "ZͨA͑ͦ͒͋ͤ͑̚L̄͑͋Ĝͨͥ̿͒̽̈́Oͥ͛ͭ!̏"; while true { break; }
    println!("{}", s);
}

The error still does not point to the correct location:

<anon>:4:45: 4:66 warning: denote infinite loops with loop { ... }, #[warn(while_true)] on by default
<anon>:4     let s = "ZͨA͑ͦ͒͋ͤ͑̚L̄͑͋Ĝͨͥ̿͒̽̈́Oͥ͛ͭ!̏"; while true { break; }
                                                     ^~~~~~~~~~~~~~~~~~~~~

[Mark-Simulacrum]

Closing as fixed.

fn main() {
    let s = ~"ZͨA͑ͦL̄͑ĜͨOͥ͛!̏"; while true { break; }
}

Now gives:

error: expected expression, found `~`
 --> test.rs:2:13
  |
2 |     let s = ~"ZͨA͑ͦ͒͋ͤ͑̚L̄͑͋Ĝͨͥ̿͒̽̈́Oͥ͛ͭ!̏"; while true { break; }
  |             ^

error: aborting due to previous error

[Mark-Simulacrum]

Never mind, closed prematurely. The ~ was caught in an earlier pass, and since it's before the while true, I assumed this was fixed when it wasn't...

warning: unused variable: `s`
 --> test.rs:2:9
  |
2 |     let s = "ZͨA͑ͦ͒͋ͤ͑̚L̄͑͋Ĝͨͥ̿͒̽̈́Oͥ͛ͭ!̏"; while true { break; }
  |         ^
  |
  = note: #[warn(unused_variables)] on by default

warning: denote infinite loops with loop { ... }
 --> test.rs:2:45
  |
2 |     let s = "ZͨA͑ͦ͒͋ͤ͑̚L̄͑͋Ĝͨͥ̿͒̽̈́Oͥ͛ͭ!̏"; while true { break; }
  |                                             ^^^^^^^^^^^^^^^^^^^^^
  |
  = note: #[warn(while_true)] on by default

[euclio]

I'd like to work on this, but I could use some help. I've written a failing UI test, and I've pulled in the unicode-width crate to libsyntax. I'm having trouble finding the right place to change how the spans are calculated in the filemap. Should I be modifying bytepos_to_file_charpos...?

[est31]

Seems like a part of the issue is still present. See #47380.

[estebank]

@est31 this is a problem that might get introduced back every now and then by forgetting to test with unicode, but should only happen on new features (like the suggestions machinery that introduced the linked case).