[ER] Line number for 'stream did not contain valid UTF-8' errors

[leonardo-m]

If by mistake I write code as this:

fn main() {
    println!("Hello ’world");
}

All I get from rustc is:

error: couldn't read ...\test.rs: stream did not contain valid UTF-8

But I'd like to receive a line number where such invalidity happens.

Using: rustc 1.48.0-nightly f3c923a13 2020-09-17 x86_64-pc-windows-gnu

[tesuji]

If rustc read source file with fs::read_to_string, that's maybe hard to report which line number contains invalid UTF-8.
Just my guess.

[leonardo-m]

To spot the problem I've used a Python script like:

for i, line in enumerate(open("mycode.rs")):
    if any(ord(c) >= 127 for c in line):
        print(i)

[SagnikHajra]

Also faced the same issue. I was using VS code.
>>> for i, line in enumerate(open("hello.rs")): ... print(line) ... ÿþfn main() {

No idea how did I end up with "ÿþ". Created a new file and it worked.

[Kevpopper]

I got same issue but i just changed .rs to .txt

[ms0503]

"ÿþ" is U+00FF and U+00FE in Unicode.
These are byte order mark (BOM) in little-endian UTF-16.

If you write your source code in UTF-8 without BOM, it should work.

[estebank]

#135557 will provide more information for these errors. For the case where the first byte of the "first" (entry) file is incorrect, we don't get a proper span to point at, so we customize the output to include the byte position in the message itself. It's not ideal, but it is better than what we had. I also encountered multiple places in our test suite that expect rust files to be valid utf-8 or crash 😄