Helix doesn't maintain BOM

[cryocz]

Summary

When a file with the UTF-8 Byte Order Mark (initial byte sequence EF BB BF) is opened, it is not preserved on save. While this generally isn't a major issue, it does mean any file changed with hx will be marked as changed by git, for example, even if you've made no character-wise changes.

Reproduction Steps

I tried this:

1. printf '\xEF\xBB\xBFtest\n' > bom-file
2. make sure file bom-file reports bom-file: Unicode text, UTF-8 (with BOM) text
3. hx bom-file and :wq immediately afterward

I expected this to happen:
file bom-file reports bom-file: Unicode text, UTF-8 (with BOM) text once again, for the BOM is maintained

Instead, this happened:
file bom-file reports bom-file: ASCII text, for the BOM is gone

Helix log

No response

Platform

Linux 6.1.14-100.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Sun Feb 26 00:31:11 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Terminal Emulator

kitty 0.26.5

Helix Version

helix 22.12

[kirawi]

Related to #4533
encoding_rs strips the BOM during decode and doesn't offer a way to encode it.

[Alexis-Lapierre]

I have made a small test branch to conserve BOM on files here master...Alexis-Lapierre:helix:wip_conserve_bom_on_file_save.

I had to shoehorn in the feature in, but it's definitely something that's possible.

I'll try and work on it for it to be clean enough to be merged.