Reline crash when loading Illegal string encoded differently from the current terminal

[hogelog]

Description

Reline crash when loading Illegal string encoded differently from the current terminal.
Readline is able to handle invalid strings as they are.

This crash is a realistic scenario, as it occurs when switching the terminal LANG setting and then loading and operating a past history file.

For example, irb, etc.

Expected

readline works well with Illegal string.

I am able to load and display history by typing up (↑) at the prompt (> ) and am able to handle it.

$ LANG=C ruby -rreadline -e 'Readline::HISTORY << "\xF0\x9F\x98\x80".force_encoding("utf-8"); puts Readline.readline("> ")'
> \360\237\230\200
😀

To use real readline (not reline), gem install readline-ext is required.

Actual

reline crashes with illegal string.

It crashes the moment I type up (↑) at the prompt (> ).

$ LANG=C ruby -Ilib/ -rreline -e 'Reline::HISTORY << "\xF0\x9F\x98\x80".force_encoding("utf-8"); puts Reline.readline("> ")'
> /Users/hogelog/repos/oss/ruby/reline/lib/reline/unicode.rb:125:in `encode': "\xF0" on US-ASCII (Encoding::InvalidByteSequenceError)
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline/unicode.rb:125:in `calculate_width'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline/line_editor.rb:1885:in `calculate_width'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline/line_editor.rb:2428:in `ed_prev_history'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline/line_editor.rb:1433:in `call'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline/line_editor.rb:1433:in `wrap_method_call'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline/line_editor.rb:1450:in `process_key'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline/line_editor.rb:1586:in `input_key'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline.rb:350:in `block (3 levels) in inner_readline'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline.rb:349:in `each'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline.rb:349:in `block (2 levels) in inner_readline'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline.rb:408:in `block in read_io'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline.rb:394:in `loop'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline.rb:394:in `read_io'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline.rb:347:in `block in inner_readline'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline.rb:345:in `loop'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline.rb:345:in `inner_readline'
        from /Users/hogelog/repos/oss/ruby/reline/lib/reline.rb:288:in `readline'
        from /Users/hogelog/.rbenv/versions/3.2.2/lib/ruby/3.2.0/forwardable.rb:240:in `readline'
        from -e:1:in `<main>'

This check was performed on
master branch (8937279)

Terminal Emulator

macOS
Terminal.app