Invalid ANSI escape code insertion for log files with existing multi-attribute escape codes

[zaneduffield]

What steps will reproduce the bug?

1. On Windows (cygwin / Git Bash) run

echo -e '[\e[1;34mINFO\e[0m]' | bat -l log

What happens?
bat attempts to add colour to 'INFO' but inserts the new ANSI escape codes in the middle of the existing escape codes.
In my example, the new escape codes start directly after the first ESC[, which corrupts the escape sequence.

It looks like this:

The issue does not occur when I replace the multi-attribute escape sequence \e[1;34m with a single-attribute escape sequence like \e[34m:

What did you expect to happen instead?
I expect that bat would insert the new escape codes directly before the first character of INFO rather than in the middle of the preceding escape sequence.
I'm aware that adding highlighting to files which already contain escape code isn't exactly a supported use case (garbled output), but this issue seems quite avoidable.

I would like this to work because I often have log files which already contain colour escape codes (e.g. on the log level word) that I would like to view with bat.

I should also note that I couldn't reproduce the issue in my Ubuntu WSL environment.

How did you install bat?

Latest version using chocolatey.

bat version and environment

Software version

bat 0.21.0 (405e5f7)

Operating system

Windows 6.2.9200

Command-line

'C:\ProgramData\chocolatey\lib\Bat\tools\bat-v0.21.0-x86_64-pc-windows-msvc\bat.exe' -l log --diagnostic 

Environment variables

SHELL='C:\Program Files\git\usr\bin\bash.exe'
PAGER=<not set>
LESS=<not set>
LANG=en_US.UTF-8
LC_ALL=<not set>
BAT_PAGER=<not set>
BAT_CACHE_PATH=<not set>
BAT_CONFIG_PATH=<not set>
BAT_OPTS=<not set>
BAT_STYLE=<not set>
BAT_TABS=<not set>
BAT_THEME=<not set>
XDG_CONFIG_HOME=<not set>
XDG_CACHE_HOME=<not set>
COLORTERM=<not set>
NO_COLOR=<not set>
MANPAGER=<not set>

Config file

Could not read contents of 'C:\Users\zane\AppData\Roaming\bat\config': The system cannot find the path specified. (os error 3).

Custom assets metadata

Could not read contents of 'C:\Users\zane\AppData\Local\bat\metadata.yaml': The system cannot find the path specified. (os error 3).

Custom assets

'C:\Users\zane\AppData\Local\bat' not found

Compile time information

• Profile: release
• Target triple: x86_64-pc-windows-msvc
• Family: windows
• OS: windows
• Architecture: x86_64
• Pointer width: 64
• Endian: little
• CPU features: fxsr,sse,sse2
• Host: x86_64-pc-windows-msvc

Less version

> C:\Program Files\git\usr\bin\less.exe --version 
less 563 (PCRE regular expressions)
Copyright (C) 1984-2020  Mark Nudelman

less comes with NO WARRANTY, to the extent permitted by law.
For information about the terms of redistribution,
see the file named README in the less distribution.
Home page: http://www.greenwoodsoftware.com/less

[keith-hall]

Thanks for reporting. I believe this issue may get solved when #2189 is merged, but unfortunately I don't have time to confirm right now.

[zaneduffield]

Thanks for the suggestion. I just pulled and compiled that branch and I get the same result.
It does seem like it could be related though.

[zaneduffield]

I just realised that the issue might be more about bat trying to add colour to the 1 in my example (which it does do successfully).
I guess what I would like is for bat to avoid inserting escape codes which markup existing escape codes. I don't know how the implementation works, but maybe it can just keep track while parsing of whether there is an escape sequence being expressed?

It may not be that simple though, since the issue doesn't occur on linux for me; some platform-dependency is at play.

[Enselic]

Thank you for reporting. Like you mention, syntax highlighting for files with ANSI escape codes is out-of-scope for bat though. It is never going to work, because syntax regexes are not written to take ANSI escape characters into account. If we fix your specific case, we are going to break some other case. Properly pre-processing ANSI escape characters is a huge task, with very small benefits, which complicates bug fixing and feature development. It is, by my estimation, not worth it.

So I propose we close this issue as "won't fix".

[zaneduffield]

Fair enough, thanks for the explanation.