UTF-8 (rather, any non-ASCII) evidently dropped/ignored

[kriskowal]

Steps to reproduce:

Parse, for example, 👨‍👩‍👧‍👦

Expected: constituent UTF-8 bytes of the four unicode characters, and the three unicode zero-width join characters, should all be send to Print.

Actual: handler.Print is not called. No other method appears to be called.

package main

import ansiterm "github.com/Azure/go-ansiterm"

func main() {
	parser := ansiterm.CreateParser("Ground", &handler{})
	parser.Parse([]byte("👨‍👩‍👧‍👦"))
}

type handler struct{}

func (h *handler) Print(b byte) error {
	panic("not implemented")
}

func (h *handler) Execute(b byte) error {
	panic("not implemented")
}

func (h *handler) CUU(int) error {
	panic("not implemented")
}

func (h *handler) CUD(int) error {
	panic("not implemented")
}

func (h *handler) CUF(int) error {
	panic("not implemented")
}

func (h *handler) CUB(int) error {
	panic("not implemented")
}

func (h *handler) CNL(int) error {
	panic("not implemented")
}

func (h *handler) CPL(int) error {
	panic("not implemented")
}

func (h *handler) CHA(int) error {
	panic("not implemented")
}

func (h *handler) VPA(int) error {
	panic("not implemented")
}

func (h *handler) CUP(int, int) error {
	panic("not implemented")
}

func (h *handler) HVP(int, int) error {
	panic("not implemented")
}

func (h *handler) DECTCEM(bool) error {
	panic("not implemented")
}

func (h *handler) DECOM(bool) error {
	panic("not implemented")
}

func (h *handler) DECCOLM(bool) error {
	panic("not implemented")
}

func (h *handler) ED(int) error {
	panic("not implemented")
}

func (h *handler) EL(int) error {
	panic("not implemented")
}

func (h *handler) IL(int) error {
	panic("not implemented")
}

func (h *handler) DL(int) error {
	panic("not implemented")
}

func (h *handler) ICH(int) error {
	panic("not implemented")
}

func (h *handler) DCH(int) error {
	panic("not implemented")
}

func (h *handler) SGR([]int) error {
	panic("not implemented")
}

func (h *handler) SU(int) error {
	panic("not implemented")
}

func (h *handler) SD(int) error {
	panic("not implemented")
}

func (h *handler) DA([]string) error {
	panic("not implemented")
}

func (h *handler) DECSTBM(int, int) error {
	panic("not implemented")
}

func (h *handler) IND() error {
	panic("not implemented")
}

func (h *handler) RI() error {
	panic("not implemented")
}

func (h *handler) Flush() error {
	return nil
}

[ncw]

I can confirm this problem. It happens for a single unicode character such as £ too.

[ncw]

Note also that some chinese characters cause an error to be reported by the parser "New state of 'nil' is invalid", eg replacing the above main with this one

func main() {
	parser := ansiterm.CreateParser("Ground", &handler{})
	n, err := parser.Parse([]byte("狐妖小红娘"))
	log.Print(n, err)
}

Gives

$ go run ansiterm.go 
2018/09/11 21:29:53 3 New state of 'nil' is invalid.

[ivandeex]

This issue celebrates a 3 years anniversary 🤘

[jstarks]

Yes. On the other hand, 6 days ago we celebrated the 5 year anniversary of a Windows release with built-in VT support.

So as painful as it is for users stuck on even older versions of Windows, it's unlikely that we'll make any significant improvements to this package. ☹️