input buffer is mess up if charset decode error

[panqingfeng]

sz maybe uninit if decode() is error

[elasticsearch-release]

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run.

[elasticsearch-release]

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run.

[ruflin]

@panqingfeng Thanks for the PR. Do you have by some log files we could test this a unit or system test?

[ruflin]

@panqingfeng Just saw you pushed an update. Any chance to share some log files do I can reproduce the issue on my side and we could create some system tests for it?

[panqingfeng]

I am not sure I can share the log to a 3rd party, I am so sorry...

Here is another way:

1. curl "https://www.google.com.hk/services/" | grep description > /tmp/out.txt
2. now you get a UTF-8 Chinese string in /tmp/out.txt
3. set filebeat encode to GBK (actually it's a UTF-8 string, so I will fail while decoding)

[ruflin]

@panqingfeng Thanks for the above guide. I will try this one will let you know if I can reproduce it.

[ruflin]

@panqingfeng I managed to reproduce based on the examples you gave above. I took your code, did some refactoring and added some tests here: #2661 Could you take a look?

[ruflin]

@panqingfeng I'm going to close this PR as we merged #2661 to backport it to 5.0. Thanks a lot for digging into this and providing a solution.