Attachment ingest processor: add resource_name field

[yangyaofei]

In the current plugin: ingest-attachment, the text file cannot be read properly if the encode is not utf-8
and contain some non-ascii characters.

I study a little about Tika which is used in ingest-attachment. Then I find out if we can tell Tika the file's name, it can recognize the file better. So I add an attachment options file_name, if there is a field defined as file_name, then this name will sent to Tika to improve the result.

But there is something not looks well. That's the gradle check. I wrote the unit test for reading different text using
different encoding. But seems there is a role to not commit no-utf8 things.

[elasticmachine]

Pinging @elastic/es-core-features (:Core/Features/Ingest)

[danhermann]

@elasticmachine ok to test

[danhermann]

But there is something not looks well. That's the gradle check. I wrote the unit test for reading different text using
different encoding. But seems there is a role to not commit no-utf8 things.

@yangyaofei, thank you for opening this PR. It's true that we have a check that prohibits files with encodings other than UTF8. Is that necessary for this PR? I believe that non-ascii characters are supported by the ingest-attachment plugin so long as they're encoded in UTF8.

[yangyaofei]

@danhermann hi, this PR is not for add non-utf8 file to the repo. it's a test file to test the main purpose , which is adding filename field to Tika through plugin to let tika recognize test file. For example. A GBK encoded text file cannot be recognized right . Before adding the filename to Tika, the encode can be recognized right

And To test that recognition, I add the non-utf8 test text file.

[danhermann]

@elasticmachine update branch

[danhermann]

@danhermann hi, this PR is not for add non-utf8 file to the repo. it's a test file to test the main purpose , which is adding filename field to Tika through plugin to let tika recognize test file. For example. A GBK encoded text file cannot be recognized right . Before adding the filename to Tika, the encode can be recognized right

And To test that recognition, I add the non-utf8 test text file.

@yangyaofei, can you try adding the following line:

exclude '**/text-cjk-*.txt'

right after

elasticsearch/plugins/ingest-attachment/build.gradle

Line 95 in d7dc93b

exclude '**/*.vsdx'

to exempt the new text files from the check that they're in UTF8 format?

[yangyaofei]

@danhermann Done

[yangyaofei]

@danhermann Is there anything I can do to merge this ? Or just wait ?

[danhermann]

@danhermann Is there anything I can do to merge this ? Or just wait ?

@yangyaofei, sorry for the delay. Now that it's compiling and passing the tests, I need to review the code. I'll try to get that done in the next week or so.

[danhermann]

@yangyaofei, thank you for your contribution here. It looks good and I think we can get it merged with a few minor changes as noted below.

[yangyaofei]

@danhermann done with that

[danhermann]

@elasticmachine update branch

[danhermann]

@danhermann done with that

Thank you, @yangyaofei. It looks good. I noticed one other thing about the tests that should be changed and then it can be merged. The suggestions below aren't ordered very well, but essentially, I would like to leave the existing unit tests unchanged so that they test the behavior of the processor when resource_name is unspecified. Your new tests would be added to a new test method that is the only one that specifies resource_name to test the behavior of the processor with the new option that you added.

[danhermann]

Great, the remaining test failures are unrelated, so I can get this merged. This is a very useful addition to the attachment processor, so thanks again for the contribution, @yangyaofei!

[fw-at-elastic]

Hey @yangyaofei! I'm Faith from Elastic's Community team. I want to encourage you to check out Elastic's Contributor Program, where you can earn points towards rewards for code contributions like this one.