Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] The index setting "ignore_malformed" is not working properly #4002

Closed
hauck-jvsh opened this issue Jul 25, 2022 · 5 comments
Closed

[BUG] The index setting "ignore_malformed" is not working properly #4002

hauck-jvsh opened this issue Jul 25, 2022 · 5 comments
Labels
bug Something isn't working Indexing & Search

Comments

@hauck-jvsh
Copy link
Contributor

Describe the bug
The "ignore_malformed" option should allow inconsistent data to be indexed, losing only the malformed fields, but it is ignoring the entire item.

To Reproduce

  1. Create an empty index with the setting "ignore_malformed": "true".
  2. insert the following data {
    "name":"test 1",
    "other_field":"string content"
    }
  3. insert the following data {
    "name":"test 2",
    "other_field":{"comment":"object content","size":2}
    }

Expected behavior
In this situation the second item should be indexed only the field "other_field" should be missed, but the hole item is lost.

Host/Environment (please complete the following information):

  • OS: centos 8
  • Version 2.1

Additional context
I'm using this to index a lot of unstructured data, and sometimes a few fields are invalid, but it shouldn't lose the valid fields.

@hauck-jvsh hauck-jvsh added bug Something isn't working untriaged labels Jul 25, 2022
@hauck-jvsh
Copy link
Contributor Author

Should I try to make a fix?

@dblock
Copy link
Member

dblock commented Jul 27, 2022

@hauck-jvsh Please!

hauck-jvsh added a commit to hauck-jvsh/OpenSearch that referenced this issue Sep 13, 2022
…gnore_malformed is set to true;

Signed-off-by: Hauck <joaoh14@gmail.com>
@hauck-jvsh
Copy link
Contributor Author

Sorry for the delay, but I finally get some time to work on this.

@hauck-jvsh
Copy link
Contributor Author

Should I close this?

@dblock
Copy link
Member

dblock commented Sep 16, 2022

Yep, assuming #4494 closes this.

@dblock dblock closed this as completed Sep 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Indexing & Search
Projects
None yet
Development

No branches or pull requests

3 participants