Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Enable Partial Flat_object Field Type #7136

Closed
mingshl opened this issue Apr 13, 2023 · 2 comments
Closed

[Enhancement] Enable Partial Flat_object Field Type #7136

mingshl opened this issue Apr 13, 2023 · 2 comments
Assignees
Labels
enhancement Enhancement or improvement to existing feature or request Search Search query, autocomplete ...etc v2.9.0 'Issues and PRs related to version v2.9.0'

Comments

@mingshl
Copy link
Contributor

mingshl commented Apr 13, 2023

Is your feature request related to a problem? Please describe.
Currently, Only root fields of a json object can be defined as a flat object. You cannot define an object that is part of another JSON object as a flat object because when a flat object it is parses to a string, the nested architecture of the leaves is lost.

Describe the solution you'd like
in flat_object, there are no subfields in mapping, but when querying, flat_object field mapper dynamically create a new flat_object field type for the subfields, if they are not existed in then mappings, then when the query finished, the flat_object field type will be pop out of stack. It does not add to mappings.

The idea solution is to make the dynamically flat_object type to remember the path before it gets partially flatten.

For example, for a sample issue document:

 {
  "issue": {
    "number": "123456",
    "labels": {
      "version": "2.1",
      "backport": [
        "2.0",
        "1.3"
      ],
      "category": {
        "type": "API",
        "level": "enhancement"
      }
    }
  }
}

Currently, we need to define the mapping from the root of JSON,

 PUT /test-index/
{
  "mappings": {
    "properties": {
      "issue": {
            "type": "flat_object"
          }
        }
    }
  }

In the enhancement, we can define part of the JSON to be other field type, such as double and keyword

ideal use case 1: issue.number to be keyword type, while issue.labels is a flat_object type.

 PUT /test-index/
{
  "mappings": {
    "properties": {
      "issue": {
        "properties": {
          "number": {
            "type": "keyword"
          },
          "labels": {
            "type": "flat_object"
          }
        }
      }
    }
  }
}

ideal use case 2: issue.number to be double type, while issue.labels is a flat_object type. Users can do aggregation the documents on issue.number. This can resolve the concerns that flat_object can not support numerical parsing operations, then the users can define part of the json to be type specific field type when aggregation, sorting are needed.

 PUT /test-index/
{
  "mappings": {
    "properties": {
      "issue": {
        "properties": {
          "number": {
            "type": "double"
          },
          "labels": {
            "type": "flat_object"
          }
        }
      }
    }
  }
}
@mingshl mingshl added the enhancement Enhancement or improvement to existing feature or request label Apr 13, 2023
@mingshl mingshl changed the title [Enhancement] Enable Partial Flat_object field type [Enhancement] Enable Partial Flat_object Field Type Apr 13, 2023
@mingshl mingshl added the Search Search query, autocomplete ...etc label Apr 14, 2023
@macohen macohen added v2.8.0 'Issues and PRs related to version v2.8.0' and removed untriaged labels Apr 17, 2023
@mingshl mingshl self-assigned this May 12, 2023
@msfroh msfroh added v2.9.0 'Issues and PRs related to version v2.9.0' and removed v2.8.0 'Issues and PRs related to version v2.8.0' labels May 25, 2023
@mingshl mingshl mentioned this issue Jun 9, 2023
6 tasks
@mingshl mingshl added the roadmap label Jul 7, 2023
@dreamer-89
Copy link
Member

@mingshl : Is this on track for 2.9 release ? If not, please update the labels accordingly

@mingshl
Copy link
Contributor Author

mingshl commented Oct 6, 2023

implemented in #7997 on 2.9

@mingshl mingshl closed this as completed Oct 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Search Search query, autocomplete ...etc v2.9.0 'Issues and PRs related to version v2.9.0'
Projects
Status: Done
Development

No branches or pull requests

4 participants