Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove mapping types #1139

Closed
dreamer-89 opened this issue Mar 2, 2022 · 2 comments
Closed

Remove mapping types #1139

dreamer-89 opened this issue Mar 2, 2022 · 2 comments

Comments

@dreamer-89
Copy link
Member

As part of v2.0 release, mapping types are getting removed from OpenSearch engine. Below are the changes in the opensearch-engine.

  • Removal of type from rest end-points
  • Removal of include_type_name parameter from API requests.

As part of this issue, please verify if type removal change on OpenSearch engine impacts this repository. If yes, then please remove the type references/usage from this plugin.

Related: OpenSearch-engine meta issue

@dreamer-89 dreamer-89 added untriaged breaking change Any change which may break existing configurations and deployments v2.0.0 labels Mar 2, 2022
@dlvenable dlvenable removed v2.0.0 breaking change Any change which may break existing configurations and deployments labels Mar 7, 2022
@dlvenable dlvenable added this to the v1.4 milestone Apr 12, 2022
@dlvenable dlvenable mentioned this issue Apr 12, 2022
4 tasks
@dlvenable
Copy link
Member

Data Prepper is not using mapping types.

The mapping files do not have types defined:

Search results are empty:
https://github.com/opensearch-project/data-prepper/search?q=include_type_name&type=code


Additionally, I verified the indices in an OpenSearch cluster do not have custom types.

Raw Span Index

GET otel-v1-apm-span/_search
{
  "query": {
    "match_all": {}
  },
  "fields": [
    "_type",
    "name"
  ],
  "_source": false
}

-->

{
  "took" : 200,
  "timed_out" : false,
  "_shards" : {
    "total" : 385,
    "successful" : 385,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "otel-v1-apm-span-000001",
        "_type" : "_doc",
        "_id" : "6f2556b503bcc3d9",
        "_score" : 1.0,
        "fields" : {
          "name" : [
            "HTTP GET"
          ]
        }
      },
      {
        "_index" : "otel-v1-apm-span-000001",
        "_type" : "_doc",
        "_id" : "69963075dd057520",
        "_score" : 1.0,
        "fields" : {
          "name" : [
            "HTTP GET /route"
          ]
        }
      },
      {
        "_index" : "otel-v1-apm-span-000001",
        "_type" : "_doc",
        "_id" : "5a4d73e300cb75e4",
        "_score" : 1.0,
        "fields" : {
          "name" : [
            "HTTP GET"
          ]
        }
      },
      {
        "_index" : "otel-v1-apm-span-000001",
        "_type" : "_doc",
        "_id" : "029fdd47e8ae0d26",
        "_score" : 1.0,
        "fields" : {
          "name" : [
            "HTTP GET"
          ]
        }
      },
      {
        "_index" : "otel-v1-apm-span-000001",
        "_type" : "_doc",
        "_id" : "63891eda74bcc48e",
        "_score" : 1.0,
        "fields" : {
          "name" : [
            "GetDriver"
          ]
        }
      },
      {
        "_index" : "otel-v1-apm-span-000001",
        "_type" : "_doc",
        "_id" : "25dc2b66de8bebe7",
        "_score" : 1.0,
        "fields" : {
          "name" : [
            "GetDriver"
          ]
        }
      },
      {
        "_index" : "otel-v1-apm-span-000001",
        "_type" : "_doc",
        "_id" : "2939dac89331b7fb",
        "_score" : 1.0,
        "fields" : {
          "name" : [
            "HTTP GET /route"
          ]
        }
      },
      {
        "_index" : "otel-v1-apm-span-000001",
        "_type" : "_doc",
        "_id" : "0adb641d390d01be",
        "_score" : 1.0,
        "fields" : {
          "name" : [
            "HTTP GET: /route"
          ]
        }
      },
      {
        "_index" : "otel-v1-apm-span-000001",
        "_type" : "_doc",
        "_id" : "69752061a95e81ab",
        "_score" : 1.0,
        "fields" : {
          "name" : [
            "HTTP GET: /route"
          ]
        }
      },
      {
        "_index" : "otel-v1-apm-span-000001",
        "_type" : "_doc",
        "_id" : "49c0702354edd5b0",
        "_score" : 1.0,
        "fields" : {
          "name" : [
            "HTTP GET: /route"
          ]
        }
      }
    ]
  }
}

Service Map Index

GET otel-v1-apm-service-map/_search
{
  "query": {
    "match_all": {}
  },
  "fields": [
    "_type",
    "name"
  ],
  "_source": false
}

-->

{
  "took" : 11,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 18,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "otel-v1-apm-service-map",
        "_type" : "_doc",
        "_id" : "u2t8FF1YHF4t/5Qa68XINw==",
        "_score" : 1.0
      },
      {
        "_index" : "otel-v1-apm-service-map",
        "_type" : "_doc",
        "_id" : "CGwFxCVlXd82ZTOmoJBHPg==",
        "_score" : 1.0
      },
      {
        "_index" : "otel-v1-apm-service-map",
        "_type" : "_doc",
        "_id" : "PfJI/obc/DiPDn8Z2BDgbQ==",
        "_score" : 1.0
      },
      {
        "_index" : "otel-v1-apm-service-map",
        "_type" : "_doc",
        "_id" : "f5ufxmZ1r8U2Xl2yq0OFrg==",
        "_score" : 1.0
      },
      {
        "_index" : "otel-v1-apm-service-map",
        "_type" : "_doc",
        "_id" : "hifEI5Vndn1Hrcttnbf0Ig==",
        "_score" : 1.0
      },
      {
        "_index" : "otel-v1-apm-service-map",
        "_type" : "_doc",
        "_id" : "OP/8YTM/rui5D131Dyl3uw==",
        "_score" : 1.0
      },
      {
        "_index" : "otel-v1-apm-service-map",
        "_type" : "_doc",
        "_id" : "9ym6PiJELiSSzSnGxq/fXQ==",
        "_score" : 1.0
      },
      {
        "_index" : "otel-v1-apm-service-map",
        "_type" : "_doc",
        "_id" : "4sQ0k4k7V5vsvCf1iJzdqQ==",
        "_score" : 1.0
      },
      {
        "_index" : "otel-v1-apm-service-map",
        "_type" : "_doc",
        "_id" : "+PJFfRyXBn3y4fl+OFD0aw==",
        "_score" : 1.0
      },
      {
        "_index" : "otel-v1-apm-service-map",
        "_type" : "_doc",
        "_id" : "VPnQIBP4WCnDhjxj1TEifg==",
        "_score" : 1.0
      }
    ]
  }
}

@dlvenable dlvenable removed this from the v1.4 milestone Apr 13, 2022
@dlvenable
Copy link
Member

I removed the 1.4 milestone. Data Prepper was not using mapping types prior to 1.4. I also checked the history and the index templates have no indication of ever using mapping types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

2 participants