Warm and Cold Storage

[dnlchen]

Will the warm storage and cold storage which are based on object storage (e.g. AWS S3) be implemented in OpenSearch?

[AmiStrn]

Hi @dnlchen !
If it is a feature request please use the template.
For an open discussion the community is active on https://discuss.opendistrocommunity.dev/c/forking-elasticsearch-kibana/50

Please post your question there in order to refine the issue you wish to put in the roadmap.

Also, since it is kind of a roadmap question, check out the roadmap :)

[dblock]

Related, #740

[LuigiCerone]

As suggested by @reta in this comment of issue #1990, I'm adding a comment to this discussion.

It would be very useful to have a way to limit search to a particular tier or tiers (hot/warm/cold) with a filter in the query like the elastic _tier.

[jkowall]

Have a look at #1968

[reta]

@jkowall I think #1968 (durability) is quite different from this one (tiered storage) - but there is an intersection indeed

[jkowall]

@jkowall I think #1968 (durability) is quite different from this one (tiered storage) - but there is an intersection indeed

This is basically a redesign of the way that UltraWarm is designed, which was “outside” of the database. This new architecture would be part of the database and be more highly performant due to the distributed nature of the implementation. I have had this discussion with folks working on it, and this is, in fact, the first step towards a more modern design of UltraWarm.

[andrross]

I'm going to resolve this in favor of the new issues covering usage of remote storage in OpenSearch: #2578 #2900