Question regarding elasticsearch indexing

[pulse00]

The documentation says there's support for elasticsearch indexing of stored content on the roadmap - the code itself seems to already have elasticsearch logic in place.

What's the status on this? Would you accept help to advance this further if needed? We're currently investigating our possible approache to indexing. Since we're already using spring-content-s3 and elasticsearch, combining the 2 pieces would be logic step for us :)

[paulcwarren]

Hi, @pulse00 there is partial code for elasticsearch. It got started soon after we added support for solr but never got finished as no one was asking for it at that time. If you would like to use it then we can bump the priority and pick it up in the next release. If I recall there isn't too much left to do. If you feel brave enough also feel free to fork the code and have a stab at it yourself. We would gladly accept a PR for that.

Are you planning to use Spring Data Elasticseach for storing the content metadata as well as for fulltext search?

[pulse00]

Are you planning to use Spring Data Elasticseach for storing the content metadata as well as for fulltext search?

Yes. in fact we've currently migrated to spring-data-elasticsearch 3.2.0-M1 which has finally support for the new high level rest client built into elasticsearch-java.

We sure have resources to work on that indexing feature in the upcoming weeks, however i've noticed that your existing codebase uses jest for communication with elasticsearch.

As we've just put some considerable effort into removing jest from our codebase (we assume that the native elasticsearch rest client will be the way to go in the future), i'm wondering what's the plan regarding jest / native rest client for spring-data-content before we start to work on a PR.

[paulcwarren]

I also assume the native client will be the best choice moving forwards so happy to switch over to the same dependencies as spring-data-elasticsearch 3.2.0-M1 in order to get that new client libraries. I would hope these will just become spring boot managed dependencies, if they aren't already.

One other point before you cut any code. Spring Content is a community project atm but our intention is that it will eventually be donated to Spring proper. To that end, we are managing contributions through standard CLAs. You would, therefore, have to sign a CLA before any PRs could be accepted. The sole intention of this is to ease the transition in the relevant Spring GitHub org should the Spring team see enough value to want to bring it in.

[pulse00]

You would, therefore, have to sign a CLA before any PRs could be accepted.

Sure, i would have expected that anyway :)

I'll discuss when we can start to work on this with my team next week and get back to you as soon as i have new info!

[paulcwarren]

Hi @pulse00,

Checking back in on this. It's been a while and I was wondering what you folks decided to do with this in the end. I have at least one other customer requiring the Spring Content Elasticsearch module to support fulltext indexing and in their case Versioning too. So, if you didn't pick this up I plan to do so for the next release (0.9.0).

If you did pick it up then would an extra pair of hands be useful?

Thanks
_Paul

[pulse00]

@paulcwarren sorry for the late reply - unfortunately we did not have much focus on this due to lack of resources - what is your timeframe for this?

we would most likely start with this mid/end june.

[paulcwarren]

Thanks for getting back to me. I know how it is re lack of resources!

Almost there with the first cut of support for full-text indexing with elasticsearch. Should be in a position to cut 0.9.0 by the end of next week, if not a little before. So, if you have something kicking off mid to end June I think this is fine and we will be done well in time.

[paulcwarren]

As discussed 0.9.0 adds basic support for elasticsearch full-text indexing and search. When you get to it please let me know if this meets all of your requirements, or not. In the meantime, I am going to close this issue. Feel free to re-open if there is anything on this thread that you wish to continue discussing. Thanks for raising this. I think this is an excellent addition to the project.