[DATAES-220] Jest Implementation Created: 12/Dec/15 Updated: 31/Dec/20 Resolved: 17/Apr/19 |
|
Status: | Closed |
Project: | Spring Data Elasticsearch |
Component/s: | Repositories |
Affects Version/s: | 1.3.1 (Gosling SR1) |
Fix Version/s: | None |
Type: | Improvement | Priority: | Major |
Reporter: | Julien Roy | Assignee: | sothawo |
Resolution: | Won't Do | Votes: | 15 |
Labels: | None | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified |
Last updater: | Spring Issues |
Pull Request URL: | https://github.com/spring-projects/spring-data-elasticsearch/pull/144 |
Description |
Hi, I work actually on projet with Spring Data Elasticsearch and AWS Elasticsearch. I developed à minimal implementation of Spring Data ES backend by the Jest Library. Are you interested for integrate an Jest implementation into Spring Data Elasticsearch ? Julien. |
Comments |
Comment by Mohsin Husen [ 15/Dec/15 ] |
Thanks for contacting for this feature. There is a huge demand to use http port while using spring data elasticsearch library either stand alone or with spring boot. We would love to have JEST implementation in spring data elasticsearch. Please fill free to open a pull request. |
Comment by Julien Roy [ 16/Dec/15 ] |
Ok thanks, i will be starting to work on this PR. Do you have any whishes about ES version compatibility ? Indeed Jest maintain two version : 1.X.X : Compatible with ES 1.7 |
Comment by Mohsin Husen [ 22/Dec/15 ] |
We are actively working to support 2.x version. better to target 2.x as of now. |
Comment by Julien Roy [ 09/Jan/16 ] |
Hi, My first implementation is visible here : It's based on your branch : issue/ Currently it's a very conservative implementation ( no changes on existing classes ). I will work on another implementation with more factorization of code between ElasticsearchTemplate and my JestElasticsearchTemplate. I thinking on classes tree like that :
What do you think about that ? Thanks for you help. |
Comment by Petar Tahchiev [ 05/Feb/16 ] |
+1 On this.. I'm not sure why it's a minor. For me this is a blocker. |
Comment by Julien Roy [ 13/Feb/16 ] |
Petar, i agree with you. Mohsin, I would like to complete my work about this feature. Thanks a lot. |
Comment by Mohsin Husen [ 16/Feb/16 ] |
Thanks Julien We will review the PR soon. and we have working branch for 2.x now. https://github.com/spring-projects/spring-data-elasticsearch/tree/DATAES-211-ES2.0 |
Comment by Mohsin Husen [ 24/Feb/16 ] |
After talking to various Spring Data devs about this feature, all have one common suggestion To have separate ElasticsearchTemplate for Jest is ok, as you need to have java code which deals with http calls using http client. but to add one more mapper for GSON can be problematic, as Spring Data elasticsearch already have jackson for mapping in and out from elasticsearch. Also adding additional third party library we will be bound to releases of other libraries too. |
Comment by Robert Gründler [ 25/Feb/16 ] |
Thanks Julien, this looks great! We would like to test the jest support PR, is there |
Comment by Mohsin Husen [ 25/Feb/16 ] |
To add above, i have created a branch https://github.com/spring-projects/spring-data-elasticsearch/tree/DATAES-220 please propose a PR to this and we will accept and add to build pipeline so that we have stable version where end user can test the feature. |
Comment by Milan Agatonovic [ 25/Feb/16 ] |
I am very interested in JEST implementation merged. |
Comment by Robert Gründler [ 01/Mar/16 ] |
Julien, should the branch in your fork compile without errors? We're trying to push it in our private repo in the meanwhile for testing, but see some compilation errors, mostly regarding `scroll`: "(cannot infer type-variable(s) T |
Comment by Mohsin Husen [ 03/Mar/16 ] |
not possible to add it in next release(Hopper RC1), as we still don't have working branch. |
Comment by Robert Gründler [ 14/Mar/16 ] |
i've been looking into getting this branch to work here https://github.com/pulse00/spring-data-elasticsearch/tree/jest-implementation-2.0 . I'm not sure if the original author has still time to finish this PR - if not our company would like to offer help implementing the currently unimplemented parts (mainly faceted search what i've seen and some scroll-mapping issues). Julien, would you be ok with this? |
Comment by Julien Roy [ 14/Mar/16 ] |
Robert, Indeed, I did not have the time to work on this feature this weeks. At this times, two question will be on hold : 1) Did you make an abstract ElasticsearchTemplate for share code between NativeElasticsearchTemplate and JestElasticsearchTemplate or just duplicate like i have doing in the current implementation ? 2) The Jest client library come with GSON dependency and Spring Data ES use Jackson. How solve this problem ? I waiting for responses from Mohsin about this questions and continue my work after. |
Comment by Robert Gründler [ 15/Mar/16 ] |
Regarding the ElasticsearchTemplate: I think we might opt for a similar approach like spring-data-redis. There you have the option of multiple underlying clients without having to change the template: The main concepts there involve connection factory and a connection interface:
Maybe this is a viable approach? Regarding the gson/jackson issue: Re-Implementing jest doesn't sound very appealing to me, so maybe we can get in touch with the jest people and check the status of the gson refactoring? -robert |
Comment by Raunak Gupta [ 20/Mar/16 ] |
One of the reason why I moved to Spring Elastic Search was because it used Jackson for JSON parsing. Looks like GSON won't leave me alone |
Comment by Petar Tahchiev [ 30/Mar/16 ] |
Any news here? What is the bottleneck of merging this? |
Comment by Robert Gründler [ 20/Apr/16 ] |
I will have time this week to work on the pull request. Fyi: I'll use jest as-is (meaning with the jackson dependency). |
Comment by Robert Gründler [ 21/Apr/16 ] |
I have merged the latest spring-data-elasticsearch master branch into the jest fork, changes can be found here: https://github.com/pulse00/spring-data-elasticsearch/tree/feature/jest To finish this there are 2 options: Option #1: Keep a single implementation of ElasticsearchTemplate working with both the elasticsearch and the jest client This option would require some substantial refactoring to be done, most likely with breaking backwards compatibility. Here's a couple of examples:
Especially the *ResultMapper classes rely on the ES client API. If introducing breaking changes is ok for the core committers, i can work on a refactoring to
Option #2: Keep the current implementation (2 separate implementations of ElasticsearchOperations) There would be 2 implementations: 1. The existing ElasticSearchTemplate Existing project would not require any changes, but the new JestElasticSearchTemplate would have some methods Examples:
Any feedback on these 2 options would be greatly appreciated. |
Comment by Robert Gründler [ 21/Apr/16 ] |
Here's my pull request with option #2 from the above comment. https://github.com/spring-projects/spring-data-elasticsearch/pull/144 This commit requires the next patchlevel version of jest though, because of https://github.com/searchbox-io/Jest/pull/340. |
Comment by Julien Roy [ 30/Jun/16 ] |
Hi Moshin, Do you see any inconveniences to I package my JestElasticSearchTemplate in new and independent Github projet and in the Maven Central ? You can see the projet in this Github repository : https://github.com/VanRoy/spring-data-jest Thanks. |
Comment by Julien Roy [ 08/Jul/16 ] |
Hi Moshin, Finally, I have published the artefact on maven central. Do not hesitate to come back to me if any problems related to this publicated. Thank a lot. |
Comment by Amin Mohammed-Coleman [ 10/Jul/16 ] |
Is the Jest implementation the only planned implementation for talking to ES via Http? ES have released (alpha) of a RestClient. Will this be used as an alternative to Jest? |
Comment by Terence Mill [ 15/Jul/16 ] |
I read a notice on spring blog that there i now support for Jest . Does it mean this has been merged to a release ? Which one , because the most current on spring repo is still the old spring-data-elasticsearch 1.1.0 https://spring.io/blog/2016/07/12/this-week-in-spring-july-12th-2016 > The first release candidate of Spring Boot 1.4 is now available, packed with a lot of great things! This new release features, among other things, a unified @EntityScan for JPA, MongoDB, Neo4j, Couchbase and Cassandra, an auto-configured RestTemplateBuilder, support for pure REST client tests via @RestClientTest, support for Jest (the Elasticsearch REST client), and upgrades to Spring Integration 4.3, Spring AMQP 1.6, Spring REST Docs 1.1, MongoDB Java Driver 3 and more! Download the bits and kick the tires right now! |
Comment by Mohsin Husen [ 15/Jul/16 ] |
@Terence, blog you are referring to is the news about Spring Boot supporting Jest client, not in the actual spring data elasticsearch. @Amin : Yes once elasticsearch 5.x is released we are planning to support Native Rest client. |
Comment by Joseph Blair [ 09/Aug/16 ] |
@Mohsin: Are you planning to release Julien's Jest implementation in the meantime while waiting for Elasticsearch 5.x? Or is it being scrapped in favor of moving directly to the native REST client? Just wondering what all of this means in terms of how long we'll need to wait before this feature is actually usable in production. Also, would this driver be compatible with the AWS Elasticsearch Service, or does Amazon need to support 5.x first? |
Comment by Julien Roy [ 11/Aug/16 ] |
@Joseph In waiting of decision, you can already use my implementation ( https://github.com/VanRoy/spring-data-jest ). Julien. |
Comment by Fabio Maffioletti [ 03/Mar/17 ] |
@Julien what is the status of this? Is it planned to be integrate any time soon? btw I tried to use your existing implementation but there are (small) issues (already pointed out on Github) that prevent a correct functionality using the latest elastic search. When do you think you will release your library? Thanks |
Comment by Julien Roy [ 03/Mar/17 ] |
Hi @Fabio, In my opinion, I think that it will never be integrated in master Spring Data. In waiting you can also use my Implementation : https://github.com/VanRoy/spring-data-jest Julien. |
Comment by Petar Tahchiev [ 01/Jun/17 ] |
Any news here? Can someone please share their thoughts on this? Will it ever make it into spring-data-elasticsearch? What is the stopper here - elasticsearch 6.0.0-alpha1 is already out. |
Comment by Tobias Günther [ 25/Oct/18 ] |
It's a real bummer that there is no progress here. A missing native support is almost a bug, especially with some providers like AWS not offering access to the node ports of the cluster. Third party solutions are suitable as aworkaround but are not updated frequently enough. |
Comment by Don Wellington [ 28/Oct/18 ] |
Tobias Günther I think |
Comment by sothawo [ 03/Apr/19 ] |
Like Don Wellington said, the ElasicsearcRestTemplate implemented in |
Comment by sothawo [ 17/Apr/19 ] |
I resolved this as won't do, as the functionality is contained in the `ElasticsearchRestClient`, so this change is not needed any more. |
Comment by Spring Issues [ 31/Dec/20 ] |
Spring Data has moved from Jira to GitHub Issues. This issue was migrated to spring-projects/spring-data-elasticsearch#793. |