Java client or plugin for custom indexer and searcher

Elastic Stack Elasticsearch
1

Hi

I am porting over an existing search system over to elastic search. We have
a few custom requirements for search. I played around with ES RESTful APIs
and then now I am trying out the java client for elastic search. I am
wondering how should i go about implementing all the custom logic, should i
use the java client or write it as a plugin ? I haven't read anything about
writing plugins yet.

Here are the two approaches i can think of, I am looking for recommendation
from the group.

  1. I would have a search service layer in between the front end and the ES
    cluster. The search service layer will implement the custom logic using the
    java api.
  2. Abstract out all the custom logic as a plugin and have the search
    service layer just communicate with the plugin through REST as it would
    communicate with the vanilla ES cluster.

With my current understanding, I feel that 2nd approach is better as
plugins have more customization capability compared to the java api. Please
advice.

Thanks
Srini

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a58c2ed2-e444-4d81-bfcb-d86219e9cee3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

I am curious to hear some thoughts on this question.

On Wednesday, April 9, 2014 4:05:52 PM UTC-7, Srinivasan Ramaswamy wrote:

Hi

I am porting over an existing search system over to Elasticsearch. We
have a few custom requirements for search. I played around with ES RESTful
APIs and then now I am trying out the java client for Elasticsearch. I am
wondering how should i go about implementing all the custom logic, should i
use the java client or write it as a plugin ? I haven't read anything about
writing plugins yet.

Here are the two approaches i can think of, I am looking for
recommendation from the group.

  1. I would have a search service layer in between the front end and the ES
    cluster. The search service layer will implement the custom logic using the
    java api.
  2. Abstract out all the custom logic as a plugin and have the search
    service layer just communicate with the plugin through REST as it would
    communicate with the vanilla ES cluster.

With my current understanding, I feel that 2nd approach is better as
plugins have more customization capability compared to the java api. Please
advice.

Thanks
Srini

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/da362235-3652-4d72-8080-d89f654ade8c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Without explaining what the custom logic is, it would be difficult to give
a recommendation.

Unless you need to modify the behavior of Elasticsearch, I would not go
down the plugin route. But once again, hard to say without details.

--
Ivan

On Thu, Apr 10, 2014 at 8:53 PM, Srinivasan Ramaswamy ursvasan@gmail.comwrote:

I am curious to hear some thoughts on this question.

On Wednesday, April 9, 2014 4:05:52 PM UTC-7, Srinivasan Ramaswamy wrote:

Hi

I am porting over an existing search system over to Elasticsearch. We
have a few custom requirements for search. I played around with ES RESTful
APIs and then now I am trying out the java client for Elasticsearch. I am
wondering how should i go about implementing all the custom logic, should i
use the java client or write it as a plugin ? I haven't read anything about
writing plugins yet.

Here are the two approaches i can think of, I am looking for
recommendation from the group.

  1. I would have a search service layer in between the front end and the
    ES cluster. The search service layer will implement the custom logic using
    the java api.
  2. Abstract out all the custom logic as a plugin and have the search
    service layer just communicate with the plugin through REST as it would
    communicate with the vanilla ES cluster.

With my current understanding, I feel that 2nd approach is better as
plugins have more customization capability compared to the java api. Please
advice.

Thanks
Srini

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/da362235-3652-4d72-8080-d89f654ade8c%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/da362235-3652-4d72-8080-d89f654ade8c%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQB41uF%3Dv2UK%2BUbDYNgi6Ty35HQ-MOLgjn0smt_Q4cs4pQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

My 2 cents: I vastly prefer the Java API. Much better performance than
REST, much better control over the many features, and while there is a lot
of code to write (yes, this is a benefit to me, as I love to write code)
there is also a lot more gained insight and learning. The Java API is very
sparsely documented IMHO and the Javadoc is bare-bones. But once I got into
it, I found that it was clean, rock-solid, and an absolute joy to use.

I use the TransportClient interface, and wrote my own simplified JSON
schema that I convert to ES mappings. I wrote my own command-line query and
update and bulk-load tools. I wrap these inside Bash scripts; the scripts
use a common set of utility scripts to set the classpath and other
configurations so that I am building, testing, and deploying with the
exact same set of jars. This is important! But if implemented with some
serious Computer Science thought blended with ease-of-use artistry, it's
well worth the effort. It means that I can deploy any ES update at any time
without waiting for some plug-in to catch up. And as an added benefit, it's
able to be built, tested, and deployed with one command; a true
"push-button" ease of use.

I add all of the ES jars in my classpath, and add the full Netty (because I
can't grok the subset that ES uses), the full Jackson 2 jars, and the LMAX
Disruptor. For production / QA servers, I write a thin HTTP REST server on
top of Netty, and talk ES on the back-end, but embed my business logic in
this server. Very high performance with a relatively tiny footprint.

The nice thing about the Java API is that it's much easier (for me, anyway)
to create complex queries and filters. While the JSON is clean, it's still
easy to muff a complex query. And the Java API lets me emit my constructed
queries as JSON, so it matches better with the on-line ES guides. And it
lets me create JSON queries that are preferred by this newsgroup and the
support folks. It's easier for me to understand the JSON that ES emits than
it is for me to construct the JSON that ES wants.

I don't know if this will help you or not; I hope that it does. This
newsgroup has been very helpful to me, and StackOverflow has also answered
a few key questions. Otherwise, I slogged through it on my own and ended up
with a code base that helps me use the features of ES that I need to easily
shine! Like any long-term investment, much effort up front means an easy go
of things in the long run.

Enjoy!

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e69c0c26-0219-4e06-b5a4-76a5edbc0ab2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.