Elasticsearch: Creating template using java


(Krushnat Khawale) #1

I want to create a custom template in elasticsearch as it by default analyses all fields.
I want only specific fields to be analysed and I want to do this using a java client.

Also where can I find required jar for the same?


(David Pilato) #2

Maven central


(Krushnat Khawale) #3

@dadoonet Thanks for the sooner reply.
Yeah visited maven central, Also downloaded it.

I am using ES 2.1.0 and the jar which is available there for 2.1.0 does not have the IndicesAdminClient
class also its just one jar.

So I've downloaded downloaded ES 2.2.0 which constist of number of jars there.
My question is,
will it work for 2.1.0?
and the second question is, are all jars are required to import?


(David Pilato) #4

Not sure about which jars you are referring to.

Here are the jars you basically need (2.1 branch):

$ mvn dependency:tree
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Elasticsearch: Core 2.1.3-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ elasticsearch ---
[INFO] org.elasticsearch:elasticsearch:jar:2.1.3-SNAPSHOT
[INFO] +- org.hamcrest:hamcrest-all:jar:1.3:test
[INFO] +- com.carrotsearch.randomizedtesting:randomizedtesting-runner:jar:2.1.16:test
[INFO] |  \- junit:junit:jar:4.11:test
[INFO] +- org.apache.lucene:lucene-test-framework:jar:5.3.1:test
[INFO] |  +- org.apache.lucene:lucene-codecs:jar:5.3.1:test
[INFO] |  \- org.apache.ant:ant:jar:1.8.2:test
[INFO] +- org.apache.httpcomponents:httpclient:jar:4.3.6:test
[INFO] |  +- org.apache.httpcomponents:httpcore:jar:4.3.3:test
[INFO] |  +- commons-logging:commons-logging:jar:1.1.3:test
[INFO] |  \- commons-codec:commons-codec:jar:1.6:test
[INFO] +- com.google.jimfs:jimfs:jar:1.0:test
[INFO] +- org.apache.lucene:lucene-core:jar:5.3.1:compile
[INFO] +- org.apache.lucene:lucene-backward-codecs:jar:5.3.1:compile
[INFO] +- org.apache.lucene:lucene-analyzers-common:jar:5.3.1:compile
[INFO] +- org.apache.lucene:lucene-queries:jar:5.3.1:compile
[INFO] +- org.apache.lucene:lucene-memory:jar:5.3.1:compile
[INFO] +- org.apache.lucene:lucene-highlighter:jar:5.3.1:compile
[INFO] +- org.apache.lucene:lucene-queryparser:jar:5.3.1:compile
[INFO] |  \- org.apache.lucene:lucene-sandbox:jar:5.3.1:compile
[INFO] +- org.apache.lucene:lucene-suggest:jar:5.3.1:compile
[INFO] |  \- org.apache.lucene:lucene-misc:jar:5.3.1:compile
[INFO] +- org.apache.lucene:lucene-join:jar:5.3.1:compile
[INFO] |  \- org.apache.lucene:lucene-grouping:jar:5.3.1:compile
[INFO] +- org.apache.lucene:lucene-spatial:jar:5.3.1:compile
[INFO] |  \- org.apache.lucene:lucene-spatial3d:jar:5.3.1:compile
[INFO] +- org.apache.lucene:lucene-expressions:jar:5.3.1:compile
[INFO] |  +- org.antlr:antlr-runtime:jar:3.5:compile
[INFO] |  +- org.ow2.asm:asm:jar:4.1:compile
[INFO] |  \- org.ow2.asm:asm-commons:jar:4.1:compile
[INFO] +- com.spatial4j:spatial4j:jar:0.5:compile
[INFO] +- com.vividsolutions:jts:jar:1.13:compile
[INFO] +- com.github.spullara.mustache.java:compiler:jar:0.8.13:compile
[INFO] +- com.google.guava:guava:jar:18.0:compile
[INFO] +- com.carrotsearch:hppc:jar:0.7.1:compile
[INFO] +- joda-time:joda-time:jar:2.8.2:compile
[INFO] +- org.joda:joda-convert:jar:1.2:compile
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.6.2:compile
[INFO] +- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.6.2:compile
[INFO] +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.6.2:compile
[INFO] |  \- org.yaml:snakeyaml:jar:1.15:compile
[INFO] +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.6.2:compile
[INFO] +- io.netty:netty:jar:3.10.5.Final:compile
[INFO] +- com.ning:compress-lzf:jar:1.0.2:compile
[INFO] +- com.tdunning:t-digest:jar:3.0:compile
[INFO] +- org.hdrhistogram:HdrHistogram:jar:2.1.6:compile
[INFO] +- commons-cli:commons-cli:jar:1.3.1:compile
[INFO] +- org.codehaus.groovy:groovy-all:jar:indy:2.4.4:compile
[INFO] +- log4j:log4j:jar:1.2.17:compile
[INFO] +- log4j:apache-log4j-extras:jar:1.2.17:compile
[INFO] +- org.slf4j:slf4j-api:jar:1.6.2:compile
[INFO] +- net.java.dev.jna:jna:jar:4.1.0:compile
[INFO] \- com.twitter:jsr166e:jar:1.1.0:compile

(Krushnat Khawale) #5

I've downloaded highlighted(RED) jar from MavenCentral, and there is only one jar with four classes.
But ES 2.2.0 jar contains all the jars you've listed above.


(Krushnat Khawale) #6

Also after googling a lot, I haven't found a single program that demonstrates using API completely.
So do you have any link of something which I can refer in my application, because I have gone through
documentation of API and it simple confuses, because snippest are explained, where a complete program will be useful.


(Krushnat Khawale) #7

May I request you to share the link to download... the above referenced jars.


(David Pilato) #8

No sorry. That's too much work. You have to search that by yourself.

That's the reason using a dependency management system like Maven or Gradle is highly recommended.
Because it's then just a few lines of xml code:

<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>2.1.1</version>
</dependency>

About the Java guide, have a look here: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/index.html


(Ivan Brusic) #9

Listen to David, it is far easier to use Maven or Gradle nowadays. You can
view the dependencies in the Maven repositories:

http://mvnrepository.com/artifact/org.elasticsearch/elasticsearch/2.2.0

Not every jar is needed under most circumstances for the client, but it is
best to have them.

Ivan


(Krushnat Khawale) #10

Thanks @Ivan and @dadoonet for helping out in this.

We are facing issue in searching data in indices. When I search in kibana for with multiword text or apply filter for a perticular column, it simple creates two different bars for each word in search text.
For example, data is
Google Inc. – 15 messages
Facebook Inc. – 25 messages
Linkedin Inc. – 33 messages

When we query the index for ASN fields, you are going to get 15 hits for Google, 25 hits for Facebook, 33 hits for Linkedin and 73 hits for Inc
And after googling we found that, by default it analyses all fields which we don't want.

As we are creating a solution along with ELK, and we have to demonstrate it by ASAP(tomorrow).
Going through API, to just create index with a template taking too much time and making us trial and error for each class and methods.

Can I expect a very simple java sample for creating an index with template?


(David Pilato) #11

Sorry but is that the same question?

You have a demo to build with ELK?

Have a look at http://david.pilato.fr/blog/2015/11/17/index-twitter-on-found/

Why do you want to create a Java App for this?

That being said, I wrote some demo code there: https://github.com/dadoonet/legacy-search

Might help.


(Ivan Brusic) #12

I would use templates, as well, templates. :slight_smile: You can define templates
beforehand and have them match any index whose name matches a pattern. You
can define the templates in JSON and just PUT the template via curl.

It is two steps, but I personally prefer to have the templates defined in
JSON so that I can have them in source control. There used to be file-based
templates, but those were removed (grrrr).

Ivan


(system) #13