ElasticSearch Cluster setup

Elastic Stack Elasticsearch
1

Hi All,

I am new on elasticsearch and want to setup it on distributed mode.

Please share any good article on the same.

Thanks,
Ankit

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi Ankit,

Elasticsearch IS a distributed search engine.

Just start new nodes on your network and Elasticsearch will detect them and do the rebalancing job for you.

You have many articles and video that talk about that. See: Elasticsearch Platform — Find real-time answers at scale | Elastic
Also, look at this page: Elasticsearch Platform — Find real-time answers at scale | Elastic

Does it help?

Le 29 janv. 2013 à 09:09, Ankit Jain ankitjaincs06@gmail.com a écrit :

Hi All,

I am new on elasticsearch and want to setup it on distributed mode.

Please share any good article on the same.

Thanks,
Ankit

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Thanks for reply ...

So, Whenever we add a new node in network using the same clustername, it
will automatically became a part of cluster.

Let me know, if i miss something

~Ankit

On Tuesday, 29 January 2013 13:49:40 UTC+5:30, David Pilato wrote:

Hi Ankit,

Elasticsearch IS a distributed search engine.

Just start new nodes on your network and Elasticsearch will detect them
and do the rebalancing job for you.

You have many articles and video that talk about that. See:
Elasticsearch Platform — Find real-time answers at scale | Elastic
Also, look at this page:
Elasticsearch Platform — Find real-time answers at scale | Elastic

Does it help?

Le 29 janv. 2013 à 09:09, Ankit Jain <ankitj...@gmail.com <javascript:>>
a écrit :

Hi All,

I am new on elasticsearch and want to setup it on distributed mode.

Please share any good article on the same.

Thanks,
Ankit

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Yes that's it.

Sometimes, it's too much magic :wink:

Imagine you have coworkers and all of you are using the default cluster name which is elasticsearch.
When one of your coworker will run its ES instance he will get all your own indices.
And if he runs something like: curl -XDELETE localhost:9200/ he will erase its indices but also yours! :wink:

So, if you have coworkers, the first thing to do is to change your cluster name in config/elasticsearch.yml

Le 29 janv. 2013 à 10:10, Ankit Jain ankitjaincs06@gmail.com a écrit :

Thanks for reply ...

So, Whenever we add a new node in network using the same clustername, it will automatically became a part of cluster.

Let me know, if i miss something

~Ankit

On Tuesday, 29 January 2013 13:49:40 UTC+5:30, David Pilato wrote:
Hi Ankit,

Elasticsearch IS a distributed search engine.

Just start new nodes on your network and Elasticsearch will detect them and do the rebalancing job for you.

You have many articles and video that talk about that. See: Elasticsearch Platform — Find real-time answers at scale | Elastic
Also, look at this page: Elasticsearch Platform — Find real-time answers at scale | Elastic

Does it help?

Le 29 janv. 2013 à 09:09, Ankit Jain ankitj...@gmail.com a écrit :

Hi All,

I am new on elasticsearch and want to setup it on distributed mode.

Please share any good article on the same.

Thanks,
Ankit

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Hi

i am also trying out E.S

what optimizations (changing parameters) should i make to the ES
configuration , or , while creating a java Node client , to achieve
maximum indexing performance ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

This is a very large question.
Mostly, Elasticsearch is by default set up with nice settings. Best settings depend really on your use case, the boxes you are running Elasticsearch on…

So generally speaking, I would say: "try with defaults!".

About indexing, use Bulk indexing instead of indexing documents one by one.

Have a look here: https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/ex1/IndexTest.java#L113

Le 29 janv. 2013 à 10:49, tarang dawer tarang.dawer@gmail.com a écrit :

Hi

i am also trying out E.S

what optimizations (changing parameters) should i make to the ES configuration , or , while creating a java Node client , to achieve maximum indexing performance ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

What kind of performance are you interested in?

  • maximum throughput?
  • fastest time to index a fixed amount of data?
  • minimal latency (for realtime indexing)?

Just to give a few points to consider for performance targets.

And, what is your environment, what is hardware you can provide? What
disks, what OS, what Java?

Have you tried the standard ES configuration? Have you tried bulk indexing?

Jörg

Am 29.01.13 10:49, schrieb tarang dawer:

Hi

i am also trying out E.S

what optimizations (changing parameters) should i make to the ES
configuration , or , while creating a java Node client , to achieve
maximum indexing performance ?

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

hi
i have 3 machines , 4 cores , 4 gb , 512 Gb each
i need optimization for maximum throughput , with real time indexing but
can afford a refreshing of index upto 20-30 seconds , until the record
becomes searchable. i am getting record one by one , so , i haven't tried
the bulk indexing.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

9

hi
i added the following configuration to the elasticsearch.yml file

index.refresh_interval: 5

indices.memory.index_buffer_size: 20%

index.translog.flush_threshold_period: 5s

index.cache.field.type: soft

min_index_buffer_size: 1024mb

max_index_buffer_size: 2048mb

indices.memory.min_shard_index_buffer_size: 512mb

along with it , added to env variables to /bin/elasticsearch

ES_MIN_MEM=1024m
ES_MAX_MEM=2048m

still could not find appreciable difference in indexing performance . is it
the case , that m i doing something wrong ? or should i add some other
parameter also . please help me out.

thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.