Set up Elastic Search On Production

Elastic Stack Elasticsearch
1

Hi,
I am very new into elastic search world.
Just created one search system using elastic search which contains around 2L docs.
I did it on my local machine and now I want to take it live on production
Currently I did all at free of cost .

My Questions are -

  1. How I can set up elastic search on multiple machine with one cluster. One machine will be master and rest will be slaves.
  2. How I can make sure that each write query will hit master and all fetch queries will hit slaves.
  3. Does any paid version of elastic search required to achieve what I want?

Thanks,
Satty

2

I am using 2.3 version of elastic search and want to go on production with the same version.

3

Some things:

  • The "L" unit isn't particularly common so you are better off using an SI
    unit.
  • Elasticsearch doesn't have master/slave replication for data. See
    minimum_master_nodes for how cluster state uses it. Data is written first
    to a "primary" shard and then written to replica shards but all shard that
    write do about the same amount of work. Elasticsearch moves shards around
    the cluster on its own and decides which copy is primary all on its own.
    For the most part forcing search traffic to the replica shards isn't a
    thing you do.
4

How I can set up Elasticsearch on multiple machine with one cluster. One machine will be master and rest will be slaves.

I'm quite sure this is covered by the documentation on elastic.co. Please come back if you have any specific questions.

Does any paid version of Elasticsearch required to achieve what I want?

No.

5

Hi Magnus,
Please share documentation URL, I have gone through elastic search docs and did not find a way to achieve what I want.

6

Nik,
Please correct me where I am wrong.
As per my understanding of ES .. ES takes care of replica and sharding in a very smart way .. developers dont need to put efforts. But all this is maintained in single ES instance. what If that instance goes down due to xyz reason-
According to my understanding this can be resolved in two ways -

  1. run 2 instances of ES on two separate machines and index all data on both machines and put these machines under LB and if any machine goes down, data will be served by another and mean while I will make the machine up which went down.
    Problem with this- Data duplicacy.
  2. I will run ES on 3 machines with same cluster and different node name and will edit config file on each server . will make one ES instance master by node.master: true , node.data: false in config file and will make other ES instances secondary(whatever u call) which will be used to serve fetch queries and if master goes down, one of secondary will be elected as primary.
    I am not able to achieve this solution. or my solution itself is incorrect.

Please share your thoughts.

7

Please share documentation URL, I have gone through Elasticsearch docs and did not find a way to achieve what I want.

Did you read and understand The Definitive Guide?

I will run ES on 3 machines with same cluster and different node name and will edit config file on each server . will make one ES instance master by node.master: true , node.data: false in config file and will make other ES instances secondary(whatever u call) which will be used to serve fetch queries and if master goes down, one of secondary will be elected as primary.

With a three-node cluster all three nodes should be master-eligible, i.e. have node.master enabled. Otherwise if your single master node goes down the whole cluster becomes inoperable. With all nodes master-eligible and if all indexes have at least one replica you will be able to shut down one of the nodes (either one) while still being operable with all data accessible.

8

Hi Nik and Magnus,
Thanks for reply.
I have achieved my goal .. steps are-

  1. Got tar file of ES and unzipped it and put same in two directorories(D1,D2)
  2. Edit config file in both location with -
    in D1 - set cluster name as test-cluster and node name as node-1
    in D2 - set cluster name as test-cluster and node name as node-2
  3. started ES instance in D1 and then started ES instances in D2
  4. if one goes down another will work as master node.
    Will try to implement same on different machines.