we consider using a dedicated master node for our cluster. The master node wont hold any data, and is not involved in search processes. We are wondering about the communication workflow right now.
All search requests by the application aim at the dedicated master node. The master node redirects the request to the corresponding shards at the data nodes. Data nodes process the requests and then they do ... what ?
Do the data nodes merge shard based search results (e.g. for pagination purposes) only 'data node internally', without the master node beeing involved, or will the master have to do the result merging to calculate the final result set?
a node that coordinates for instance a search is called a coordinating node. It sends shard specific requests to the relevant data nodes and will then combine the responses received into a single search response that is sent back to the client.
Depending on your use case and load, having dedicated master node(s) can be advisable and if so, they would often not be used as coordinating nodes as well. The stability and performance of master nodes is important to the cluster.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.