Route indexing requests to a specific node instead of a shard

Elastic Stack Elasticsearch
1

Hi, I searched through the docs but I could not find a way to route index requests to a specific node, instead of a specific shard.

If routing to a specific node is not possible, then is it possible to create shard-node mappings as index templates?

Thanks in advance!

2

Indexing is always first done on the primary shard and then replicated. Elasticsearch can move primary shards around in response to cluster events so it is not possible to tie it to specific nodes.

3

Consider I have disabled cluster rebalancing and shard replication.

So, can I specify a node-shard mapping for an index (preferably using templates) ?

4

Usually one of the main reasons to have multiple nodes in a cluster is to achieve high availability. Not having any replicas and diabling cluster rebalancing would remove that, which sounds risky.

What is the problem you are trying to solve? Do you need your data to be resilient and available?

5

I can accept data loss, but I am trying to reduce index request routing traffic between nodes.

What I need is:

node -1 <- receives bulk requests from multiple log forwarders.
node -2 <- receives bulk requests from multiple log forwarders.

Now I prefer the request received to be processed/written to the same node (shard in the node), instead of being routed to other shard.

This is my case.

6

What is the rationale behind this requirement? Are the nodes far apart?

If you do not care about availability and resiliency you could set up two separate single node clusters and query using cross cluster search.

1 Like
7

They perform high network intensive tasks.

Yes @Christian_Dahlqvist , but I don't want the search query to be updated based on the number of clusters (/nodes) added.

8

What you are trying to do goes against how Elasticsearch was designed to work, so I would not recommend going down this route. It might be possible through shard allocation filtering but would as far as I can tell give you a cluster that is not very resilient nor highly available.

Why is the network traffic between the nodes such aproblem for you?

9

@Christian_Dahlqvist Thanks for your help, I guess I'll go along with the cross cluster approach.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.