Hi, we have very different throughput scenarios for our cluster regarding the same indexes. Our product will be available through our private intranet, but it will be released publicly too, and available through internet. In the second scenario our throughput will be extremely larger than in the first one. Therefore I'd like to know if there is a way to use a coordinating node to route traffic for two differente subset of nodes, as if i had two smaller clusters inside one big cluster and could use the coordinating node to differentiate between the two, is it possible?
I think, if you assign "rack" or some other node attribute, then use that to send the "private" shards to private nodes, and another node attribute for the "public" node, then the routing from the coordinating nodes would automatically flow to shard owning nodes.
1 Like
Can you explain why you want to isolate the two different kinds of traffic? Why not just combine it all together?
Because we could have different latency requirements for the different type of users, those inside our organization and those who come from outside. This way we could have different hadware sizing for nodes serving the intranet requests and nodes serving the internet requests. Just to be clear, our expected throughput for intranet users should be around 5ops/s, but for users from internet this value should be around 50ops/s.
I see, makes sense. It's possible you might be able to achieve this with node attributes as suggested above, using ?preference=only_nodes:attr:value although this only affects search routing and not indexing or other cluster-wide things. For proper isolation I'd recommend setting up separate clusters and using cross-cluster replication to keep them in sync.
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.