How to use client nodes

mad_min · January 5, 2016, 4:43am

I have a cluster of 4 master, 8 data and 2 client/query nodes. This is ingesting logs from logstash. Currently, logstash is sending to all the data nodes (assumption: its load balancing). Is this the most optimal way? Should logstash send to the client/query nodes instead? Currently, the client nodes are what Kibana and Grafana speak to and have the most RAM and cores provisioned for it.

Thanks in advance for any pointers.

warkolm · January 5, 2016, 4:44am

Depends on your EPS and query load. If you do lots of queries then leaving things as they are is fine.

That's an odd number, why 4?

mad_min · January 5, 2016, 4:48am

Thanks @warkolm.

I do foresee burst of queries coming in (~100). But not more. However, the queries could result in tons of data.
Well 4 masters because I started with 3 and they were spread out in different subnets and I felt one of them was flaky because of subneting issues so I brought another one up, but thats not the case so I kept 4. Ideally they should be 'odd' in number? (Will bring one down then)

Thanks.

warkolm · January 5, 2016, 4:49am

Ideally they should be odd, do you have this set at least?

mad_min · January 5, 2016, 5:04am

Yes, thanks for the pointer. I have it set to 3.

Slightly overwhelmed with all the knobs there are to squeeze out the best performance, will get there soon hopefully. (Hint: pointer to a nice 'Tips'n'Tricks would help)

warkolm · January 5, 2016, 5:28am

There's no single source for that, you kinda need to just go through each thing and optimise it.
That's what the forums (and support) are for

mad_min · January 6, 2016, 6:56pm

Yeah, totally agree. I am reaching that same conclusion. Plus a lot of content on the web helps as well.