Hi,
I am wondering if I can create more nodes and clusters for better index via logstash for apache logs, and for better and faster query is it necessary to create more indices and clusters , is there a source can indicate this point? Thank you very much, the Kibana runing the query at the backend was extremely slow! I am at the dead end actually.
Horizontal scaling is one of Elasticsearch's main selling points, so yes, adding nodes is often a good way to increase performance. However, every case is different and adding nodes isn't always the best option. It depends on things like the hardware, the type of data, the query and indexing load, what kinds of queries, and so on.
I think the existing documentation at elastic.co does a pretty good job of explaining how Elasticsearch works and what to consider when designing indexes and sizing shards.
I agree. What I want to add here is that in order to do so, you can use the automatic load balancing function provided by EC2 and Azure. And also, make sure you make full use of your existing hardware (https://www.elastic.co/guide/en/elasticsearch/reference/1.4/setup-configuration.html).
Hope it helps.
thanks , do you mean that a whole standalone server totally for Kibana (even elasticsearch)?
It sounds like you need a cluster for ES. I don't think Kibana has anything to do with the speed, since it's just a front end that generates query to ES. Shard is usually managed by ES automatically. If you don't want to the define the number of primary and replica, which needs some experiments, there is no need to touch it.
For Logstash, I don't know too much, but I have seen some people say additional config is needed when data goes big.