Kibana 4 Running ordinary search so so slow


#1

Hi is there any one can help me this issue?

How come Kibana 4 Running ordinary search so so slow, I use the apache access logs for the raw data, I just search the goolebot and load last 60 days, it actually really take a long time. I am crazy.

I am wondering if I need to figure out the elastic index or query problem?

Thanks,


(Mark Walkom) #2

KB just reads from ES, so if it is slow then it's likely ES is having issues.

Are you monitoring your stack?


#3

Thanks for you reply. I am wondering what is the monitor you refer to? Any plugins, cause we just use one node, so I am unsure if it is doable for the monitor? Thanks again.


(Mark Walkom) #4

Check out Marvel - https://www.elastic.co/downloads/marvel


#5

Thanks, any free plugin that can be instead of Marvel, I think it is a pricing tool.


(Mark Walkom) #6

Marvel is free.


(Horst Birne) #7

only Marvel 2.X is free to use right? Marvel for ESCluster 1.X still require license.


#8

Hi @warkolm Thanks for your help, I think I have already figured out how to install Marvel at the ELK.
just see the data, I am unsure if i need to increase the nodes or clusters, cause to search one item at Kibana, it loading so slowly.

Cluster: elasticsearchStatus: YellowNodes: 1Indices: 51Memory: 765MB / 4GBTotal Shards: 438Unassigned Shards: 219Documents: 194,253,753Data: 127GBUptime: 6 minutesVersion: 2.1.0


(Mark Walkom) #9

It requires a license, but it's a free license.


(Mark Walkom) #10

What is the load on the system like?


Any idea on Kibana Request Per Second low to 0.43? Thanks!
#11

it should always take more than 1 min to load the data, if I select year to date and then i search agent:googlebot AND response:404, that is what i am getting trouble. I am unsure what is the good level from the Marvel, in terms of the speed for searching the data by Kibana search box, it is not a good idea. Thanks for the reply. Very Appreciated.


(Mark Walkom) #12

It sounds like you are overloading your cluster, running a query over a years worth of data is likely to be slow given you only have 4GB of heap.

Try giving more resources to your cluster.


#13

Sounds good, I am going to consider doing these things

  1. add nodes;
  2. add clusters;
  3. add more resources to the cluster as you mentioned, it is said that 4GB memory to 16 GB

any helpful doing so?

btw, what is the i ideal time response in general for running 1 year data with the AND search query ? Thanks :)


(Mark Walkom) #14

That depends on too many things to say.


#15

OK, i will do some more research and Thank you again. :grinning:


(Thomas Decaux) #16

It slows because Kibana is using msearch, mean searches in parallel . You can see an AJAX call to "/elasticsearch/_msearch".

You need lot of RAM on the node.


(system) #17