Kibana dashbboard times out with large ES data

[antoine.brun]

Hello,

we have a production ES cluster (v2.4) with 4 data nodes (24Gb, 4CPU) (a screenshot below shows the full cluster details.

There are 2,668,920,207 docs in the cluster, these are syslogs collected from a UTM device.
These docs are distributed over 356 index (one per day), index name format is ubilogs-

We are using Kibana to show dashboard and we use ubilogs-* to build our visualisations and dashboards and most of the time Kibana times out when trying to build the visualisations even if the time range is very short (15 min, 1h,...)
It seems that the search from Kibana is searchig over all indexes even for a small time range.

Is there a way to control this or optimise?
If we want a dashboard over last 1h, it should only search in a few index (only one actually) and it shouldn't time out

Antoine

[jbudz (Jon Budzenski)]

This is supported in 4.6, the configuration for it would happen on the index pattern creation page. Specifying a time field should be sufficient, or you can use the deprecated event time based index names. Can you confirm that the index pattern in kibana is configured with a time field?

[Mayotte (DMA)]

Hi Jon,

We can confirm that we use time field.

Thanks,
Diyaldine

[Mayotte (DMA)]

Hello,

We have already updated to Kibana 4.6.6

We remarked that when in a dashboard, we have visualizations with aggregation with 8 fields for example (as below) it's take more time to display or a timeout error is sent.

So, are there few configurations that we must to apply in Kibana or Elasticsearch side concerning fields used in visualizations with aggregations ?

Thanks,
Diyaldine