Kibana performance while reindex

Elastic Stack Kibana
1

Hello,

Can Kibana's performance be affected while I reindex?

For example I want to reindex good amount of data and it will take some days. During those days will I have reduced performance in loading visualisations or dashboards?

Thank you very much in advance!

/Angelos

2

When it comes to visualizations on dashboards, Kibana is just the frontend putting the pixels together and all the heavy work fetching data and aggregating it happens in Elasticsearch, triggered by Kibana sending search requests.

When you are reindexing, your ES cluster will experience higher load than normal - this can absolutely slow down performance of search requests as well and thus increase the time till a dashboard is fully rendered. However, it depends on a lot of factors how much it will actually be slowed down - testing in the evening with a partial reindex is probably the easiest way to get reliable data.

You can throttle the reindexing (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-throttle) to spread it out over a longer time frame while going easy on your cluster resources, keeping Kibana fast. It probably makes sense to play around a little to find a config fitting your case.

3

Hi Joe,

Thank you for your response!

I reindex 10b documents, so it will take some weeks.
But what I discovered is that visualisations or discoveries are affecting all the other index patterns except for the one that reindex.

So lets say that we have the "first" index pattern and I want to reindex and named it "test-first".
If you look the results bellow the "test-first" is not affected but the "first" is affected the most.

Reindexing period**:
0.8b documents
Index pattern: first 50s, 45s
Reindex - Index pattern: test-first 28s, 28s

No reindexing time period***:
0.8b documents
Index pattern: first 14s, 14s, 13s
Reindex - Index pattern: test-first 27s, 27s, 27s

** Reindexing period is the period(months) that data is still increasing(updating)
*** No reindexing time period is the period(months) that is not updating any more because it is finished.

Is this normal?

/Angelos

4

It's an interesting pattern I can't explain without digging deeper into it, but it intuitively makes sense as first is getting hit with lots of read requests from the re-indexing so the Kibana requests probably get queued, while test-first is written to but is not experiencing a high frequency of search requests. Not sure whether that's actually what's going on though.

If you want to go deeper here I suggest posting the question in the Elasticsearch subforum.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.