Version 6.3 _flush not flushing

Elastic Stack Elasticsearch
1

We are having trouble with flushing our translog now that we are on version 6.x. We are using default settings, other than our index has refresh_interval set to 60s.

Indices often have huge translogs (1GB) at the end of the hour. It seems the default translog settings in 6.3 would cause a flush to occur automatically.

In previous versions, we'd do a _flush after the indexing was complete for an index, to ensure the translog was processed and removed. The _flush API does not appear to work now.

Any help would be appreciated.

2

I just found the "manage" button in Kibana for the indices. Used it to flush the index and the translog_size_in_bytes did not change.

3

In Elasticsearch 6.x recovery can be made based on translog, which can be much more efficient than copying full indices, so it is kept around a bit longer than before to facilitate this. This can however be tuned using index settings. This blog post provides some additional background.

4

Thanks for the help Christian.

In that blog post you referred, it talks about sequence IDs and the default settings translog.retention.size and translog.retention.age. The defaults (512m and 12h) respectively seems to be exceeded. I have indices that are more than 12 hours old, but which still have large translogs.

And second, when I issue /index/_flush, why does that not flush? Shouldn't I still be able to force a flush?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.