Why my expired translog files are not deleted?


elasticsearch version : 6.1.3
when call IndexShard#afterWriteOperation to flush , i occured (29125)[https://github.com/elastic/elasticsearch/pull/29125]

index.translog.retention.size: 512M

when my .tlog's size > 5G,i ocured endless flush loop
the first .tlog file generated before 19h, total .tlog sizes have reached 6.5g

In IndexShard#afterWriteOperation shouldPeriodicallyFlush or shouldRollTranslogGeneration will call translog.trimUnreferencedReaders() , why my translog always growing has not been deleted by index.translog.retention.age or index.translog.retention.size policy ?

The answer to your question is the issue to which you linked, https://github.com/elastic/elasticsearch/pull/29125, which fixes a bug in the very old version you are using. You should upgrade.

@DavidTurner Thanks for your reply.

I see 29125 fixed the wrong logic in shouldPeriodicallyFlush() but rollTranslogGeneration() or flush() calls translog.trimUnreferencedReaders(), i want to know why
translog files has not been deleted .

    public void rollTranslogGeneration() throws EngineException {
        try (ReleasableLock ignored = readLock.acquire()) {
        } catch (AlreadyClosedException e) {
            throw e;
        } catch (Exception e) {
            try {
                failEngine("translog trimming failed", e);
            } catch (Exception inner) {
            throw new EngineException(shardId, "failed to roll translog", e);

BTW, Is there any other way to avoid this problem besides upgrading?

Sorry, 6.1 is well past the end of its supported life and I'm not in a position to investigate further. I don't believe there is a workaround that doesn't involve upgrading.

Thanks again

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.