Elasticsearch 1.7.3 not adding sync_id to stale index shards


#1

Hi,

We upgraded a ES cluster from 1.4 to 1.7.3 recently. Main reason: getting sync_id on all stale index shards, making recoveries faster.

We use ES for ELK. Logstash creates new indices every day, so after a day rollover a set of indices becomes stale. Documentation says:

Elasticsearch tracks the indexing activity of each shard. Shards that have not received any indexing operations for 5 minutes are automatically marked as inactive. This presents an opportunity for Elasticsearch to reduce shard resources and also perform a special kind of flush, called synced flush. A synced flush performs a normal flush, then adds a generated unique marker (sync_id) to all shards.

We see lots of index shards without a sync_id (using the _stats/commit?level=shards API). These indices are absolutely stale: the commit.translog_id value (a millisecond epoch timestamp) says so.

Of course we can trigger this by calling the _flush/synced API, but I wonder what is happening.

Curious.

Thanks!


(system) #2