# Dec 21, 2017: \[EN\]\[Elasticsearch\] Knobs to turn for better indexing performance

**URL:** <https://discuss.elastic.co/t/dec-21-2017-en-elasticsearch-knobs-to-turn-for-better-indexing-performance/112332>\
**Category:** Advent Calendar\
**Created:** [December 21, 2017, 11:00am UTC](https://discuss.elastic.co/t/dec-21-2017-en-elasticsearch-knobs-to-turn-for-better-indexing-performance/112332 "2017-12-21T11:00:48Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sherry\_Ger](https://avatars.discourse-cdn.com/v4/letter/s/bc79bd/32.png) [@Sherry\_Ger](https://discuss.elastic.co/u/Sherry_Ger)\
**Post date:** [December 21, 2017, 11:00am UTC](https://discuss.elastic.co/t/dec-21-2017-en-elasticsearch-knobs-to-turn-for-better-indexing-performance/112332/1 "2017-12-21T11:00:48Z")

</div>

You have a high volume logging use case and have followed these recommendations:

- [Important Elasticsearch Settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html)
- [Important System Settings for Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/system-config.html)

What other knobs can you turn to improve indexing performance?

- Set [index.refresh\_interval](https://www.elastic.co/guide/en/elasticsearch/reference/6.1/index-modules.html#dynamic-index-settings) to 30s to 60s if near real time search is not a requirement. By default, this is set to 1s.
- Increase [indices.memory.index\_buffer\_size](https://www.elastic.co/guide/en/elasticsearch/reference/6.1/indexing-buffer.html). It defaults to 10% of the total JVM heap allocated to a node that is to be used as the indexing buffer across all active shards.
- Disable [\_field\_names](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-field-names-field.html#_disabling_literal__field_names_literal) if [exists](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html) query is not in use.

You can find more details [here](https://www.elastic.co/guide/en/elasticsearch/reference/6.1/tune-for-indexing-speed.html).

If your use case can tolerate increased risk of data loss in event of hardware failures, these options will push the write througput even further.

- Boost [index.translog.flush\_threshold\_size](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-translog.html#_translog_settings). Once the translog reaches the specified size, a flush will take place. Defaults 512mb.
- Set [index.translog.durability](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-translog.html#_translog_settings) to `async`. This setting is risky as all acknowledged writes since the last commit will be discarded if a hardware failure should occur. Depending on the use case, this may be worth considering.

All except [indices.memory.index\_buffer\_size](https://www.elastic.co/guide/en/elasticsearch/reference/6.1/indexing-buffer.html) are index level settings and dynamically configurable. Also, you can add them to an index template to make them defaults for all indices matching the `index-patterns`. For example,

```auto
PUT _template/logs
{
  "order": 0,
  "index_patterns": "logs-*",
  "settings": {
    "refresh_interval": "30s",
    "number_of_shards": "3",
    "translog": {
      "flush_threshold_size": "1gb",
      "durability": "async"
    },
    "unassigned": {
      "node_left": {
        "delayed_timeout": "5m"
      }
    },
    "query": {
      "default_field": "message"
    },
    "number_of_replicas": "1"
  },
  "mappings": {
    "doc": {
      "_field_names": {
        "enabled": false
      }
    }
  }
}

```

And what does the future hold for better indexing performance? In the up and coming Elasticsearch version 7, we are working towards an [intelligent refresh](https://github.com/elastic/elasticsearch/pull/27500), where we will skip the `refresh` on a shard that has not been searched on for a period of time (30s by default) and perform the `refresh` at the next scheduled interval if a search request should arrive for the shard.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 23, 2018, 8:31am UTC](https://discuss.elastic.co/t/dec-21-2017-en-elasticsearch-knobs-to-turn-for-better-indexing-performance/112332/3 "2018-08-23T08:31:25Z")

</div>