# Index rejections on ElasticSearch

**URL:** <https://discuss.elastic.co/t/index-rejections-on-elasticsearch/122940>\
**Category:** Elasticsearch\
**Created:** [March 7, 2018, 6:08pm UTC](https://discuss.elastic.co/t/index-rejections-on-elasticsearch/122940 "2018-03-07T18:08:37Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jorge\_Sanchex](https://avatars.discourse-cdn.com/v4/letter/j/7cd45c/32.png) [@Jorge\_Sanchex](https://discuss.elastic.co/u/Jorge_Sanchex)\
**Post date:** [March 7, 2018, 6:08pm UTC](https://discuss.elastic.co/t/index-rejections-on-elasticsearch/122940/1 "2018-03-07T18:08:37Z")

</div>

We are using Elasticsearch to store log files which are sent from the rsyslogd emelasticsearch plugin, the configuration of the omelasticsearch is following:

```
template(name="es_json"
  type="list") {
    constant(value="{")
      constant(value="\"timestamp\":\"") property(name="timereported" dateFormat="rfc3339")
      constant(value="\",\"host\":\"") property(name="hostname")
      constant(value="\",\"program\":\"") property(name="app-name" format="jsonr")
      constant(value="\",\"severity\":\"") property(name="syslogseverity-text")
      constant(value="\",\"severity_code\":") property(name="syslogseverity")
      constant(value=",\"facility\":\"") property(name="syslogfacility-text")
      constant(value="\",\"facility_code\":") property(name="syslogfacility")
      constant(value=",\"pri\":") property(name="pri")
      constant(value=",\"tag\":\"") property(name="syslogtag" format="jsonr")
      constant(value="\",\"message\":\"") property(name="msg" format="jsonr")
    constant(value="\"}")
}

# Log all messages to Elasticsearch
action(type="omelasticsearch"
    server="elasticsearch"
    template="es_json"
    searchIndex="logs_index"
    dynSearchIndex="on"
    action.resumeinterval="10"
    queue.type="linkedlist" # run asynchronously
    queue.filename="rsyslog_queue" # queue files
    queue.checkpointinterval="100"
    queue.size="40000"
    queue.maxdiskspace="500m" # space limit on disk
    queue.discardmark="10000"
    queue.discardseverity="4" # Discard Warning, Notice, Informational and Debug
    queue.highwatermark="20000"
    queue.lowwatermark="14000"
    action.resumeretrycount="-1" # infinite retries if host is down
    queue.saveonshutdown="on" # save messages to disk on shutdown
    queue.timeoutenqueue="0" # Immediately discard after 0ms if it can't be written
    queue.dequeuebatchsize="1024"
    queue.dequeueslowdown="1000")

```

Here are the thread rejections:

> [17:52:54 root@host:~]# curl -s -G 'elasticsearch:9200/\_cat/thread\_pool?v'  
> host ip bulk.active bulk.queue bulk.rejected index.active index.queue index.rejected search.active search.queue search.rejected  
> 1.1.1.1 1.1.1.1.1 0 0 0 20 26 373757 0 0 0

The version of Elasticsearch:

> ```
> {
> "name" : "elasticsearch",
> "cluster_name" : "logs",
> "version" : {
> "number" : "2.1.1",
> "build_hash" : "40e2c53a6b6c2972b3d13846e450e66f4375bd71",
> "build_timestamp" : "2015-12-15T13:05:55Z",
> "build_snapshot" : false,
> "lucene_version" : "5.3.1"
> },
> "tagline" : "You Know, for Search"
> }
> 
> ```

This is the configuration of the elasticsearch:

> discovery.zen.ping.multicast.enabled: false  
> discovery.zen.ping.unicast.hosts: ["10.10.0.68"]  
> discovery.zen.minimum\_master\_nodes: 1  
> network.bind\_host: 10.10.0.68  
> network.publish\_host: 10.10.0.68  
> cluster.name: logs  
> node.name: elasticsearch  
> bootstrap.mlockall: true  
> index.number\_of\_shards: 1  
> index.number\_of\_replicas: 0  
> index.refresh\_interval: 30s  
> index.translog.flush\_threshold\_ops: 20000  
> threadpool.search.type: fixed  
> threadpool.search.size: 20  
> threadpool.search.queue\_size: 100  
> threadpool.bulk.type: fixed  
> threadpool.bulk.size: 60  
> threadpool.bulk.queue\_size: 300  
> threadpool.index.type: fixed  
> threadpool.index.size: 20  
> threadpool.index.queue\_size: 100  
> indices.breaker.fielddata.limit: 25%  
> indices.breaker.request.limit: 40%  
> indices.breaker.total.limit: 70%  
> indices.memory.index\_buffer\_size: 512mb  
> indices.memory.min\_shard\_index\_buffer\_size: 12mb  
> indices.memory.min\_index\_buffer\_size: 96mb  
> indices.memory.max\_index\_buffer\_size: 512mb  
> indices.fielddata.cache.size: 15%  
> indices.fielddata.cache.filter.size: 15%  
> path.conf: /etc/elasticsearch  
> path.logs: /var/log/elasticsearch  
> path.data: /elasticsearch/data  
> index.max\_result\_window: 10000

As you can see we are using indexing operation instead of the bulk operation, I know that the bulk operation would be suggestion but is there anything else we shall focuse on?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 4, 2018, 6:08pm UTC](https://discuss.elastic.co/t/index-rejections-on-elasticsearch/122940/2 "2018-04-04T18:08:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.