# Docker ES 5.5.0 Low performance when ingest 1000 docs/sec

**URL:** <https://discuss.elastic.co/t/docker-es-5-5-0-low-performance-when-ingest-1000-docs-sec/97155>\
**Category:** Elasticsearch\
**Created:** [August 15, 2017, 10:24pm UTC](https://discuss.elastic.co/t/docker-es-5-5-0-low-performance-when-ingest-1000-docs-sec/97155 "2017-08-15T22:24:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rodrigongarcia](https://avatars.discourse-cdn.com/v4/letter/r/a4c791/32.png) [@rodrigongarcia](https://discuss.elastic.co/u/rodrigongarcia)\
**Post date:** [August 15, 2017, 10:24pm UTC](https://discuss.elastic.co/t/docker-es-5-5-0-low-performance-when-ingest-1000-docs-sec/97155/1 "2017-08-15T22:24:53Z")

</div>

Hello,  
First the general data:

- Version ES: 5.5.0
- Version ingestion: logstash-input-kafka (4.2.0)
- Operating System: docker - official compiled docker file:

> ```
> FROM docker.elastic.co/logstash/logstash:5.5.0
> 
> RUN rm -f /usr/share/logstash/pipeline/logstash.conf \
> && rm -f /usr/share/logstash/config/logstash.yml \
> && rm -f /usr/share/logstash/config/log4j2.properties
> 
> ADD pipeline/ /usr/share/logstash/pipeline/
> ADD config/ /usr/share/logstash/config/
> 
> RUN chown -R logstash:logstash /usr/share/logstash/config/ \
> && chmod -R +x /usr/share/logstash/config/ \
> && logstash-plugin remove logstash-input-kafka \
> && logstash-plugin remove logstash-output-kafka \
> && logstash-plugin install --version 4.2.0 logstash-input-kafka \
> && logstash-plugin install --version 4.0.4 logstash-output-kafka \
> && logstash-plugin list --verbose kafka
> 
> ```

Config File (if you have sensitive info, please remove it): Example input config:

```
bootstrap_servers => "${KAFKA_BOOTSTRAP_RC}"
client_id => "${HOSTNAME}_rc"
group_id => "${CONSUMER_GROUP}_rc"
topics_pattern => "${KAFKA_TOPICS}"
consumer_threads => "12"
decorate_events => true
auto_offset_reset => "earliest"
heartbeat_interval_ms => "300"
fetch_max_wait_ms => "300"
fetch_min_bytes => "20971520"
poll_timeout_ms => 300

```

Logstash Workers : 12 for each 6 nodes.  
Question:  
I would like consume 2000 messages/sec. But only consume 200/sec per node whit this config.  
Also when they ingest ES get some bad performance.  
I have 5 instances (all master/ingest/data)  
Gretings.

Edit: add some images:  
logstash performance:

 ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/9/9/996e33c5f25940d21060f42c47cbccbfcf72cac0.png)  
One node perfermance:  
 ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/6/9/69597c078e3db2c9e639e437066118beb4bef7d0.png)  
Elastic overview:  
 ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/f/5/f52520b96f4c5e8a892fdf9276517766d09ded46.png)  
Elastic Nodes general:  
 ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/f/9/f9b1db741b8e123e3533744fe7e9a9d14145c66c.png)  
Master performance:  
 ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/1/6/1651d7a72a10dd93e7c82649309bbf7082181553.png)  
i don't know for what see that gaps.  
 ![imagen](https://us1.discourse-cdn.com/elastic/original/3X/4/8/48be6ecb118f0b23edef326717dbcac450430ecc.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 12, 2017, 10:24pm UTC](https://discuss.elastic.co/t/docker-es-5-5-0-low-performance-when-ingest-1000-docs-sec/97155/2 "2017-09-12T22:24:56Z")

</div>

Closed on September 12, 2017, 10:24pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.