# Logstash 6.2.4 Slow Performance Kinesis Input when using json codec

**URL:** <https://discuss.elastic.co/t/logstash-6-2-4-slow-performance-kinesis-input-when-using-json-codec/132188>\
**Category:** Logstash\
**Created:** [May 16, 2018, 5:14pm UTC](https://discuss.elastic.co/t/logstash-6-2-4-slow-performance-kinesis-input-when-using-json-codec/132188 "2018-05-16T17:14:35Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![amarsh](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@amarsh](https://discuss.elastic.co/u/amarsh)\
**Post date:** [May 16, 2018, 5:14pm UTC](https://discuss.elastic.co/t/logstash-6-2-4-slow-performance-kinesis-input-when-using-json-codec/132188/1 "2018-05-16T17:14:35Z")

</div>

I'm seeing significant performance impacts when using **codec =\> "json"** with the Kinesis input plugin and wanted to see if anyone has recommendations or suggestions for speeding up the processing.

On a 4 CPU 16 GB box running RedHat with the xmx and xms set to 8 GB, the following config is able to process about 50,000+ per minute 4 workers and a batch size of 10000 into a 8 node ElasticSearch cluster. This is without the **code** setting.

```
input {
        kinesis {
                kinesis_stream_name => "kinesis-stream"
                application_name => "logstash-kinesis-poc"
                region => "us-east-1"
                profile => "default"
         }
}
output {
        stdout { codec => rubydebug }
        elasticsearch {
                hosts => ["http://elastic_box_one:9200","http://elastic_box_two:9200"]
                doc_as_upsert => true
                template_overwrite => false
                index => "kinesis-poc-%{+YYYY-MM-dd}"
                template_name => "poc"
        }
}

```

When the config has **type** and **codec** added the performance drops to 2100+ per minute at 4 workers and batch of 10,000.

```
input {
        kinesis {
                kinesis_stream_name => "kinesis-stream"
                application_name => "logstash-kinesis-poc"
                region => "us-east-1"
                profile => "default"
                type => "message"
                codec => "json"
         }
}
output {
        stdout { codec => rubydebug }
        elasticsearch {
                hosts => ["http://elastic_box_one:9200","http://elastic_box_two:9200"]
                doc_as_upsert => true
                template_overwrite => false
                index => "kinesis-poc-%{+YYYY-MM-dd}"
                template_name => "poc"
        }
}

```

Performance increases to about 5000 per minutes when the workers are left at 4 and the batch is reduced to 100.

We also tried using **filter** to set the json source, but that had the same level of performance decrease as using **codec =\> "json"** in the input.

```
filter {
         json {
             source => "message"
         }
 }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 13, 2018, 5:14pm UTC](https://discuss.elastic.co/t/logstash-6-2-4-slow-performance-kinesis-input-when-using-json-codec/132188/2 "2018-06-13T17:14:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.