# Logstash performance issue with filter and influxdb

**URL:** <https://discuss.elastic.co/t/logstash-performance-issue-with-filter-and-influxdb/81408>\
**Category:** Logstash\
**Created:** [April 6, 2017, 5:27am UTC](https://discuss.elastic.co/t/logstash-performance-issue-with-filter-and-influxdb/81408 "2017-04-06T05:27:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![k1ng87](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/k1ng87/32/88865_2.png) [@k1ng87](https://discuss.elastic.co/u/k1ng87)\
**Post date:** [April 6, 2017, 5:27am UTC](https://discuss.elastic.co/t/logstash-performance-issue-with-filter-and-influxdb/81408/1 "2017-04-06T05:27:25Z")

</div>

I currenlty have a logstash config that is written as such and I've noticed something interesting. The throughput of this filter is about 400 messages/sec which is awfully slow.

```
input {
        kafka {
                topics => ["kafka-jmx"]
                bootstrap_servers => "kafka1.cloud.com:9092"
                consumer_threads => 10
        }

}
filter {
        json {
                source => "message"
        }
        grok {
                patterns_dir => "/home/ec2-user/logstash-5.2.0/bin/patterns/"
                match => {"metric_path" => [
                                                                                        "%{DATA:kafka_host}\.%{DATA:kafka_metric_group}:type=%{DATA:kafka_metric_type},name=%{WORD:kafka_metric_name},topic=%{KTOPIC:kafka_topic},partition=%{KPARTITION:topic_partition}\.%{GREEDYDATA:attr_type}",
                                                                                        "%{DATA:kafka_host}\.%{DATA:kafka_metric_group}:type=%{DATA:kafka_metric_type},name=%{WORD:kafka_metric_name},topic=%{KTOPIC:kafka_topic}\.%{GREEDYDATA:attr_type}",
                                                                                        "%{DATA:kafka_host}\.%{DATA:kafka_metric_group}:type=%{DATA:kafka_metric_type},name=%{GREEDYDATA:kafka_metric_name}\.%{GREEDYDATA:attr_type}"
                                                                                       ]
                         }
        }
               ruby {
                        code => "event.set('time', event.get('@timestamp').to_f * 1000 )"
                }
                mutate {
                        remove_field => ["message"]
                        convert => {"time" => "integer"
                                                "metric_value_number" => "integer"
                        }
                }
}

output {
        influxdb {
                host => "ip-address"
                port => 9050
                db => "sdp_metrics"
                measurement => "jmx"
                retention_policy => "one_week"
                allow_time_override => "true"
                exclude_fields => ["@timestamp", "@version"]
                flush_size => 100
                use_event_fields_for_data_points => "true"
                coerce_values => {"metric_value_number" => "integer"}
                send_as_tags => ["kafka_host", "kafka_metric_group", "kafka_metric_type", "kafka_metric_name", "attr_type"]

                }
      }

```

I run the filter like this:

```
LS_HEAP_SIZE=16g LS_JAVA_OPTS='-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=9999' ./logstash -f jmx_influx.conf -w 60 -b 350

```

I've noticed that if I change the output to just `output { stdout { } }`, the throughput increases to 2k/sec. If I then also remove the filter, it goes to 5k/sec. How can I understand why my throughput drop from 2k/sec to 400k/sec when I change the output using the influxdb output?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 4, 2017, 5:27am UTC](https://discuss.elastic.co/t/logstash-performance-issue-with-filter-and-influxdb/81408/2 "2017-05-04T05:27:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.