# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=454 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 455 --- ## [Logstash reading from Redis Slows Significantly After Several Days](https://discuss.elastic.co/t/logstash-reading-from-redis-slows-significantly-after-several-days/205908)
**Author:** [@justin\_spies](https://discuss.elastic.co/u/justin_spies)\ **Replies:** 0\ **Last updated:** [October 30, 2019, 4:12pm UTC](https://discuss.elastic.co/t/logstash-reading-from-redis-slows-significantly-after-several-days/205908 "2019-10-30T16:12:11Z")
For our Kubernetes cluster we have ELK setup as follows: Filebeat -\> Redis (AWS ElastiCache) -\> Logstash -\> ElasticSearch Logstash is on one cluster and ElasticSearch is on a separate Cluster. Both Logstash and ES are … --- ## [Contributing to Logstash plugins open source](https://discuss.elastic.co/t/contributing-to-logstash-plugins-open-source/205903)
**Author:** [@AustinTag](https://discuss.elastic.co/u/AustinTag)\ **Replies:** 0\ **Last updated:** [October 30, 2019, 3:54pm UTC](https://discuss.elastic.co/t/contributing-to-logstash-plugins-open-source/205903 "2019-10-30T15:54:41Z")
I have had a PR to a logstash owned plugin open for a few weeks: https://github.com/logstash-plugins/logstash-mixin-aws/pull/39 Is there any way to escalate the PR, or does anyone have tips on getting this looked at by… --- ## [How I can delete element from array?](https://discuss.elastic.co/t/how-i-can-delete-element-from-array/205868)
**Author:** [@gleb](https://discuss.elastic.co/u/gleb)\ **Replies:** 2\ **Last updated:** [October 30, 2019, 3:04pm UTC](https://discuss.elastic.co/t/how-i-can-delete-element-from-array/205868 "2019-10-30T15:04:47Z")
Hello! I receive messages like this: "app\_risk = 0 app\_risk =2" or "app\_risk = 0" After kv filter, I have field "app\_risk": \[ "0", "2"\] or \["0"\]. How I can delete the first element of array - "0", if app\_risk\[1\] exis… --- ## [Webhdfs write path](https://discuss.elastic.co/t/webhdfs-write-path/205892)
**Author:** [@sidi](https://discuss.elastic.co/u/sidi)\ **Replies:** 0\ **Last updated:** [October 30, 2019, 3:04pm UTC](https://discuss.elastic.co/t/webhdfs-write-path/205892 "2019-10-30T15:04:18Z")
Hi, I'm using logstash to write data from elasticsearch to hdfs. Everything is going well but th eproblem is when logstash write data in hdfs there is the following exception: webhdfs write caused an exception: {"Remo… --- ## [\[LogStash::Runner\] Logstash - java.lang.OutOfMemoryError: Java heap space](https://discuss.elastic.co/t/logstash-runner-logstash-java-lang-outofmemoryerror-java-heap-space/205881)
**Author:** [@Bharath\_Pusuluri](https://discuss.elastic.co/u/Bharath_Pusuluri)\ **Replies:** 1\ **Last updated:** [October 30, 2019, 2:47pm UTC](https://discuss.elastic.co/t/logstash-runner-logstash-java-lang-outofmemoryerror-java-heap-space/205881 "2019-10-30T14:47:40Z")
I am trying to run a Logstash service ingesting the data returning from curl command. Somehow, the Logstash is failing saying with the following warnings and errors. I tried running the Logstash with option - LS\_HEAP\_SI… --- ## [JDBC Input with Aggregate Filter](https://discuss.elastic.co/t/jdbc-input-with-aggregate-filter/205617)
**Author:** [@Ahmad\_Samir](https://discuss.elastic.co/u/Ahmad_Samir)\ **Replies:** 6\ **Last updated:** [October 30, 2019, 2:41pm UTC](https://discuss.elastic.co/t/jdbc-input-with-aggregate-filter/205617 "2019-10-30T14:41:06Z")
Hello, I have a plugin that gets data from SQL server and aggregates some records with a unique ID, But unfortaintly I didn't get any output. My config file is as bellow. input { jdbc { jdbc\_connection\_string =\> … --- ## [Netflow Module : Logstash geoip2.exception](https://discuss.elastic.co/t/netflow-module-logstash-geoip2-exception/205873)
**Author:** [@Elena\_Kaidrikov](https://discuss.elastic.co/u/Elena_Kaidrikov)\ **Replies:** 0\ **Last updated:** [October 30, 2019, 1:58pm UTC](https://discuss.elastic.co/t/netflow-module-logstash-geoip2-exception/205873 "2019-10-30T13:58:39Z")
The Kibana Dashboard is empty, despite of Logstash receives the netflow traffic. The logstash log has the following prints : \[DEBUG\]\[org.logstash.filters.GeoIPFilter\] IP not found! exception=com.maxmind.geoip2.excepti… --- ## [\[Logstash 6.5.2\] Problem parsing long messages](https://discuss.elastic.co/t/logstash-6-5-2-problem-parsing-long-messages/205872)
**Author:** [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\ **Replies:** 1\ **Last updated:** [October 30, 2019, 1:57pm UTC](https://discuss.elastic.co/t/logstash-6-5-2-problem-parsing-long-messages/205872 "2019-10-30T13:57:25Z")
Hi there, I'm using Logstash to parse a java json log which may have a stacktrace nested field. I wanted to use json filter to automatically parse the input (since it is a json with possible nested jsons). Now, if I ap… --- ## [Attempting to reconnect to backoff(async(tcp://localhost:5044)) with 11 reconnect attempt(s)](https://discuss.elastic.co/t/attempting-to-reconnect-to-backoff-async-tcp-localhost-5044-with-11-reconnect-attempt-s/205842)
**Author:** [@kavindu\_manaram](https://discuss.elastic.co/u/kavindu_manaram)\ **Replies:** 1\ **Last updated:** [October 30, 2019, 1:41pm UTC](https://discuss.elastic.co/t/attempting-to-reconnect-to-backoff-async-tcp-localhost-5044-with-11-reconnect-attempt-s/205842 "2019-10-30T13:41:06Z")
My filebeat, logstash and kibana works perfectly and I set logstash as a windows service usiing Nssm.exe after that filebeat refuse to connect logstash with error Attempting to reconnect to backoff(async(tcp://localhos… --- ## [JSON parse error - Java stack trace](https://discuss.elastic.co/t/json-parse-error-java-stack-trace/205858)
**Author:** [@DenizParlak](https://discuss.elastic.co/u/DenizParlak)\ **Replies:** 0\ **Last updated:** [October 30, 2019, 12:36pm UTC](https://discuss.elastic.co/t/json-parse-error-java-stack-trace/205858 "2019-10-30T12:36:36Z")
Hi, Logstash version 7.3.1, running with Docker compose. Firstly, we don't want to use Filebeat to resolve this problem, at least for now. You can see below the output which we don't want like this. Here our default… --- ## [Errore](https://discuss.elastic.co/t/errore/205852)
**Author:** [@tomas1](https://discuss.elastic.co/u/tomas1)\ **Replies:** 0\ **Last updated:** [October 30, 2019, 12:25pm UTC](https://discuss.elastic.co/t/errore/205852 "2019-10-30T12:25:44Z")
Errore: impossibile trovare o caricare la classe principale Files\\elasticstack\\l ogstash-7.4.1\\logstash-core\\lib\\jars\\animal-sniffer-annotations-1.14.jar;C:\\Prog ram --- ## [Unable to connect to SQL database using jdbc driver](https://discuss.elastic.co/t/unable-to-connect-to-sql-database-using-jdbc-driver/205838)
**Author:** [@mduduzi](https://discuss.elastic.co/u/mduduzi)\ **Replies:** 0\ **Last updated:** [October 30, 2019, 11:48am UTC](https://discuss.elastic.co/t/unable-to-connect-to-sql-database-using-jdbc-driver/205838 "2019-10-30T11:48:25Z")
Hi Am experiencing connection issues when trying to connect to sql database using jdbc driver. I'm using intergrated security for connection but error shows l require a jdbc user. Is it correct to be requested of a use… --- ## [Error: com.mysql.jdbc.Driver not loaded. Are you sure you've included the correct jdbc driver in :jdbc\_driver\_library?](https://discuss.elastic.co/t/error-com-mysql-jdbc-driver-not-loaded-are-you-sure-youve-included-the-correct-jdbc-driver-in-jdbc-driver-library/205833)
**Author:** [@thotakura2](https://discuss.elastic.co/u/thotakura2)\ **Replies:** 0\ **Last updated:** [October 30, 2019, 11:23am UTC](https://discuss.elastic.co/t/error-com-mysql-jdbc-driver-not-loaded-are-you-sure-youve-included-the-correct-jdbc-driver-in-jdbc-driver-library/205833 "2019-10-30T11:23:41Z")
i am trying to import mysql table into elastic search using logistash and kibana. i gave the following config file. input{ jdbc{ jdbc\_connection\_string =\> "jdbc:mysql://172.158.2.18:3306/edst" jdbc\_user… --- ## [Dynamic log file updated automatically](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092)
**Author:** [@theo1991](https://discuss.elastic.co/u/theo1991)\ **Replies:** 6\ **Last updated:** [October 30, 2019, 8:38am UTC](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092 "2019-10-30T08:38:52Z")
Hello, My problem is that I have a dynamic file with log entries, and there're changing always. For example : a ticket with an id (never change) and a number of events, but this number can grow every minutes. Is there… --- ## [Is it possible to set a specific variable when starting the logstash?](https://discuss.elastic.co/t/is-it-possible-to-set-a-specific-variable-when-starting-the-logstash/205804)
**Author:** [@Yungyoung\_Ok](https://discuss.elastic.co/u/Yungyoung_Ok)\ **Replies:** 0\ **Last updated:** [October 30, 2019, 7:46am UTC](https://discuss.elastic.co/t/is-it-possible-to-set-a-specific-variable-when-starting-the-logstash/205804 "2019-10-30T07:46:55Z")
question: Is it possible to set a specific variable when starting the logstash? If 1 is possible, Can I run above function on conditional? problem: I am putting data into Elasticsearch through a logstash. In the log… --- ## [Not able to telnet IP with port 5044](https://discuss.elastic.co/t/not-able-to-telnet-ip-with-port-5044/205790)
**Author:** [@Joseph\_John](https://discuss.elastic.co/u/Joseph_John)\ **Replies:** 2\ **Last updated:** [October 30, 2019, 6:15am UTC](https://discuss.elastic.co/t/not-able-to-telnet-ip-with-port-5044/205790 "2019-10-30T06:15:49Z")
Hi All, Good morning. I have installed logtash 7.4.1-1 , my service starts, no error messages in the logs, but I am not able to telnet to the IP with port 5044 when I did a "netstat -anp | grep 5044" it shows netsta… --- ## [Duplicating message parts in Elastic dashboard](https://discuss.elastic.co/t/duplicating-message-parts-in-elastic-dashboard/205791)
**Author:** [@kavindu\_manaram](https://discuss.elastic.co/u/kavindu_manaram)\ **Replies:** 2\ **Last updated:** [October 30, 2019, 6:13am UTC](https://discuss.elastic.co/t/duplicating-message-parts-in-elastic-dashboard/205791 "2019-10-30T06:13:24Z")
I had shipped log to ES with the integration of filebeat and logstash. This is my logstash.conf # Read input from filebeat by listening to port 5044 on which filebeat will send the data input { beats { type =\> … --- ## [How to avoid the repetition of code in the output section in Logstash](https://discuss.elastic.co/t/how-to-avoid-the-repetition-of-code-in-the-output-section-in-logstash/205792)
**Author:** [@arunpmohan](https://discuss.elastic.co/u/arunpmohan)\ **Replies:** 1\ **Last updated:** [October 30, 2019, 5:59am UTC](https://discuss.elastic.co/t/how-to-avoid-the-repetition-of-code-in-the-output-section-in-logstash/205792 "2019-10-30T05:59:23Z")
Here is the snippet of my output section of logstash: output { if \["log\_type" \] == "syslog" { if "vcsa-4-westus2" in \[message\] { elasticsearch { hosts =\> \["elastic-monitor.service.discover:9200"\] user =\> … --- ## [Problem when use ElasticSearch with SQLServer](https://discuss.elastic.co/t/problem-when-use-elasticsearch-with-sqlserver/205624)
**Author:** [@Hoang\_Nguyen1](https://discuss.elastic.co/u/Hoang_Nguyen1)\ **Replies:** 2\ **Last updated:** [October 30, 2019, 3:46am UTC](https://discuss.elastic.co/t/problem-when-use-elasticsearch-with-sqlserver/205624 "2019-10-30T03:46:17Z")
I don't know whether when we update data in SqlServer to ES by Logstash, ES will reindex all or not ? --- ## [Creating graph based on keyword](https://discuss.elastic.co/t/creating-graph-based-on-keyword/205776)
**Author:** [@mbrown007](https://discuss.elastic.co/u/mbrown007)\ **Replies:** 1\ **Last updated:** [October 30, 2019, 2:52am UTC](https://discuss.elastic.co/t/creating-graph-based-on-keyword/205776 "2019-10-30T02:52:45Z")
I want to create a bar graph or even a pie graph based on a text field. This text field does say in ES that it is Aggregatable. There is the field name and then field name.keyword. (Home Type & Home Type.keyword) I have… --- ## [Parsing failure Jmeter to logstash](https://discuss.elastic.co/t/parsing-failure-jmeter-to-logstash/205581)
**Author:** [@Trevorp](https://discuss.elastic.co/u/Trevorp)\ **Replies:** 2\ **Last updated:** [October 30, 2019, 1:31am UTC](https://discuss.elastic.co/t/parsing-failure-jmeter-to-logstash/205581 "2019-10-30T01:31:25Z")
I m trying to use beats to send Jmeter data to log stash, I can get the data in Ok , But not in the format I want. Logstash puts it all into one field. My Logstash.config file looks like Read input from filebeat by li… --- ## [Logstash skipping files while reading config](https://discuss.elastic.co/t/logstash-skipping-files-while-reading-config/205767)
**Author:** [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)\ **Replies:** 5\ **Last updated:** [October 30, 2019, 12:23am UTC](https://discuss.elastic.co/t/logstash-skipping-files-while-reading-config/205767 "2019-10-30T00:23:44Z")
Logstash just starts and shuts down due to configpathloader error. Please explain what the issue is. Below is the config file- filebeat.inputs: type: log paths: /var/log/system.log /var/log/wifi.log type: log pa… --- ## [Logstash pipeline monitoring](https://discuss.elastic.co/t/logstash-pipeline-monitoring/205734)
**Author:** [@funlearning321](https://discuss.elastic.co/u/funlearning321)\ **Replies:** 2\ **Last updated:** [October 29, 2019, 9:45pm UTC](https://discuss.elastic.co/t/logstash-pipeline-monitoring/205734 "2019-10-29T21:45:17Z")
Hello, Can i please know if there is a way to monitor the logstash pipelines that can trigger an alert if the pipeline is down using ELK? we have a X-pack. Thanks --- ## [Vault audit log issue sending to elasticsearch](https://discuss.elastic.co/t/vault-audit-log-issue-sending-to-elasticsearch/205757)
**Author:** [@polaaditya](https://discuss.elastic.co/u/polaaditya)\ **Replies:** 0\ **Last updated:** [October 29, 2019, 8:55pm UTC](https://discuss.elastic.co/t/vault-audit-log-issue-sending-to-elasticsearch/205757 "2019-10-29T20:55:13Z")
My filebeat config filebeat.inputs: - type: log enabled: true json.keys\_under\_root: true json.overwrite\_keys: true json.message\_key: log paths: - /var/log/vault\_audit.log … --- ## [Ruby filter inline code keeps logstash to restart](https://discuss.elastic.co/t/ruby-filter-inline-code-keeps-logstash-to-restart/204526)
**Author:** [@paano](https://discuss.elastic.co/u/paano)\ **Replies:** 6\ **Last updated:** [October 29, 2019, 6:41pm UTC](https://discuss.elastic.co/t/ruby-filter-inline-code-keeps-logstash-to-restart/204526 "2019-10-29T18:41:13Z")
I have a local docker ELK stack OSS image running and it works. Version: 6.5.4 When deployed into the enterprise ELK stack which is 6.5.4, it crashes the logstash instance. Error: \[2019-10-21T10:58:20,568\]\[ERROR\]\[log… --- ## [Ingesting Json for Elasticsearch using Logstash](https://discuss.elastic.co/t/ingesting-json-for-elasticsearch-using-logstash/205737)
**Author:** [@bkdavis](https://discuss.elastic.co/u/bkdavis)\ **Replies:** 0\ **Last updated:** [October 29, 2019, 6:20pm UTC](https://discuss.elastic.co/t/ingesting-json-for-elasticsearch-using-logstash/205737 "2019-10-29T18:20:03Z")
I'm very new to Logstash and am trying to figure out how to properly ingest json into an elasticsearch index. I have a .json file that is perfectly formatted for my elasticsearch index however I can't figure out the bes… --- ## [How to Merge couple of JOSN with respect to one provided field value](https://discuss.elastic.co/t/how-to-merge-couple-of-josn-with-respect-to-one-provided-field-value/205733)
**Author:** [@Diganta](https://discuss.elastic.co/u/Diganta)\ **Replies:** 0\ **Last updated:** [October 29, 2019, 5:34pm UTC](https://discuss.elastic.co/t/how-to-merge-couple-of-josn-with-respect-to-one-provided-field-value/205733 "2019-10-29T17:34:54Z")
Hi , Please find my input of both Json: \< \[{"tan":"00000000000001","features":\[{"gender":"V","age\_range":"68+","province":"Flevoland","household\_size":"3","household\_income":"-","car\_ownership":"1 auto","number\_… --- ## [Logstash 7.4 Input file for .log files not working in windows](https://discuss.elastic.co/t/logstash-7-4-input-file-for-log-files-not-working-in-windows/205726)
**Author:** [@sanurani](https://discuss.elastic.co/u/sanurani)\ **Replies:** 2\ **Last updated:** [October 29, 2019, 5:30pm UTC](https://discuss.elastic.co/t/logstash-7-4-input-file-for-log-files-not-working-in-windows/205726 "2019-10-29T17:30:43Z")
Attempting to create logfile dashboard in Kibana via ELK. Tried simple STDIN which worked and was able to see the data on Kibana as well. where as file input option not working in windows, especially .log files . I… --- ## [Missing geo.location](https://discuss.elastic.co/t/missing-geo-location/205595)
**Author:** [@Jan\_Kaspar](https://discuss.elastic.co/u/Jan_Kaspar)\ **Replies:** 3\ **Last updated:** [October 29, 2019, 5:23pm UTC](https://discuss.elastic.co/t/missing-geo-location/205595 "2019-10-29T17:23:06Z")
Hi all, I would like to ask for help. I am parsing data from our firewall and there are fields containing IP address. I would like to use Coordinates map to visualize accesses. Problem is that i am missing geo.location … --- ## [Logstash Error onstomp](https://discuss.elastic.co/t/logstash-error-onstomp/205728)
**Author:** [@pmatreides](https://discuss.elastic.co/u/pmatreides)\ **Replies:** 0\ **Last updated:** [October 29, 2019, 5:22pm UTC](https://discuss.elastic.co/t/logstash-error-onstomp/205728 "2019-10-29T17:22:53Z")
I'm no longer seeing messages in my KIbana dashboard. When I restart all of the services I'm getting a stop error which is: Error: OnSTomp::ConnectionTimeoutError the error stack points to /usr/share/logstash/vendor/b… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=453) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=455)