# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1045 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1046 --- ## [Date section of filter generating a \_grokparsefailure](https://discuss.elastic.co/t/date-section-of-filter-generating-a--grokparsefailure/73055)
**Author:** [@lexnn](https://discuss.elastic.co/u/lexnn)\ **Replies:** 4\ **Last updated:** [January 31, 2017, 12:56pm UTC](https://discuss.elastic.co/t/date-section-of-filter-generating-a--grokparsefailure/73055 "2017-01-31T12:56:14Z")
Hi, I'm new to Logstash and I was wondering if someone could point me in the right direction. I'm using Logstash version 2.4 and setup this filter to parse out IIS logs from Azure. If I comment out the data section… --- ## [Filebeat to Logstash ERROR, wsasend, wsarecv](https://discuss.elastic.co/t/filebeat-to-logstash-error-wsasend-wsarecv/73363)
**Author:** [@loutocky](https://discuss.elastic.co/u/loutocky)\ **Replies:** 0\ **Last updated:** [January 31, 2017, 12:40pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-error-wsasend-wsarecv/73363 "2017-01-31T12:40:24Z")
Hi, I am using ELK stack version 5.1.2 and I have problem with sending logs from one worker (node) to central server. Everything I configured on localhost and it worked perfectly, but on development environment not. On… --- ## [Logstash java api](https://discuss.elastic.co/t/logstash-java-api/73323)
**Author:** [@sharongur](https://discuss.elastic.co/u/sharongur)\ **Replies:** 1\ **Last updated:** [January 31, 2017, 8:22am UTC](https://discuss.elastic.co/t/logstash-java-api/73323 "2017-01-31T08:22:34Z")
Is there a java api i can use with logstash? My Main goal is to insert a large CSV object into ES. I wrote a java program that inserts an index into ES. I read logstash can import CSV file very easily. Is there a… --- ## [Extract File Name from source](https://discuss.elastic.co/t/extract-file-name-from-source/73288)
**Author:** [@sde](https://discuss.elastic.co/u/sde)\ **Replies:** 2\ **Last updated:** [January 31, 2017, 7:27am UTC](https://discuss.elastic.co/t/extract-file-name-from-source/73288 "2017-01-31T07:27:33Z")
Hello, I am trying to read log file using Filebeat from an application server and write to a remote directory using Logstash. I am not able to extract the filename from the source field. Need help to extract the file… --- ## [Log file tailing | Logstash Configuration](https://discuss.elastic.co/t/log-file-tailing-logstash-configuration/73312)
**Author:** [@Joseph](https://discuss.elastic.co/u/Joseph)\ **Replies:** 2\ **Last updated:** [January 31, 2017, 6:48am UTC](https://discuss.elastic.co/t/log-file-tailing-logstash-configuration/73312 "2017-01-31T06:48:19Z")
Hi, I have a local log file which is updated constantly. I have the following input filter in place: file { path =\> \["/home/joseph/Desktop/audit.log"\] start\_position =\> "beginning" sin… --- ## [How to properly shutdown an input plugin](https://discuss.elastic.co/t/how-to-properly-shutdown-an-input-plugin/72809)
**Author:** [@djannot](https://discuss.elastic.co/u/djannot)\ **Replies:** 3\ **Last updated:** [January 31, 2017, 6:41am UTC](https://discuss.elastic.co/t/how-to-properly-shutdown-an-input-plugin/72809 "2017-01-31T06:41:06Z")
I'm trying to modify an input plugin to stop under some conditions, but when I execute the stop function, it generates an exception. How can I shutdown properly ? --- ## [Using filebeat to pass log files to logstash error](https://discuss.elastic.co/t/using-filebeat-to-pass-log-files-to-logstash-error/72860)
**Author:** [@brianwong](https://discuss.elastic.co/u/brianwong)\ **Replies:** 2\ **Last updated:** [January 31, 2017, 2:51am UTC](https://discuss.elastic.co/t/using-filebeat-to-pass-log-files-to-logstash-error/72860 "2017-01-31T02:51:24Z")
Hi I have setup both logstash and filebeat such that filebeat will send logs to localhost:5043 while logstash will listen for input on port 5043. However I keep getting the error " ERR Connecting error publishing events… --- ## [Multiline Codec Not Retrieving Any Data](https://discuss.elastic.co/t/multiline-codec-not-retrieving-any-data/73240)
**Author:** [@seth.yes](https://discuss.elastic.co/u/seth.yes)\ **Replies:** 1\ **Last updated:** [January 30, 2017, 11:12pm UTC](https://discuss.elastic.co/t/multiline-codec-not-retrieving-any-data/73240 "2017-01-30T23:12:35Z")
I'm using the Multiline codec with the File input in hopes to retrieve multiline log files from a file that is being populated on the same box I'm running Logstash (v5.1.2) & Elasticsearch (v.2.4.2). I've isolated just … --- ## [Grok filter for log4j logs inorder to parse messages from logs](https://discuss.elastic.co/t/grok-filter-for-log4j-logs-inorder-to-parse-messages-from-logs/72700)
**Author:** [@ritika](https://discuss.elastic.co/u/ritika)\ **Replies:** 4\ **Last updated:** [January 25, 2017, 7:16pm UTC](https://discuss.elastic.co/t/grok-filter-for-log4j-logs-inorder-to-parse-messages-from-logs/72700 "2017-01-25T19:16:35Z")
Hi, I am completely new to ELK and installed ELK stack on my windows computer. My goal is to read and analyse log files (which are actually log4j files) sitting on different server. I am able make logstash read my fil… --- ## [Logstash not take from queue](https://discuss.elastic.co/t/logstash-not-take-from-queue/73276)
**Author:** [@jojojo](https://discuss.elastic.co/u/jojojo)\ **Replies:** 2\ **Last updated:** [January 30, 2017, 9:07pm UTC](https://discuss.elastic.co/t/logstash-not-take-from-queue/73276 "2017-01-30T21:07:23Z")
i have logstash the take JSON from rabbit and put on elastic the JSON come to rabbit but i see that the json stay there and not take out. when i reset the logstah service that not help(or help for little time) when i res… --- ## [Storing logs: 'message' field doubles index size?](https://discuss.elastic.co/t/storing-logs-message-field-doubles-index-size/73263)
**Author:** [@John16](https://discuss.elastic.co/u/John16)\ **Replies:** 1\ **Last updated:** [January 30, 2017, 7:21pm UTC](https://discuss.elastic.co/t/storing-logs-message-field-doubles-index-size/73263 "2017-01-30T19:21:45Z")
Hello, I am using filebeat + logstash to store logs in ES. When I browse my index data, I see structured fields my log lines are parsed into (date, http return code, request line, etc) as well as 'message' field which… --- ## [Processing multiline data](https://discuss.elastic.co/t/processing-multiline-data/73046)
**Author:** [@pajesz](https://discuss.elastic.co/u/pajesz)\ **Replies:** 2\ **Last updated:** [January 30, 2017, 7:00pm UTC](https://discuss.elastic.co/t/processing-multiline-data/73046 "2017-01-30T19:00:10Z")
Hello, I want to process an exception log file but the problem is that the data is multiline with exception message, datetime etc in the first row and further random number of rows that is the stack trace. I want to pa… --- ## [Http output not working : http output failure for Logstash 5.0\[SOLVED\]](https://discuss.elastic.co/t/http-output-not-working-http-output-failure-for-logstash-5-0-solved/71517)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 3\ **Last updated:** [January 30, 2017, 6:54pm UTC](https://discuss.elastic.co/t/http-output-not-working-http-output-failure-for-logstash-5-0-solved/71517 "2017-01-30T18:54:20Z")
Hi All Here is my output conf file, i am not able to post data to endpoint output { http { url =\> "https://abcd.com/dataEventRequests" http\_method =\> "post" content\_type =\> "application/vn… --- ## [Only String and Array types are splittable. field:\[field\]\[Name\] is of type = Hash](https://discuss.elastic.co/t/only-string-and-array-types-are-splittable-field-field-name-is-of-type-hash/73252)
**Author:** [@graham.dalessandro](https://discuss.elastic.co/u/graham.dalessandro)\ **Replies:** 1\ **Last updated:** [January 30, 2017, 5:23pm UTC](https://discuss.elastic.co/t/only-string-and-array-types-are-splittable-field-field-name-is-of-type-hash/73252 "2017-01-30T17:23:58Z")
Hello, I just upgraded from LS 2.4 to 5.1.2 and i am having an issue with splitting values using the "split" filter. This exact configuration works in 2.4 without any issue but i am unable to get past the error in 5.1… --- ## [JDBC driver classpath not mapped even i mentioned](https://discuss.elastic.co/t/jdbc-driver-classpath-not-mapped-even-i-mentioned/73250)
**Author:** [@gmmurugan](https://discuss.elastic.co/u/gmmurugan)\ **Replies:** 0\ **Last updated:** [January 30, 2017, 5:10pm UTC](https://discuss.elastic.co/t/jdbc-driver-classpath-not-mapped-even-i-mentioned/73250 "2017-01-30T17:10:52Z")
Im getting this error even thought i set my driver paths in classpath of jdk(/etc/profile.d/jdk.sh) as export CLASSPATH=$JAVA\_HOME/lib/dt.jar:$JAVA\_HOME/lib/tools.jar:$JAVA\_HOME/lib/db2jcc\_license\_cisuz.jar:$JAVA\_HOME/li… --- ## [Logstash fail to parse dates when sending to elastic but not on rybydebug](https://discuss.elastic.co/t/logstash-fail-to-parse-dates-when-sending-to-elastic-but-not-on-rybydebug/73248)
**Author:** [@lmalossi](https://discuss.elastic.co/u/lmalossi)\ **Replies:** 0\ **Last updated:** [January 30, 2017, 5:04pm UTC](https://discuss.elastic.co/t/logstash-fail-to-parse-dates-when-sending-to-elastic-but-not-on-rybydebug/73248 "2017-01-30T17:04:46Z")
Hi, I get messages tagged with "dateparsefailure" only when sending messages to elastic but not on rubydebug this is my typical message: \<02srv006y6.ad02.eni.intranet\> \<\> \<\> \<29d1948389fc1539:5ed6b398:159d6d2c9a… --- ## [JSON formatted log contains "message" field confusing Logstash](https://discuss.elastic.co/t/json-formatted-log-contains-message-field-confusing-logstash/65028)
**Author:** [@JeremyinNC](https://discuss.elastic.co/u/JeremyinNC)\ **Replies:** 11\ **Last updated:** [January 30, 2017, 4:29pm UTC](https://discuss.elastic.co/t/json-formatted-log-contains-message-field-confusing-logstash/65028 "2017-01-30T16:29:54Z")
We have servers which are writing their logs out in JSON format. An example: {"message":{"description":"System error, contact application support team","exception":"java.util.NoSuchElementException","user\_agent":"Jodd … --- ## [Doubts about creating indexes based on log timestamps](https://discuss.elastic.co/t/doubts-about-creating-indexes-based-on-log-timestamps/73120)
**Author:** [@lcanas](https://discuss.elastic.co/u/lcanas)\ **Replies:** 4\ **Last updated:** [January 30, 2017, 4:21pm UTC](https://discuss.elastic.co/t/doubts-about-creating-indexes-based-on-log-timestamps/73120 "2017-01-30T16:21:03Z")
Hi guys! during the last days I've been reading documentation in order to get my apache logs imported in different indexes based on the date of the log entries. With the configuration below I got it running with indexe… --- ## [Logstash is not listening on any port](https://discuss.elastic.co/t/logstash-is-not-listening-on-any-port/45088)
**Author:** [@guy.naftaly](https://discuss.elastic.co/u/guy.naftaly)\ **Replies:** 4\ **Last updated:** [January 30, 2017, 3:43pm UTC](https://discuss.elastic.co/t/logstash-is-not-listening-on-any-port/45088 "2017-01-30T15:43:43Z")
hello all, I installed Elasticsearch/Logstash/Kibana on CentOS 7 . Kibana and Elasticsearch works fine. Logstash version logstash-2.2 I have an issue with the Logstash – it is not listening on the input ports I created… --- ## [Logstash 5.1.1 - Couldn't find any filter plugin named 'multiline'](https://discuss.elastic.co/t/logstash-5-1-1-couldnt-find-any-filter-plugin-named-multiline/71263)
**Author:** [@MikeSiz](https://discuss.elastic.co/u/MikeSiz)\ **Replies:** 11\ **Last updated:** [January 30, 2017, 3:42pm UTC](https://discuss.elastic.co/t/logstash-5-1-1-couldnt-find-any-filter-plugin-named-multiline/71263 "2017-01-30T15:42:15Z")
Hi, I am trying to migrate from logstash 2.3.4 to newer 5.1.1. After default installation (registering logstash 5.1 repo and yum install logstash) I found that logstash cannot find "multiline' plugin. I tried: /usr/… --- ## [Parsing Asterisk Logs](https://discuss.elastic.co/t/parsing-asterisk-logs/73176)
**Author:** [@taurs](https://discuss.elastic.co/u/taurs)\ **Replies:** 3\ **Last updated:** [January 30, 2017, 12:25pm UTC](https://discuss.elastic.co/t/parsing-asterisk-logs/73176 "2017-01-30T12:25:00Z")
I am trying to Parse some Asterisk Event Logs. 1124909007|NONE|NONE|NONE|CONFIGRELOAD| Can anyone help me with this logstash grok for the logs mentioned above. --- ## [LS posts same document to ES multiple times](https://discuss.elastic.co/t/ls-posts-same-document-to-es-multiple-times/73197)
**Author:** [@dtrizna](https://discuss.elastic.co/u/dtrizna)\ **Replies:** 2\ **Last updated:** [January 30, 2017, 11:52am UTC](https://discuss.elastic.co/t/ls-posts-same-document-to-es-multiple-times/73197 "2017-01-30T11:52:28Z")
Hi, community! My ELK stack has some strange behavior. Recently I found, that every document in ES is indexed multiple times. Documents are Netflow flows. They have different \_id values, although it is the same docume… --- ## [Logstash 5.1 with JDBC input plug-in: Always Restarting](https://discuss.elastic.co/t/logstash-5-1-with-jdbc-input-plug-in-always-restarting/73178)
**Author:** [@Basem\_Mohammed](https://discuss.elastic.co/u/Basem_Mohammed)\ **Replies:** 1\ **Last updated:** [January 30, 2017, 10:54am UTC](https://discuss.elastic.co/t/logstash-5-1-with-jdbc-input-plug-in-always-restarting/73178 "2017-01-30T10:54:12Z")
I tried to read the content of table from my Database. I faced the problem of java heap memory in logstash 2.x But It was working with a small resultset .. When I upgrade to 5.1 and using X-Pack.. Now I couldn't manage t… --- ## [Logstash5.1.2-- allow proxy to download the plug-in from the internet](https://discuss.elastic.co/t/logstash5-1-2-allow-proxy-to-download-the-plug-in-from-the-internet/73083)
**Author:** [@rlv\_praveen](https://discuss.elastic.co/u/rlv_praveen)\ **Replies:** 2\ **Last updated:** [January 30, 2017, 10:07am UTC](https://discuss.elastic.co/t/logstash5-1-2-allow-proxy-to-download-the-plug-in-from-the-internet/73083 "2017-01-30T10:07:39Z")
Hi, I tried to update my Logstash mongo db plug in but it is showing me couldnt download the mongo db plug in. In one of the piost i saw as we have to enable the proxy to download, can you guys help me in setting the … --- ## [Using Elastic stack](https://discuss.elastic.co/t/using-elastic-stack/73171)
**Author:** [@avivc](https://discuss.elastic.co/u/avivc)\ **Replies:** 4\ **Last updated:** [January 30, 2017, 9:36am UTC](https://discuss.elastic.co/t/using-elastic-stack/73171 "2017-01-30T09:36:51Z")
Hi, As a new user of Elastic, I'm trying to to establish an architecture for my needs. I'll describe my Dilemmas: I'm using a service API(CA agile control) that holds thousands data records. In this API, it is possible t… --- ## [Logstash 5 with kafka input plugin 4.1.1 not decoding avro messages](https://discuss.elastic.co/t/logstash-5-with-kafka-input-plugin-4-1-1-not-decoding-avro-messages/73173)
**Author:** [@Naushad\_Ansari](https://discuss.elastic.co/u/Naushad_Ansari)\ **Replies:** 0\ **Last updated:** [January 30, 2017, 9:31am UTC](https://discuss.elastic.co/t/logstash-5-with-kafka-input-plugin-4-1-1-not-decoding-avro-messages/73173 "2017-01-30T09:31:39Z")
I am using Logstash 5.1.2 with logstash-input-kafka 4.1.1 plugin to read Avro data from kafka version 0.9 but I am getting error after starting logstash and logstash stops. I am using logstash-codec-avro 3.0.0-java. I… --- ## [Delete "field" inside array](https://discuss.elastic.co/t/delete-field-inside-array/72867)
**Author:** [@adrianolimit](https://discuss.elastic.co/u/adrianolimit)\ **Replies:** 10\ **Last updated:** [January 30, 2017, 8:59am UTC](https://discuss.elastic.co/t/delete-field-inside-array/72867 "2017-01-30T08:59:05Z")
Hello everyone, I'm using the "Twitter input" and I want to delete some fields that I don't need. For example, I would like to remove (so not index in Elasticsearch), the field "indices" inside the field "\[entities… --- ## [\[Solved\] Logstash 5 not starting when using Kafka input version 4](https://discuss.elastic.co/t/solved-logstash-5-not-starting-when-using-kafka-input-version-4/64716)
**Author:** [@IngaFeick](https://discuss.elastic.co/u/IngaFeick)\ **Replies:** 4\ **Last updated:** [January 30, 2017, 8:13am UTC](https://discuss.elastic.co/t/solved-logstash-5-not-starting-when-using-kafka-input-version-4/64716 "2017-01-30T08:13:07Z")
Hallos! I set up a logstash 5.0.0 and downgraded the kafka plugin version as described in the breaking changes document (https://www.elastic.co/guide/en/logstash/5.0/breaking-changes.html) because our kafka cluster run… --- ## [Indexing a Modbus over TCP packet with Logstash](https://discuss.elastic.co/t/indexing-a-modbus-over-tcp-packet-with-logstash/44622)
**Author:** [@nba](https://discuss.elastic.co/u/nba)\ **Replies:** 1\ **Last updated:** [January 30, 2017, 7:02am UTC](https://discuss.elastic.co/t/indexing-a-modbus-over-tcp-packet-with-logstash/44622 "2017-01-30T07:02:52Z")
I am trying to index a pcap capture witch composed by a Modbus/TCP encapsulation. So before writing my own modbus.conf file to parse all field, I want to know is there something like that ? I am looking for a simply pcap… --- ## [Ignore unwanted data from message field multiline logstash](https://discuss.elastic.co/t/ignore-unwanted-data-from-message-field-multiline-logstash/73016)
**Author:** [@Aditya\_Srivastava](https://discuss.elastic.co/u/Aditya_Srivastava)\ **Replies:** 1\ **Last updated:** [January 30, 2017, 6:33am UTC](https://discuss.elastic.co/t/ignore-unwanted-data-from-message-field-multiline-logstash/73016 "2017-01-30T06:33:41Z")
I am using multiline input codec in logstash with currently no filter. Below is a sample of how my output looks. "@timestamp" =\> "2017-01-27T12:03:46.189Z", "message" =\> "{\\"@timestamp\\":\\"2017-01-27T11:35:55.477Z\\"… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1044) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1046)