# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=893 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 894 --- ## [How to parse a json message](https://discuss.elastic.co/t/how-to-parse-a-json-message/103652)
**Author:** [@Navinder](https://discuss.elastic.co/u/Navinder)\ **Replies:** 7\ **Last updated:** [October 12, 2017, 11:02am UTC](https://discuss.elastic.co/t/how-to-parse-a-json-message/103652 "2017-10-12T11:02:03Z")
I have the following input in grok. { "payload": { "user\_id": "323232323323223233", "order\_id": "4232313-32323232-232322323", "ts": "2017-09-01T00:59:00.0138639Z", "promo\_codes": \[{ "amount": 10.0, }\], "shipping… --- ## [ISO8601 date format : malformed error message in elasticsearch](https://discuss.elastic.co/t/iso8601-date-format-malformed-error-message-in-elasticsearch/103250)
**Author:** [@ORich](https://discuss.elastic.co/u/ORich)\ **Replies:** 8\ **Last updated:** [October 12, 2017, 8:45am UTC](https://discuss.elastic.co/t/iso8601-date-format-malformed-error-message-in-elasticsearch/103250 "2017-10-12T08:45:57Z")
Dear experts, I am dealing with parsing issues with date format. The date format looks like ISO 8601 : grok filter is perfectly running but no way to keep the log date time stamp as @timestamp in the date section. This … --- ## [Logstash output not recognising columns in kibana](https://discuss.elastic.co/t/logstash-output-not-recognising-columns-in-kibana/103673)
**Author:** [@bobwilly](https://discuss.elastic.co/u/bobwilly)\ **Replies:** 0\ **Last updated:** [October 12, 2017, 7:52am UTC](https://discuss.elastic.co/t/logstash-output-not-recognising-columns-in-kibana/103673 "2017-10-12T07:52:19Z")
im trying to get my .CSV file in Kibana for visualisation. It feels like Im close to get it work but I can\`t figer out how to get my output right. In Kibana I see my .csv file as: message: News,test@email.com,10.10.10.… --- ## [Error when filter multiline of rabbitmq log](https://discuss.elastic.co/t/error-when-filter-multiline-of-rabbitmq-log/103654)
**Author:** [@vuxuanlai](https://discuss.elastic.co/u/vuxuanlai)\ **Replies:** 10\ **Last updated:** [October 12, 2017, 7:40am UTC](https://discuss.elastic.co/t/error-when-filter-multiline-of-rabbitmq-log/103654 "2017-10-12T07:40:00Z")
When i parse log from rabbitmq, i got the error as below: \` \[2017-10-12T04:01:18,006\]\[ERROR\]\[logstash.agent \] Cannot create pipeline {:reason=\>"The setting type in plugin multiline is obsolete and is no long… --- ## [Error during importing csv file in elastic search by logstash](https://discuss.elastic.co/t/error-during-importing-csv-file-in-elastic-search-by-logstash/103666)
**Author:** [@Paras\_Kohli](https://discuss.elastic.co/u/Paras_Kohli)\ **Replies:** 0\ **Last updated:** [October 12, 2017, 7:38am UTC](https://discuss.elastic.co/t/error-during-importing-csv-file-in-elastic-search-by-logstash/103666 "2017-10-12T07:38:27Z")
Bundler::GemNotFound: Could not find gem 'ftw (~\> 0.0.42) java' in any of the gem sources listed in your Gemfile or installed on this machine. verify\_gemfile\_dependencies\_are\_found! at C:/Users/C5262079/Downloads/logsta… --- ## [Logstash OOM - understanding heap sizing](https://discuss.elastic.co/t/logstash-oom-understanding-heap-sizing/103495)
**Author:** [@dandrestor](https://discuss.elastic.co/u/dandrestor)\ **Replies:** 13\ **Last updated:** [October 12, 2017, 6:36am UTC](https://discuss.elastic.co/t/logstash-oom-understanding-heap-sizing/103495 "2017-10-12T06:36:42Z")
Hello Could you help me understand logstash's memory requirements? I am currently getting these errors: \[2017-10-11T09:45:02,586\]\[INFO \]\[org.logstash.beats.BeatsHandler\] Exception: java.lang.OutOfMemoryError: Java heap… --- ## [How to read JSON file using logstash and convert String date format to Date time format](https://discuss.elastic.co/t/how-to-read-json-file-using-logstash-and-convert-string-date-format-to-date-time-format/103561)
**Author:** [@Vignesh\_Kumar](https://discuss.elastic.co/u/Vignesh_Kumar)\ **Replies:** 5\ **Last updated:** [October 12, 2017, 6:33am UTC](https://discuss.elastic.co/t/how-to-read-json-file-using-logstash-and-convert-string-date-format-to-date-time-format/103561 "2017-10-12T06:33:20Z")
The Input is like json, Need to get this "TimeStamp": "20170926110036" { "Properties": { "Client Name": "Chubb", "Portfolio": "Chubb-Transfer" }, "Capture": \[ { "CaptureGUID": "caa1f5ba-1e93-4926-b3ac-e30d0d9d4cb… --- ## [Grok\_filter pattern for different types of log events in a single log file](https://discuss.elastic.co/t/grok-filter-pattern-for-different-types-of-log-events-in-a-single-log-file/103616)
**Author:** [@mouryayi](https://discuss.elastic.co/u/mouryayi)\ **Replies:** 1\ **Last updated:** [October 12, 2017, 5:24am UTC](https://discuss.elastic.co/t/grok-filter-pattern-for-different-types-of-log-events-in-a-single-log-file/103616 "2017-10-12T05:24:50Z")
I had a log file in which some patterns is of one kind and some others are of another kind for suppose 20-Sep-2017 11:17:32.257 WARNING \[http-nio-8080-exec-405\] 2017-09-20 11:17:34,370 http-nio-8080-exec-334 WARN How … --- ## [Logstash - select data by Range](https://discuss.elastic.co/t/logstash-select-data-by-range/103581)
**Author:** [@pksingla](https://discuss.elastic.co/u/pksingla)\ **Replies:** 1\ **Last updated:** [October 12, 2017, 5:22am UTC](https://discuss.elastic.co/t/logstash-select-data-by-range/103581 "2017-10-12T05:22:42Z")
We have following web log transaction data. we need to count number of transactions every 30 second. input data file: 2017-10-11 10:42:19,server1,load,1 2017-10-11 10:42:19,server1,load,1 2017-10-11 10:42:20,server1,… --- ## [Grok pattern failing domain.com vs http://domain.com/](https://discuss.elastic.co/t/grok-pattern-failing-domain-com-vs-http-domain-com/103268)
**Author:** [@prophoto](https://discuss.elastic.co/u/prophoto)\ **Replies:** 14\ **Last updated:** [October 12, 2017, 5:20am UTC](https://discuss.elastic.co/t/grok-pattern-failing-domain-com-vs-http-domain-com/103268 "2017-10-12T05:20:49Z")
I am relatively new to ELK and have gotten pretty far setting up my custom grok pattern for apache logs. I have them slightly customized. I have it 90% of the way there but I'm seeing a failure when the log line has "dom… --- ## [File input plugin not sending to elasticsearch](https://discuss.elastic.co/t/file-input-plugin-not-sending-to-elasticsearch/102482)
**Author:** [@cm47](https://discuss.elastic.co/u/cm47)\ **Replies:** 3\ **Last updated:** [October 12, 2017, 5:19am UTC](https://discuss.elastic.co/t/file-input-plugin-not-sending-to-elasticsearch/102482 "2017-10-12T05:19:25Z")
Hi, I am trying to index AWS CloudTrail logs in elastic search using the file input plugin for logstash. The format of the JSON file it is reading is multiline events of json, separated by new line. From my understandi… --- ## [Logstash: parsing snmp traps](https://discuss.elastic.co/t/logstash-parsing-snmp-traps/103208)
**Author:** [@Robert.I](https://discuss.elastic.co/u/Robert.I)\ **Replies:** 3\ **Last updated:** [October 12, 2017, 5:14am UTC](https://discuss.elastic.co/t/logstash-parsing-snmp-traps/103208 "2017-10-12T05:14:17Z")
Hello All, At this moment, all the traps that i received in Logstash that have the error code, will write in elastic and in a file: output { if \[type\] == "snmptrap" { elasticsearch { hosts =\> \["myhost.domain.com:9200"… --- ## [Logstash 6 includes both 2.6.2 and 2.8.2 for slf4j](https://discuss.elastic.co/t/logstash-6-includes-both-2-6-2-and-2-8-2-for-slf4j/103615)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 0\ **Last updated:** [October 11, 2017, 8:35pm UTC](https://discuss.elastic.co/t/logstash-6-includes-both-2-6-2-and-2-8-2-for-slf4j/103615 "2017-10-11T20:35:11Z")
When I start logstash 6.0.0-rc1 after upgrading from 5.6.2 it complains that there are two bindings in the path SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in \[jar:file:/usr/share/logstash/l… --- ## [Snmptrap input sample messages](https://discuss.elastic.co/t/snmptrap-input-sample-messages/103257)
**Author:** [@krishna\_chaitanya](https://discuss.elastic.co/u/krishna_chaitanya)\ **Replies:** 1\ **Last updated:** [October 11, 2017, 8:22pm UTC](https://discuss.elastic.co/t/snmptrap-input-sample-messages/103257 "2017-10-11T20:22:39Z")
Hi, I am trying out Logstash's snmptrap input plugin to send snmptrap messages to Elasticsearch. I don't have SNMP agent installed on my machine. But, I am trying to generate sample SNMP trap messages using Netcat, but … --- ## [Logstash Kafka plugin error](https://discuss.elastic.co/t/logstash-kafka-plugin-error/103599)
**Author:** [@ashvita](https://discuss.elastic.co/u/ashvita)\ **Replies:** 0\ **Last updated:** [October 11, 2017, 6:37pm UTC](https://discuss.elastic.co/t/logstash-kafka-plugin-error/103599 "2017-10-11T18:37:44Z")
Hi, I have configured Logstash to receive from 2 Kafka topics using 2 diff conf files but it is only receiving from 1 topic. I can see messages coming on console consumer but logstash is unable to fetch it. I am unable… --- ## [Logstash fails to create index in elasticsearch after upgrading to 5.6.3](https://discuss.elastic.co/t/logstash-fails-to-create-index-in-elasticsearch-after-upgrading-to-5-6-3/103589)
**Author:** [@imortalsolitude](https://discuss.elastic.co/u/imortalsolitude)\ **Replies:** 0\ **Last updated:** [October 11, 2017, 5:58pm UTC](https://discuss.elastic.co/t/logstash-fails-to-create-index-in-elasticsearch-after-upgrading-to-5-6-3/103589 "2017-10-11T17:58:42Z")
I recently upgraded my ELK stack from 5.6.1 to 5.6.3 and now logstash can no longer create index in elasticsearch. I have another instance running with 5.6.1 with x-pack and it has no issues. The one running on 5.6.3 doe… --- ## [Malware detected on download](https://discuss.elastic.co/t/malware-detected-on-download/103442)
**Author:** [@ra.girard](https://discuss.elastic.co/u/ra.girard)\ **Replies:** 1\ **Last updated:** [October 11, 2017, 4:50pm UTC](https://discuss.elastic.co/t/malware-detected-on-download/103442 "2017-10-11T16:50:23Z")
I can't download logstash, I always have a malware detection error from our internal security tool URL: https://artifacts.elastic.co/downloads/logstash/logstash-5.6.3.zip Categories URL: Internet Services Reputation: … --- ## [Error: Expected one of #, =\> at line](https://discuss.elastic.co/t/error-expected-one-of-at-line/103579)
**Author:** [@maddie0726](https://discuss.elastic.co/u/maddie0726)\ **Replies:** 3\ **Last updated:** [October 11, 2017, 4:26pm UTC](https://discuss.elastic.co/t/error-expected-one-of-at-line/103579 "2017-10-11T16:26:08Z")
I have Logstash Kibana and Elasticsearch (all version 5.6.2) configured and running on Windows Server 2012 R2, running through a remote desktop connection. I am working on my logstash config file to read data from a .txt… --- ## [Reading previous week indice with elastic input plugin](https://discuss.elastic.co/t/reading-previous-week-indice-with-elastic-input-plugin/103236)
**Author:** [@RickT](https://discuss.elastic.co/u/RickT)\ **Replies:** 3\ **Last updated:** [October 11, 2017, 3:01pm UTC](https://discuss.elastic.co/t/reading-previous-week-indice-with-elastic-input-plugin/103236 "2017-10-11T15:01:10Z")
Hello, today, a mystery... How to read only the previous week indice of Elastic using input plugin Elastic (in logstash conf). For example, today just the 40th week. The actives indices are : data\_test-2017.39 data… --- ## [Yield called out of block](https://discuss.elastic.co/t/yield-called-out-of-block/103551)
**Author:** [@kanchan](https://discuss.elastic.co/u/kanchan)\ **Replies:** 1\ **Last updated:** [October 11, 2017, 2:26pm UTC](https://discuss.elastic.co/t/yield-called-out-of-block/103551 "2017-10-11T14:26:48Z")
Hi, I use code below in configuration file to create new event,but it is giving exception. filter { ruby { init =\> "require 'json'" code =\>" clientValue = event.get('message') field… --- ## [Logstash not working as a service](https://discuss.elastic.co/t/logstash-not-working-as-a-service/103560)
**Author:** [@Amar4d4](https://discuss.elastic.co/u/Amar4d4)\ **Replies:** 0\ **Last updated:** [October 11, 2017, 1:46pm UTC](https://discuss.elastic.co/t/logstash-not-working-as-a-service/103560 "2017-10-11T13:46:18Z")
logstash is working fine for a process when started through command but it is not logging when it is run as a service using sudo systemctl start logstash.service It is active when i given above command but it is not l… --- ## [Wrong ruby code](https://discuss.elastic.co/t/wrong-ruby-code/103546)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 1\ **Last updated:** [October 11, 2017, 12:56pm UTC](https://discuss.elastic.co/t/wrong-ruby-code/103546 "2017-10-11T12:56:46Z")
ruby { # isolate the server name from the "source" field # adding the field 'inputserver' code =\> 'event.set("inputserver", event.get("source.split('/', -… --- ## [What happens to events that are not processed by any output](https://discuss.elastic.co/t/what-happens-to-events-that-are-not-processed-by-any-output/102417)
**Author:** [@rbrandstaedter](https://discuss.elastic.co/u/rbrandstaedter)\ **Replies:** 10\ **Last updated:** [October 11, 2017, 11:56am UTC](https://discuss.elastic.co/t/what-happens-to-events-that-are-not-processed-by-any-output/102417 "2017-10-11T11:56:30Z")
I'm wondering what happens to events that are not written by any output plugin? Reason why I'm asking is that I have a logstash instance running for a couple of days and for no reason (no increased load) it dies with an… --- ## [Create new event in ruby](https://discuss.elastic.co/t/create-new-event-in-ruby/103535)
**Author:** [@kanchan](https://discuss.elastic.co/u/kanchan)\ **Replies:** 0\ **Last updated:** [October 11, 2017, 11:15am UTC](https://discuss.elastic.co/t/create-new-event-in-ruby/103535 "2017-10-11T11:15:57Z")
Hi, I am unable to create new event in filter tab.my configuration file code is: input { beats { port =\> 5044 } } filter { ruby { init =\> "require 'json'" code =\>" clientValue = event.get('messag… --- ## [Can not install Lumberjack plugin offline on Ubuntu](https://discuss.elastic.co/t/can-not-install-lumberjack-plugin-offline-on-ubuntu/103167)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 7\ **Last updated:** [October 11, 2017, 11:11am UTC](https://discuss.elastic.co/t/can-not-install-lumberjack-plugin-offline-on-ubuntu/103167 "2017-10-11T11:11:51Z")
Hi, I am trying to install Lumberjack but it errors out, root@elknode2:/opt/logstash# ./bin/logstash-plugin install file:///opt/logstash-input-lumberjack-3.1.3.zip Validating file:///opt/logstash-input-lumberjack-3.1.3… --- ## [Elastic stack - get last message which fails on logstash](https://discuss.elastic.co/t/elastic-stack-get-last-message-which-fails-on-logstash/100867)
**Author:** [@Mojster](https://discuss.elastic.co/u/Mojster)\ **Replies:** 7\ **Last updated:** [October 11, 2017, 10:22am UTC](https://discuss.elastic.co/t/elastic-stack-get-last-message-which-fails-on-logstash/100867 "2017-10-11T10:22:47Z")
I'm indexing custom longs into Logstash. In some file I'm getting timeouts & I can't determine which line is causing them. As I can se, Filebeats is shipping them in batches. Is there any option to get this single mes… --- ## [\ **Author:** [@Mayotte](https://discuss.elastic.co/u/Mayotte)\ **Replies:** 1\ **Last updated:** [October 11, 2017, 9:46am UTC](https://discuss.elastic.co/t/udp-0-error-unable-to-copy-file-var-log-logstash-logstash-plain-2017-10-11-log/103524 "2017-10-11T09:46:01Z") Hello, I configured Logstash for netflow packets. Devices send correctly packets to my server. In /var/log/logstash/logstash-plain.log I can see that logstash recover packets. The issue is that data are not sent to m… --- ## [XML Multiline pattern problems with Logstash](https://discuss.elastic.co/t/xml-multiline-pattern-problems-with-logstash/103350)
**Author:** [@MiguelLopes](https://discuss.elastic.co/u/MiguelLopes)\ **Replies:** 0\ **Last updated:** [October 10, 2017, 11:09am UTC](https://discuss.elastic.co/t/xml-multiline-pattern-problems-with-logstash/103350 "2017-10-10T11:09:43Z")
Hello all, We are having difficulties setting up a multiline pattern that must be triggered whenever xml data is found in the event. We are facing the following problem.. If we set the timestamp as an multi-line patter… --- ## [Input plugin vs filter plugin](https://discuss.elastic.co/t/input-plugin-vs-filter-plugin/103492)
**Author:** [@Ondro\_Tadanai](https://discuss.elastic.co/u/Ondro_Tadanai)\ **Replies:** 0\ **Last updated:** [October 11, 2017, 7:54am UTC](https://discuss.elastic.co/t/input-plugin-vs-filter-plugin/103492 "2017-10-11T07:54:33Z")
Hi all, i have question regarding to logstash architecture, i will use twitter-input-plugin to scrape some tweets in real-time. When I get tweet from logstash-twitter-input i need to call 2 web-services to get additio… --- ## [Logstash replace string problem](https://discuss.elastic.co/t/logstash-replace-string-problem/103401)
**Author:** [@Arkadiusz](https://discuss.elastic.co/u/Arkadiusz)\ **Replies:** 6\ **Last updated:** [October 11, 2017, 7:35am UTC](https://discuss.elastic.co/t/logstash-replace-string-problem/103401 "2017-10-11T07:35:13Z")
Hi I'm new to the logstash and don't know if what I want to do is even possible. I need to replace some information in logs provided by an application. My example input is: 2017-09-18 10:45:25,404 ERROR some variable… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=892) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=894)