# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=988 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 989 --- ## [JSON parse - not seeing the newlines](https://discuss.elastic.co/t/json-parse-not-seeing-the-newlines/84911)
**Author:** [@Vedran\_Maricevic](https://discuss.elastic.co/u/Vedran_Maricevic)\ **Replies:** 2\ **Last updated:** [May 8, 2017, 11:55am UTC](https://discuss.elastic.co/t/json-parse-not-seeing-the-newlines/84911 "2017-05-08T11:55:15Z")
Using ELK 5.0.0. I am trying to parse simple JSON document: {"timestamp":"2012-01-01 02:00:01", "severity":"ERROR", "messages":"Foo failed", "fieldone": "this if the value of a field one", "fieldtwo": "ttthis if the v… --- ## [Does Logstash has a limit size for each event-message?](https://discuss.elastic.co/t/does-logstash-has-a-limit-size-for-each-event-message/26268)
**Author:** [@Akudva](https://discuss.elastic.co/u/Akudva)\ **Replies:** 6\ **Last updated:** [May 8, 2017, 11:14am UTC](https://discuss.elastic.co/t/does-logstash-has-a-limit-size-for-each-event-message/26268 "2017-05-08T11:14:15Z")
Hello, I am using file input plugin and wanted to know if Logstash has a size limit for each message/event? If yes, is it configurable? I am not able to find it in any documentation. Thanks, Ashish --- ## [Date match failure failure with given format](https://discuss.elastic.co/t/date-match-failure-failure-with-given-format/84890)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 3\ **Last updated:** [May 8, 2017, 10:54am UTC](https://discuss.elastic.co/t/date-match-failure-failure-with-given-format/84890 "2017-05-08T10:54:47Z")
Hi, I am getting below error in ES logs. This is related to the match failure. I don't how it appears in ES? \[2017-05-08 10:30:19,469\]\[WARN \]\[indices.cluster \] \[data-node-3\] \[\[esb-2017.05.08\]\[0\]\] marking and… --- ## [CSV date parser failure while uploading it to logstash](https://discuss.elastic.co/t/csv-date-parser-failure-while-uploading-it-to-logstash/84722)
**Author:** [@Manjunath.premkumar](https://discuss.elastic.co/u/Manjunath.premkumar)\ **Replies:** 5\ **Last updated:** [May 8, 2017, 10:34am UTC](https://discuss.elastic.co/t/csv-date-parser-failure-while-uploading-it-to-logstash/84722 "2017-05-08T10:34:24Z")
Hi , I am stuck in parsing the date field in CSV and tried my best but no luck. CSV contains 2 fields, Data and Responsetime (separated by comma) and I want to plot graph in KIBANA with x axis as Date and response as Y… --- ## [Logstash - shutdown process appears to be stalled](https://discuss.elastic.co/t/logstash-shutdown-process-appears-to-be-stalled/84916)
**Author:** [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)\ **Replies:** 0\ **Last updated:** [May 8, 2017, 10:22am UTC](https://discuss.elastic.co/t/logstash-shutdown-process-appears-to-be-stalled/84916 "2017-05-08T10:22:26Z")
I am getting this error out of nowhere. Logstash was running correctly and then this error occured, \[2017-05-08T10:16:43,819\]\[WARN \]\[logstash.agent \] stopping pipeline {:id=\>"main"} \[2017-05-08T10:16:49,190\]\[… --- ## [Elasticsearch input plugin not working](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266)
**Author:** [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Replies:** 14\ **Last updated:** [May 8, 2017, 7:38am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-not-working/84266 "2017-05-08T07:38:20Z")
Hi all, I'm trying a simple pipeline where I use elasticsearch to get data and send them again to elastic but updated. For some reason the pipeline stops... Here the config I'm using: input{ elasticsearch { … --- ## [Multiline log parsing](https://discuss.elastic.co/t/multiline-log-parsing/84251)
**Author:** [@shammi](https://discuss.elastic.co/u/shammi)\ **Replies:** 2\ **Last updated:** [May 8, 2017, 7:14am UTC](https://discuss.elastic.co/t/multiline-log-parsing/84251 "2017-05-08T07:14:10Z")
Hi, I am using grok pattern to get date from filename and that is working fine. Now i need to parse multiline log event along with date from filename. Log: 8480 12:33:38 ERROR Error while rendering view: '/Views/Arti… --- ## [Logstash Parsing Issue - Community Input request](https://discuss.elastic.co/t/logstash-parsing-issue-community-input-request/84807)
**Author:** [@andynz2017](https://discuss.elastic.co/u/andynz2017)\ **Replies:** 2\ **Last updated:** [May 8, 2017, 6:58am UTC](https://discuss.elastic.co/t/logstash-parsing-issue-community-input-request/84807 "2017-05-08T06:58:13Z")
Hi All, Got a bit of a curly issue which I'm reaching out to the community to help resolve. I have parsed out a field within Exchange 2010 Transport log which appears to be presentative of a string, but contains both… --- ## [Logstash logs all inputs to syslog](https://discuss.elastic.co/t/logstash-logs-all-inputs-to-syslog/84499)
**Author:** [@nicklaw](https://discuss.elastic.co/u/nicklaw)\ **Replies:** 2\ **Last updated:** [May 8, 2017, 1:27am UTC](https://discuss.elastic.co/t/logstash-logs-all-inputs-to-syslog/84499 "2017-05-08T01:27:18Z")
Logstash version: 5.3.2 OS: Ubuntu 16.04 I have issue that sees anything that comes through the Logstash Input get appended to the system's syslog, even if I have log.level: fatal set in the configuration file. Is this… --- ## [Unable to offline install the logstash-filter-translate plugin](https://discuss.elastic.co/t/unable-to-offline-install-the-logstash-filter-translate-plugin/55315)
**Author:** [@Valeriy](https://discuss.elastic.co/u/Valeriy)\ **Replies:** 5\ **Last updated:** [May 7, 2017, 11:27pm UTC](https://discuss.elastic.co/t/unable-to-offline-install-the-logstash-filter-translate-plugin/55315 "2017-05-07T23:27:56Z")
Hello, I install the plugin it shows this error: bin/plugin install --local /var/lib/user/logstash-filter-translate-2.1.4.gem Validating /var/lib/user/logstash-filter-translate-2.1.4.gem Installing logstash-filter-tran… --- ## [Conf File for Logstash is not working](https://discuss.elastic.co/t/conf-file-for-logstash-is-not-working/84835)
**Author:** [@kartikeya\_sharma\_13](https://discuss.elastic.co/u/kartikeya_sharma_13)\ **Replies:** 1\ **Last updated:** [May 7, 2017, 12:35pm UTC](https://discuss.elastic.co/t/conf-file-for-logstash-is-not-working/84835 "2017-05-07T12:35:26Z")
Is there anything wrong with this config file .I am right now logstash v 5.4.0 input { file{ path =\>"C:/ELK/Data/iis\_data.csv" sincedb\_path =\> "C:/ELK/Data/.sincedb" start\_position =\> "beginning" type=\>"csv" } file{ … --- ## [Need help with the Grok Pattern](https://discuss.elastic.co/t/need-help-with-the-grok-pattern/84831)
**Author:** [@karnamonkster](https://discuss.elastic.co/u/karnamonkster)\ **Replies:** 1\ **Last updated:** [May 7, 2017, 12:47pm UTC](https://discuss.elastic.co/t/need-help-with-the-grok-pattern/84831 "2017-05-07T12:47:07Z")
Hi, I need to match the following log messages within one event extract that i get from my head-end systems.\\ Event\_number 26.03.2017 00:00:06 252:location1 rtuid1 pipesizemm componentID1 LEVEL INDICATION Alarm limi… --- ## [Issue with logstash filter](https://discuss.elastic.co/t/issue-with-logstash-filter/84467)
**Author:** [@ashwin](https://discuss.elastic.co/u/ashwin)\ **Replies:** 3\ **Last updated:** [May 7, 2017, 4:57am UTC](https://discuss.elastic.co/t/issue-with-logstash-filter/84467 "2017-05-07T04:57:49Z")
Hello Everyone, I am using below command to load mongodb stats to Elasticsearch using logstatsh. input { exec { command =\> "mongo test\_mongo\_db\_1 --eval 'db.stats()' --quiet" interval =\> 7 type =\> "db.stats" … --- ## [Docker logs to logstash](https://discuss.elastic.co/t/docker-logs-to-logstash/84828)
**Author:** [@ab-agni](https://discuss.elastic.co/u/ab-agni)\ **Replies:** 0\ **Last updated:** [May 7, 2017, 2:38am UTC](https://discuss.elastic.co/t/docker-logs-to-logstash/84828 "2017-05-07T02:38:29Z")
Is there a recommended approach to send docker logs to log stash. I see 3rd party drivers like gelf, but want to know about choices like over tcp/udp or http based. --- ## [Getting geo\_point from Twitter](https://discuss.elastic.co/t/getting-geo-point-from-twitter/33529)
**Author:** [@JamesK](https://discuss.elastic.co/u/JamesK)\ **Replies:** 3\ **Last updated:** [May 6, 2017, 11:12pm UTC](https://discuss.elastic.co/t/getting-geo-point-from-twitter/33529 "2017-05-06T23:12:35Z")
Afternoon all, I've been using logstash to feed twitter data into elasticsearch so I can visualise it with kibana. Have successfully changed config file to work with last week's new releases, so no problems there, but I'… --- ## [S3 output plugin prefix string interpolation](https://discuss.elastic.co/t/s3-output-plugin-prefix-string-interpolation/84455)
**Author:** [@FlorinAndrei](https://discuss.elastic.co/u/FlorinAndrei)\ **Replies:** 1\ **Last updated:** [May 6, 2017, 7:19pm UTC](https://discuss.elastic.co/t/s3-output-plugin-prefix-string-interpolation/84455 "2017-05-06T19:19:53Z")
With the current Logstash version (5.3), and the S3 output plugin, I see the prefix option "supports string interpolation". My question is, can I interpolate based on current date/time, with multiple levels of directorie… --- ## [Logstash giving error regarding log4j](https://discuss.elastic.co/t/logstash-giving-error-regarding-log4j/84767)
**Author:** [@Rohit\_Singh](https://discuss.elastic.co/u/Rohit_Singh)\ **Replies:** 1\ **Last updated:** [May 6, 2017, 1:47pm UTC](https://discuss.elastic.co/t/logstash-giving-error-regarding-log4j/84767 "2017-05-06T13:47:26Z")
Using latest logstash 5.4.0, getting this error on startup ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Sending Logstash's logs to /var/log/ wh… --- ## [Logstash-filter-grok repeatedly](https://discuss.elastic.co/t/logstash-filter-grok-repeatedly/84712)
**Author:** [@kuziakhmedov](https://discuss.elastic.co/u/kuziakhmedov)\ **Replies:** 4\ **Last updated:** [May 6, 2017, 4:34am UTC](https://discuss.elastic.co/t/logstash-filter-grok-repeatedly/84712 "2017-05-06T04:34:13Z")
Good time of the day. I'm parsing log files like these 2017.04.25 09:55:22 \[LOG\] \[4cdb70e9-7e41-48c6-a3b4-889790630641\] \[10.4.26.21:19285\] Open 2017.04.25 09:55:22 \[LOG\] \[4cdb70e9-7e41-48c6-a3b4-889790630641\] \[10.4.26.21… --- ## [Race condition causes Exception when using json\_lines with UDP input](https://discuss.elastic.co/t/race-condition-causes-exception-when-using-json-lines-with-udp-input/84790)
**Author:** [@qmando](https://discuss.elastic.co/u/qmando)\ **Replies:** 0\ **Last updated:** [May 5, 2017, 9:55pm UTC](https://discuss.elastic.co/t/race-condition-causes-exception-when-using-json-lines-with-udp-input/84790 "2017-05-05T21:55:41Z")
I've discovered a race condition while using UDP input with json\_lines codec. I tested using the latest logstash docker image. Logstash config: input { udp { port =\> 13371 codec =\> json\_lines … --- ## [Howto Parse JSON into Hash](https://discuss.elastic.co/t/howto-parse-json-into-hash/84474)
**Author:** [@f9atanzh](https://discuss.elastic.co/u/f9atanzh)\ **Replies:** 5\ **Last updated:** [May 5, 2017, 5:43pm UTC](https://discuss.elastic.co/t/howto-parse-json-into-hash/84474 "2017-05-05T17:43:56Z")
Hello, I'm working on reconfiguring log messages that come in as JSON strings so that I can POST them to an HTTP service with headers. Using beats input for Logstash. The main hurdle here is that I'd like to convert th… --- ## [How to compare date in Logstash?](https://discuss.elastic.co/t/how-to-compare-date-in-logstash/32250)
**Author:** [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\ **Replies:** 3\ **Last updated:** [May 5, 2017, 5:01pm UTC](https://discuss.elastic.co/t/how-to-compare-date-in-logstash/32250 "2017-05-05T17:01:12Z")
Hi All, There is one requirement where i need to compare log date with some constant date. and then perform indexing in ES. is there any way where i can compare two dates in logstash? Thanks, Rohan --- ## [Cisco UCS message match](https://discuss.elastic.co/t/cisco-ucs-message-match/84760)
**Author:** [@kernelpanic](https://discuss.elastic.co/u/kernelpanic)\ **Replies:** 0\ **Last updated:** [May 5, 2017, 4:45pm UTC](https://discuss.elastic.co/t/cisco-ucs-message-match/84760 "2017-05-05T16:45:46Z")
Hello, we're receiving messages from the Cisco UCS system in the following format: \<181\>: 2017 May 5 17:34:40 BST: last message repeated 1 time The Grok debugger at https://grokdebug.herokuapp.com/ confirms the follow… --- ## [How to stop logstash with php exec](https://discuss.elastic.co/t/how-to-stop-logstash-with-php-exec/84626)
**Author:** [@henrilabarre](https://discuss.elastic.co/u/henrilabarre)\ **Replies:** 5\ **Last updated:** [May 5, 2017, 3:18pm UTC](https://discuss.elastic.co/t/how-to-stop-logstash-with-php-exec/84626 "2017-05-05T15:18:57Z")
Hi all I use php to manage input info and after use a php exec to load the logstash conf file like that : the php file : \ **Author:** [@mr\_chang7](https://discuss.elastic.co/u/mr_chang7)\ **Replies:** 4\ **Last updated:** [May 5, 2017, 3:26pm UTC](https://discuss.elastic.co/t/es-timestamp-from-logstash-collectd-codec/84734 "2017-05-05T15:26:11Z") I'm following the link below to set up the collectd and logstash while storing the data in the ES. https://www.elastic.co/guide/en/logstash/current/plugins-codecs-collectd.html Just wondering what does the timestamp… --- ## [Add field from JSON -logstash filter](https://discuss.elastic.co/t/add-field-from-json-logstash-filter/84721)
**Author:** [@lherrera](https://discuss.elastic.co/u/lherrera)\ **Replies:** 1\ **Last updated:** [May 5, 2017, 2:08pm UTC](https://discuss.elastic.co/t/add-field-from-json-logstash-filter/84721 "2017-05-05T14:08:00Z")
Continuing the discussion from Add field from JSON / logstash filter: Hi there, I am having difficulty extracting a field from a log message using the json filter, I hope you can help. I am using logstash 5.2.1, the … --- ## [Kibana not recognizing a date](https://discuss.elastic.co/t/kibana-not-recognizing-a-date/84715)
**Author:** [@Djelouah\_Laala](https://discuss.elastic.co/u/Djelouah_Laala)\ **Replies:** 1\ **Last updated:** [May 5, 2017, 2:02pm UTC](https://discuss.elastic.co/t/kibana-not-recognizing-a-date/84715 "2017-05-05T14:02:20Z")
Hey guys, I have two examples : "datetxecheclasthour":"2017-04-28T12:50:00.077Z" "txannczhier\_date":"2017-04-25 00:00:00 +0200" In the first case kibana recognize datetxecheclasthour as a date but in the second case i… --- ## [Wrong source name](https://discuss.elastic.co/t/wrong-source-name/84723)
**Author:** [@theplayer777](https://discuss.elastic.co/u/theplayer777)\ **Replies:** 0\ **Last updated:** [May 5, 2017, 1:34pm UTC](https://discuss.elastic.co/t/wrong-source-name/84723 "2017-05-05T13:34:45Z")
HI there, I have some troubles with the source field on logstash that is incorrect for som logs. Here is an example (taken from Kibana): message: 2017-05-05 14:12:48,395 \[ INFO\] \[ID=XXXXXX\] \[User=\] \[org.my.class\] Suc… --- ## [Logstash and syslog messages from Cisco UCS](https://discuss.elastic.co/t/logstash-and-syslog-messages-from-cisco-ucs/84713)
**Author:** [@kernelpanic](https://discuss.elastic.co/u/kernelpanic)\ **Replies:** 2\ **Last updated:** [May 5, 2017, 1:30pm UTC](https://discuss.elastic.co/t/logstash-and-syslog-messages-from-cisco-ucs/84713 "2017-05-05T13:30:38Z")
Hello, I've configured our Cisco UCS / Fabric Interconnect device to send its syslogs to Logstash, and using tcpdump -n dst port 514 I can see that the devices are connecting to that port on the Logstash node. However, … --- ## [Logstash jdbc input plugin multiple tables on elasticsearch output plugin](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-multiple-tables-on-elasticsearch-output-plugin/84506)
**Author:** [@Maurice\_Lionel\_Ndion](https://discuss.elastic.co/u/Maurice_Lionel_Ndion)\ **Replies:** 2\ **Last updated:** [May 5, 2017, 9:43am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-multiple-tables-on-elasticsearch-output-plugin/84506 "2017-05-05T09:43:38Z")
Hello everyone, I am delighted to be part of this community . I start with the elastic stack suite and i'm confronted with an issue I use the plugin jdbc of logstash to access a mysql bd containing seven tables, I th… --- ## [Logstash as service Advantages](https://discuss.elastic.co/t/logstash-as-service-advantages/84661)
**Author:** [@Kishore](https://discuss.elastic.co/u/Kishore)\ **Replies:** 3\ **Last updated:** [May 5, 2017, 9:19am UTC](https://discuss.elastic.co/t/logstash-as-service-advantages/84661 "2017-05-05T09:19:10Z")
Please advise me that, what are all the advantages to run the logstash as service instead of process. [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=987) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=989)