# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=964 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 965 --- ## [Kerberos support for Logstash 2.2.4](https://discuss.elastic.co/t/kerberos-support-for-logstash-2-2-4/89574)
**Author:** [@ankeet.jain](https://discuss.elastic.co/u/ankeet.jain)\ **Replies:** 0\ **Last updated:** [June 15, 2017, 2:17pm UTC](https://discuss.elastic.co/t/kerberos-support-for-logstash-2-2-4/89574 "2017-06-15T14:17:40Z")
Hello All, This is my first post in the discussion board and I am pretty much new to this technology stack ( ELK ). I am trying to connect with Kafka using logstash's Kafka plugin. I have listed version details for … --- ## [Slow Data loading to elasticsearch](https://discuss.elastic.co/t/slow-data-loading-to-elasticsearch/89261)
**Author:** [@gowtham.go](https://discuss.elastic.co/u/gowtham.go)\ **Replies:** 14\ **Last updated:** [June 15, 2017, 1:47pm UTC](https://discuss.elastic.co/t/slow-data-loading-to-elasticsearch/89261 "2017-06-15T13:47:50Z")
Hi, I am new to Elasticsearch. I am trying to create a reporting application with Elastic search as Data Store. I get Input files, try and index it into ES via Logstash, and search / filter in ES for report output. The … --- ## [Kafka Input plugin not working in Logstash 5.4.1](https://discuss.elastic.co/t/kafka-input-plugin-not-working-in-logstash-5-4-1/89389)
**Author:** [@gowtham.go](https://discuss.elastic.co/u/gowtham.go)\ **Replies:** 1\ **Last updated:** [June 15, 2017, 1:27pm UTC](https://discuss.elastic.co/t/kafka-input-plugin-not-working-in-logstash-5-4-1/89389 "2017-06-15T13:27:23Z")
Hi, I am a Novice and I am trying to use a Kafka stream as input for Logstash and load it to Elastic. One major doubt i am having is, If we use Kafka as source, where will the data be stored? (As per my understanding ES… --- ## [Logstash proftpd log](https://discuss.elastic.co/t/logstash-proftpd-log/42378)
**Author:** [@ismael\_boumedien](https://discuss.elastic.co/u/ismael_boumedien)\ **Replies:** 7\ **Last updated:** [June 15, 2017, 1:00pm UTC](https://discuss.elastic.co/t/logstash-proftpd-log/42378 "2017-06-15T13:00:56Z")
Hi guys, I'm working on logstash and I want to send proftpd and sftp logs to elasticsearch. But I'm facing some issues with doing that. No index is genreated in ES. Can any one help me :slightly\_smiling: File conf: #… --- ## [Date filter seems to remove message](https://discuss.elastic.co/t/date-filter-seems-to-remove-message/89329)
**Author:** [@Evertvz](https://discuss.elastic.co/u/Evertvz)\ **Replies:** 4\ **Last updated:** [June 15, 2017, 11:27am UTC](https://discuss.elastic.co/t/date-filter-seems-to-remove-message/89329 "2017-06-15T11:27:57Z")
Hello, My company is considering switching from graylog to logstash. For this I am setting up a proof of concept and I'm running into a strange problem when doing so. When using the date filter, the message disappears.… --- ## [Logstash doesn't receive syslog](https://discuss.elastic.co/t/logstash-doesnt-receive-syslog/89409)
**Author:** [@Pippo](https://discuss.elastic.co/u/Pippo)\ **Replies:** 2\ **Last updated:** [June 15, 2017, 8:03am UTC](https://discuss.elastic.co/t/logstash-doesnt-receive-syslog/89409 "2017-06-15T08:03:41Z")
Hi, I'm setting up Logstash to receive syslog, but it seems Logstash doesn't receive anything. I can monitor Logstash in Elasticsearch, and I see I have no received/sent events : I'm running ELK on Docker. Here i… --- ## [Data loss (logstash to webhdfs)](https://discuss.elastic.co/t/data-loss-logstash-to-webhdfs/89521)
**Author:** [@hello4usharath](https://discuss.elastic.co/u/hello4usharath)\ **Replies:** 0\ **Last updated:** [June 15, 2017, 7:55am UTC](https://discuss.elastic.co/t/data-loss-logstash-to-webhdfs/89521 "2017-06-15T07:55:04Z")
We getting particular issue where data are getting missed during particular time stamp while transferring through logstash by webhdfs interface to hadoop. --- ## [Filebeat not sending logs to logstash (Versions 5.4.1)](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-logstash-versions-5-4-1/89366)
**Author:** [@James88](https://discuss.elastic.co/u/James88)\ **Replies:** 8\ **Last updated:** [June 15, 2017, 7:37am UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-to-logstash-versions-5-4-1/89366 "2017-06-15T07:37:55Z")
I have installed ELK version 5.4.1. Everything went well but for some reason, filebeat cannot send logs to logstash. 2017-06-14T13:59:57+03:00 INFO No non-zero metrics in the last 30s 2017-06-14T14:00:27+03:00 INFO No … --- ## [Issue in parsing the data in logstash](https://discuss.elastic.co/t/issue-in-parsing-the-data-in-logstash/89411)
**Author:** [@viki](https://discuss.elastic.co/u/viki)\ **Replies:** 1\ **Last updated:** [June 15, 2017, 6:04am UTC](https://discuss.elastic.co/t/issue-in-parsing-the-data-in-logstash/89411 "2017-06-15T06:04:25Z")
Hi, Facing parsing error with spaces, Input : 2017-06-13 04:02:18,933 INFO SIRAHULOG Thread ID:221:AppSessionFilter:doPost:user session:Everything matched We are grep "INFO" (it has two spaces between "INFO" and … --- ## [Configuring x-pack for SSl communication betweeen elasticsearch and logstash](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985)
**Author:** [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Replies:** 15\ **Last updated:** [June 15, 2017, 6:04am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985 "2017-06-15T06:04:22Z")
Please help me in using x-pack. My Requirement: My setup is kibana and elasticsearch resides on one machine. Logstash and filebeat resides on another machine. My requirement is to secure the connection with SSL certif… --- ## [Logstash cannot assign correct date to log timestamp](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249)
**Author:** [@John\_06](https://discuss.elastic.co/u/John_06)\ **Replies:** 10\ **Last updated:** [June 15, 2017, 5:57am UTC](https://discuss.elastic.co/t/logstash-cannot-assign-correct-date-to-log-timestamp/89249 "2017-06-15T05:57:10Z")
Could anybody clarify if it's possible to add date part to log timestamp in Logstash not in 'UTC' but in servers time zone. For example, if I have log record timestamp like this (without date part): 21:17:43,124 INF… --- ## [Logstash filtering nested JSON fields (mutate / grok)](https://discuss.elastic.co/t/logstash-filtering-nested-json-fields-mutate-grok/89263)
**Author:** [@sarveshgeetha](https://discuss.elastic.co/u/sarveshgeetha)\ **Replies:** 3\ **Last updated:** [June 15, 2017, 5:47am UTC](https://discuss.elastic.co/t/logstash-filtering-nested-json-fields-mutate-grok/89263 "2017-06-15T05:47:12Z")
Hello Folks, input { http\_poller { urls =\> { test2 =\> { method =\> get url =\> "http://www.coincap.io/history/1day/XRP" } } request\_timeout =\> 60 schedule =\> { cron =\> "\* \* \* … --- ## [Filter decode from Base64](https://discuss.elastic.co/t/filter-decode-from-base64/89282)
**Author:** [@Xavier\_Clavien](https://discuss.elastic.co/u/Xavier_Clavien)\ **Replies:** 3\ **Last updated:** [June 15, 2017, 5:28am UTC](https://discuss.elastic.co/t/filter-decode-from-base64/89282 "2017-06-15T05:28:39Z")
Hi I'm running Docker image sebp/elk with : -logstash: 5.4.0 -elasticsearch: Version: 5.4.0, Build: 780f8c4/2017-04-28T17:43:27.229Z, JVM: 1.8.0\_121 -kibana: 5.4.0 -ruby: jruby 1.7.25 (1.9.3p551) 2016-04-13 867cb8… --- ## [How to convert timestamp in logstash](https://discuss.elastic.co/t/how-to-convert-timestamp-in-logstash/89350)
**Author:** [@viki](https://discuss.elastic.co/u/viki)\ **Replies:** 2\ **Last updated:** [June 15, 2017, 4:28am UTC](https://discuss.elastic.co/t/how-to-convert-timestamp-in-logstash/89350 "2017-06-15T04:28:23Z")
Hi, I'm new for logstash. I have a "time" field which has a value "2017-06-13 08:08:18,647" . I need convert "time" field value as "2017-06-13 08:08:18" Thanks in Advance --- ## [Logstash output Chinese garbled](https://discuss.elastic.co/t/logstash-output-chinese-garbled/88646)
**Author:** [@mrzhaoatbj](https://discuss.elastic.co/u/mrzhaoatbj)\ **Replies:** 1\ **Last updated:** [June 15, 2017, 4:28am UTC](https://discuss.elastic.co/t/logstash-output-chinese-garbled/88646 "2017-06-15T04:28:03Z")
hi all, I was troubled by a problem for a long time. I made a .mmdb file for fun with the perl writer maxmind provided, there are some chinese words in it, location { accuracy\_radius 20, latitu… --- ## [Logstash merge event,need help!](https://discuss.elastic.co/t/logstash-merge-event-need-help/88033)
**Author:** [@JIELOGAN](https://discuss.elastic.co/u/JIELOGAN)\ **Replies:** 2\ **Last updated:** [June 15, 2017, 4:25am UTC](https://discuss.elastic.co/t/logstash-merge-event-need-help/88033 "2017-06-15T04:25:27Z")
Hi ,I want to merge some events ,but i did not find any plugin could work.My data like this: H Comware Platform Software Comware Software, Version 5.20, Release 2202 Copyright (c) 2004-2009 Hhu. H S-C-E … --- ## [Parsing plain text with Logstash filter](https://discuss.elastic.co/t/parsing-plain-text-with-logstash-filter/89085)
**Author:** [@John\_06](https://discuss.elastic.co/u/John_06)\ **Replies:** 5\ **Last updated:** [June 15, 2017, 3:03am UTC](https://discuss.elastic.co/t/parsing-plain-text-with-logstash-filter/89085 "2017-06-15T03:03:30Z")
The log file is one line of plain text. There is part of text like this: ...there is some text here, Results: 0, Errors: 0, there is some text here... The goal is to create new fields results and errors and assign pr… --- ## [How to make 2 separaotrs?](https://discuss.elastic.co/t/how-to-make-2-separaotrs/89187)
**Author:** [@Yimjunhyeok](https://discuss.elastic.co/u/Yimjunhyeok)\ **Replies:** 10\ **Last updated:** [June 15, 2017, 1:33am UTC](https://discuss.elastic.co/t/how-to-make-2-separaotrs/89187 "2017-06-15T01:33:58Z")
Hello guys. This is a my first question. I want to make 2 separaotrs For example, this is a example log ex: a, b, c|| d, f, g|| h Using logstash, how to declare 2 separators on logstash filter? filter { csv { … --- ## [Logstash Configuration Error](https://discuss.elastic.co/t/logstash-configuration-error/89432)
**Author:** [@ViJaY\_S](https://discuss.elastic.co/u/ViJaY_S)\ **Replies:** 2\ **Last updated:** [June 15, 2017, 1:04am UTC](https://discuss.elastic.co/t/logstash-configuration-error/89432 "2017-06-15T01:04:29Z")
Hello, Could you please help to fix the logstash configuration error, I see the below error while running configuration test. Due to this issue importing new logs to ELK server is stopped, Sending logstash logs t… --- ## [Send logs using logback socket appender to logstach tcp server](https://discuss.elastic.co/t/send-logs-using-logback-socket-appender-to-logstach-tcp-server/89273)
**Author:** [@msq747](https://discuss.elastic.co/u/msq747)\ **Replies:** 2\ **Last updated:** [June 14, 2017, 10:46pm UTC](https://discuss.elastic.co/t/send-logs-using-logback-socket-appender-to-logstach-tcp-server/89273 "2017-06-14T22:46:34Z")
Hi all, I am trying to send java application logs using a socket appender to a tcp logstash server. The configuration of the server is as follows: input { tcp { port =\> 21000 mode =\> "server" } } output {… --- ## [Disable scroll in Logstash Elasticsearch Input](https://discuss.elastic.co/t/disable-scroll-in-logstash-elasticsearch-input/89468)
**Author:** [@Dave\_Waters](https://discuss.elastic.co/u/Dave_Waters)\ **Replies:** 0\ **Last updated:** [June 14, 2017, 10:38pm UTC](https://discuss.elastic.co/t/disable-scroll-in-logstash-elasticsearch-input/89468 "2017-06-14T22:38:11Z")
Is there a method to disable using the scroll\_id on the Logstash Input Plugin for Elasticsearch? It appears that because the query uses a scroll\_id, it causes data to be missed (always viewing a snap shot). I am using… --- ## [Logstash json file for twitter search](https://discuss.elastic.co/t/logstash-json-file-for-twitter-search/89466)
**Author:** [@Lbr1](https://discuss.elastic.co/u/Lbr1)\ **Replies:** 0\ **Last updated:** [June 14, 2017, 10:24pm UTC](https://discuss.elastic.co/t/logstash-json-file-for-twitter-search/89466 "2017-06-14T22:24:55Z")
Good evening community, I'm new to elastic and logstash and I'm doing some work on twitter's feelings analysis. I created this .json file: input { twitter { consumer\_key =\> ".." consumer\_secret =\> "..." oauth\_tok… --- ## [Error regarding log4j: well worn topic](https://discuss.elastic.co/t/error-regarding-log4j-well-worn-topic/87943)
**Author:** [@reswob](https://discuss.elastic.co/u/reswob)\ **Replies:** 1\ **Last updated:** [June 14, 2017, 9:31pm UTC](https://discuss.elastic.co/t/error-regarding-log4j-well-worn-topic/87943 "2017-06-14T21:31:33Z")
I'm brand new to Logstash, I'm trying to get an Filebeats - LogStash - Elasticsearch pipeline set up using the online documentation. But I'm running into a problem where I get the following error: \[root@elastic-01 logs… --- ## [Kv include keys doesn't work](https://discuss.elastic.co/t/kv-include-keys-doesnt-work/89436)
**Author:** [@davelncn](https://discuss.elastic.co/u/davelncn)\ **Replies:** 1\ **Last updated:** [June 14, 2017, 8:02pm UTC](https://discuss.elastic.co/t/kv-include-keys-doesnt-work/89436 "2017-06-14T20:02:30Z")
Hi, I am using Logstash 5.4.1. I don't why kv include\_keys doesn't work as expected. could you please provide some advice? if there is some error in my config or this is known issue? Many thanks! below is my config: g… --- ## [How to insert Data to Elasticsearch using Logstash?](https://discuss.elastic.co/t/how-to-insert-data-to-elasticsearch-using-logstash/89130)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 5\ **Last updated:** [June 14, 2017, 5:46pm UTC](https://discuss.elastic.co/t/how-to-insert-data-to-elasticsearch-using-logstash/89130 "2017-06-14T17:46:05Z")
Hi, I am trying to insert data to Elasticsearch using Logstash. I created an index 'twitter' and my config file is as follows, input{ file{ path=\>"C:/Users/Sample.log" codec=\>json } } output{ stdout{ codec=\>ru… --- ## [Current year in to the date filter](https://discuss.elastic.co/t/current-year-in-to-the-date-filter/89394)
**Author:** [@Raghu19](https://discuss.elastic.co/u/Raghu19)\ **Replies:** 2\ **Last updated:** [June 14, 2017, 3:22pm UTC](https://discuss.elastic.co/t/current-year-in-to-the-date-filter/89394 "2017-06-14T15:22:36Z")
Dear experts, I am generating a config file to parse a log file, which has date in the form of only MON dd. e.g., Jun 11, Aug 01 etc if i use the date filter, is there a way i can assign current year (2017) to this date? … --- ## [Upgrade to logstash 5, got mapper\_parsing\_error](https://discuss.elastic.co/t/upgrade-to-logstash-5-got-mapper-parsing-error/89216)
**Author:** [@davelncn](https://discuss.elastic.co/u/davelncn)\ **Replies:** 2\ **Last updated:** [June 14, 2017, 2:37pm UTC](https://discuss.elastic.co/t/upgrade-to-logstash-5-got-mapper-parsing-error/89216 "2017-06-14T14:37:55Z")
Hi, I upgraded logstash from v2.2 to 5.4 recently, got mapper\_parsing\_error, same grok pattern works on previous version, and verified on "http://grokconstructor.appspot.com/" as well, could someone help me out on this i… --- ## [\[SOLVED\] Logstash 5.4 Grok match partially working, can't figure why](https://discuss.elastic.co/t/solved-logstash-5-4-grok-match-partially-working-cant-figure-why/88667)
**Author:** [@Sebastien\_Lelak](https://discuss.elastic.co/u/Sebastien_Lelak)\ **Replies:** 2\ **Last updated:** [June 14, 2017, 2:26pm UTC](https://discuss.elastic.co/t/solved-logstash-5-4-grok-match-partially-working-cant-figure-why/88667 "2017-06-14T14:26:33Z")
Greetings, For my first thread to you, I would like to submit an issue to your expertise. I have been using ELK stack (5.4) to gather data from our local radio station stream. I set up a Logstash file input which look… --- ## [Help with converting timestamp](https://discuss.elastic.co/t/help-with-converting-timestamp/89215)
**Author:** [@pmohan](https://discuss.elastic.co/u/pmohan)\ **Replies:** 2\ **Last updated:** [June 14, 2017, 2:25pm UTC](https://discuss.elastic.co/t/help-with-converting-timestamp/89215 "2017-06-14T14:25:22Z")
below is my log line , I use the json filter and that works perfectly fine. I just need to get the log time stamp sorted in a readable format . timestamp:"unix timestamp, in nanoseconds, when the log message was created… --- ## [Error while parsing logs with XML data](https://discuss.elastic.co/t/error-while-parsing-logs-with-xml-data/89378)
**Author:** [@Kiranmai\_Reddy](https://discuss.elastic.co/u/Kiranmai_Reddy)\ **Replies:** 0\ **Last updated:** [June 14, 2017, 1:00pm UTC](https://discuss.elastic.co/t/error-while-parsing-logs-with-xml-data/89378 "2017-06-14T13:00:20Z")
I am trying to parse logs with XML data using XML filter but it is giving me syntax error. here is my log line: this is configuration of XML filter: xml { source =\> "siebelfexception" store\_xml =\> false xpath … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=963) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=965)