# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=795 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 796 --- ## [How to connect mongo with elasticsearch using logstash?](https://discuss.elastic.co/t/how-to-connect-mongo-with-elasticsearch-using-logstash/125352)
**Author:** [@EditAxha](https://discuss.elastic.co/u/EditAxha)\ **Replies:** 0\ **Last updated:** [March 23, 2018, 10:47am UTC](https://discuss.elastic.co/t/how-to-connect-mongo-with-elasticsearch-using-logstash/125352 "2018-03-23T10:47:46Z")
I want to connect mongodb with elasticsearch. I have used logstash-input-mongodb plugin, but this doesn't take my elasticsearch data updates. So when i update a document at mongodb it doesn't change at elasticsearch. I … --- ## [How can I send logstash logs to windows event?](https://discuss.elastic.co/t/how-can-i-send-logstash-logs-to-windows-event/125351)
**Author:** [@Yusuf\_Karatoprak](https://discuss.elastic.co/u/Yusuf_Karatoprak)\ **Replies:** 0\ **Last updated:** [March 23, 2018, 10:44am UTC](https://discuss.elastic.co/t/how-can-i-send-logstash-logs-to-windows-event/125351 "2018-03-23T10:44:15Z")
I am trying to add logstash logs to windows logs? I got an error, @jdno helped me to getrid of error. Other question : On the other hand; I want to write logs to windows event logs by using win32/eventlog.(https://g… --- ## [5.6.8 java.lang.OutOfMemoryError after 6-10 days](https://discuss.elastic.co/t/5-6-8-java-lang-outofmemoryerror-after-6-10-days/124265)
**Author:** [@stefws](https://discuss.elastic.co/u/stefws)\ **Replies:** 7\ **Last updated:** [March 23, 2018, 10:37am UTC](https://discuss.elastic.co/t/5-6-8-java-lang-outofmemoryerror-after-6-10-days/124265 "2018-03-23T10:37:48Z")
Since we patched from 5.6.5 to 5.6.8 we're seeing logstash memory leakage so it'll now run out of memory after 6-10 days, nothing changed in our filters etc. So assuming there's a leak somewhere maybe. Before memory usag… --- ## [Problem with codec multiline input](https://discuss.elastic.co/t/problem-with-codec-multiline-input/125337)
**Author:** [@flochon](https://discuss.elastic.co/u/flochon)\ **Replies:** 3\ **Last updated:** [March 23, 2018, 10:08am UTC](https://discuss.elastic.co/t/problem-with-codec-multiline-input/125337 "2018-03-23T10:08:00Z")
I want parse my log multiline with the input of Logstash like this : input { beats { port =\> 5443 if \[message\] =~ ".\*---.\*" { codec =\> multiline { multiline\_tag =\> "MULTILINE" pattern =\> … --- ## [Inconsistent performance with jdbc query](https://discuss.elastic.co/t/inconsistent-performance-with-jdbc-query/125335)
**Author:** [@Mariusz\_Gorski](https://discuss.elastic.co/u/Mariusz_Gorski)\ **Replies:** 0\ **Last updated:** [March 23, 2018, 9:25am UTC](https://discuss.elastic.co/t/inconsistent-performance-with-jdbc-query/125335 "2018-03-23T09:25:09Z")
Hello, I have stumbled upon weird performance inconsistency with logstash jdbc plugin. I have defined sql query which is supposed to pull some columns from database and inject them into elasticsearch. If i define query … --- ## [Regex in S3 input plugin](https://discuss.elastic.co/t/regex-in-s3-input-plugin/124486)
**Author:** [@Eswar\_Kumar\_Musiboin](https://discuss.elastic.co/u/Eswar_Kumar_Musiboin)\ **Replies:** 1\ **Last updated:** [March 23, 2018, 9:09am UTC](https://discuss.elastic.co/t/regex-in-s3-input-plugin/124486 "2018-03-23T09:09:35Z")
I neeed to get prefix in S3 input by using Regex.. But there is no support for Regex in the doc.. please find the solution ........ Folders list like this: logs/20180304/app1/app1.log logs/23180304/app2/app2.log log… --- ## [Logstash does not read Files](https://discuss.elastic.co/t/logstash-does-not-read-files/125141)
**Author:** [@YannickWeber](https://discuss.elastic.co/u/YannickWeber)\ **Replies:** 3\ **Last updated:** [March 23, 2018, 8:45am UTC](https://discuss.elastic.co/t/logstash-does-not-read-files/125141 "2018-03-23T08:45:08Z")
Hello, I am new to Logstash and I am trying to read my logfiles. While I can parse single logs using stdin logstash does not seem to scan the configured files. my .config file looks like this: input { file { path =\> … --- ## [Ip2location i/o timeout](https://discuss.elastic.co/t/ip2location-i-o-timeout/125203)
**Author:** [@vitich](https://discuss.elastic.co/u/vitich)\ **Replies:** 0\ **Last updated:** [March 22, 2018, 1:45pm UTC](https://discuss.elastic.co/t/ip2location-i-o-timeout/125203 "2018-03-22T13:45:27Z")
Hi! Usually I use standard geoip, but have decided to test ip2location for comparison. When I add ip2location, my ELK server doesn't receive the log from heavy load nginx server. Less loaded servers continue to transf… --- ## [How to take backup Logstash service?](https://discuss.elastic.co/t/how-to-take-backup-logstash-service/125320)
**Author:** [@SathishKannan](https://discuss.elastic.co/u/SathishKannan)\ **Replies:** 2\ **Last updated:** [March 23, 2018, 8:20am UTC](https://discuss.elastic.co/t/how-to-take-backup-logstash-service/125320 "2018-03-23T08:20:14Z")
how to take backup of config /etc/logstash/conf.d/.. ? --- ## [Mutate in CSV filter not working with stdin](https://discuss.elastic.co/t/mutate-in-csv-filter-not-working-with-stdin/124987)
**Author:** [@kevkos236](https://discuss.elastic.co/u/kevkos236)\ **Replies:** 2\ **Last updated:** [March 23, 2018, 8:01am UTC](https://discuss.elastic.co/t/mutate-in-csv-filter-not-working-with-stdin/124987 "2018-03-23T08:01:16Z")
Hello there, we are using logstash to import data from csv files into elasticsearch. The logstash config worked so far but I was now trying to input the CSV files with stdin instead of file. The stdin input works but I'… --- ## [Not able to visualise netflow data in kibana](https://discuss.elastic.co/t/not-able-to-visualise-netflow-data-in-kibana/125241)
**Author:** [@csudaya](https://discuss.elastic.co/u/csudaya)\ **Replies:** 4\ **Last updated:** [March 23, 2018, 7:38am UTC](https://discuss.elastic.co/t/not-able-to-visualise-netflow-data-in-kibana/125241 "2018-03-23T07:38:39Z")
Hi - I have installed ELK stack and started the services. Using a nflow generator , generating netflow version 5 packets. I can see the packaets reaching the server and dictionary file getting refreshed in logstash . H… --- ## [Does codec netflow support L7 protocol name?](https://discuss.elastic.co/t/does-codec-netflow-support-l7-protocol-name/125303)
**Author:** [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Replies:** 1\ **Last updated:** [March 23, 2018, 7:22am UTC](https://discuss.elastic.co/t/does-codec-netflow-support-l7-protocol-name/125303 "2018-03-23T07:22:48Z")
Hi everyone, I export netflow to logstash and use the netflow codec plugin to analyze the flows, but I found that the flow data doesn't include L7 protocol name field. this is the json doc: { "\_index": "netflow-%{ye… --- ## [Grok pattern/expression for "YYYY/MM/dd HH:mm:ss"](https://discuss.elastic.co/t/grok-pattern-expression-for-yyyy-mm-dd-hh-mm-ss/122821)
**Author:** [@ruranga](https://discuss.elastic.co/u/ruranga)\ **Replies:** 2\ **Last updated:** [March 23, 2018, 4:53am UTC](https://discuss.elastic.co/t/grok-pattern-expression-for-yyyy-mm-dd-hh-mm-ss/122821 "2018-03-23T04:53:23Z")
Hi, Issue regarding grok patter. I need to create a grok pattern for following log. But I'm having problems with 2018/03/06 10:21:49 (YYYY/MM/dd HH:mm:ss) date format. Can anyone please help Sample log: INFO | jvm 1… --- ## [Http\_poller Host name '192.168.1.35' does not match the certificate subject provided by the peer](https://discuss.elastic.co/t/http-poller-host-name-192-168-1-35-does-not-match-the-certificate-subject-provided-by-the-peer/125111)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 5\ **Last updated:** [March 23, 2018, 12:55am UTC](https://discuss.elastic.co/t/http-poller-host-name-192-168-1-35-does-not-match-the-certificate-subject-provided-by-the-peer/125111 "2018-03-23T00:55:32Z")
Hi, I'm struggling to get the http\_poller working with a piece of equipment running a self signed cert. I've created the cert with the following command openssl s\_client -showcerts -connect 192.168.1.35:443 \ **Author:** [@dao](https://discuss.elastic.co/u/dao)\ **Replies:** 2\ **Last updated:** [March 22, 2018, 11:48pm UTC](https://discuss.elastic.co/t/6-2-2-cannot-connect-elastic-search-secured-cluster/125198 "2018-03-22T23:48:14Z") Hello I have been stucked for a while on this. I have an elasticsearch cluster with x-pack enabled. I have added encryption using this tutorial: https://www.elastic.co/guide/en/elasticsearch/reference/6.2/configuring-tl… --- ## [Add field or tag on failure](https://discuss.elastic.co/t/add-field-or-tag-on-failure/124647)
**Author:** [@pfseth](https://discuss.elastic.co/u/pfseth)\ **Replies:** 2\ **Last updated:** [March 22, 2018, 10:26pm UTC](https://discuss.elastic.co/t/add-field-or-tag-on-failure/124647 "2018-03-22T22:26:25Z")
The logstash documentation on "common options" says, one can add field or tags on success Is there anyway i can add field or tag on failure ? --- ## [Extract fields from syslog "message" part for visualization in Kibana](https://discuss.elastic.co/t/extract-fields-from-syslog-message-part-for-visualization-in-kibana/125209)
**Author:** [@telefonen](https://discuss.elastic.co/u/telefonen)\ **Replies:** 3\ **Last updated:** [March 22, 2018, 10:21pm UTC](https://discuss.elastic.co/t/extract-fields-from-syslog-message-part-for-visualization-in-kibana/125209 "2018-03-22T22:21:15Z")
I am a total beginner when it comes to LogStash and it's filtering mechanisms. I have tried google but I don't seem to get the answer to my question there (or I cannot make sense of the answer by myself). I am using a sy… --- ## [Kafka input plugin does not set @metadata fields](https://discuss.elastic.co/t/kafka-input-plugin-does-not-set-metadata-fields/125256)
**Author:** [@jeffkirk1](https://discuss.elastic.co/u/jeffkirk1)\ **Replies:** 3\ **Last updated:** [March 22, 2018, 9:16pm UTC](https://discuss.elastic.co/t/kafka-input-plugin-does-not-set-metadata-fields/125256 "2018-03-22T21:16:12Z")
The kafka input plugin does not appear to set the @metadata fields as described in this Logstash documentation. I'm using Elasticsearch 6.2.2 and Logstash 6.2.2 on Docker images, running on a CentOS 7.4 server. I wante… --- ## [Unable to detect new logstash instances](https://discuss.elastic.co/t/unable-to-detect-new-logstash-instances/125085)
**Author:** [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)\ **Replies:** 12\ **Last updated:** [March 22, 2018, 8:29pm UTC](https://discuss.elastic.co/t/unable-to-detect-new-logstash-instances/125085 "2018-03-22T20:29:17Z")
Hi , i have ELK stack running and X-pack monitoring is able to detect first 3 logstash node and able to monitor . But i have added 3 more logstash nodes are not able to detect by x-pack monitoring . \[2018-03-21T23:33:3… --- ## [Ruby/Mutate: converting to double](https://discuss.elastic.co/t/ruby-mutate-converting-to-double/125250)
**Author:** [@chuchurocket27](https://discuss.elastic.co/u/chuchurocket27)\ **Replies:** 3\ **Last updated:** [March 22, 2018, 8:24pm UTC](https://discuss.elastic.co/t/ruby-mutate-converting-to-double/125250 "2018-03-22T20:24:28Z")
Based on the documentation for the mutate filter it looks like I cannot convert a value to a double, only a float. Is there a way with a Ruby filter to convert to a double instead? --- ## [Why forwarding logs by Logstash to Elasticsearch is not working](https://discuss.elastic.co/t/why-forwarding-logs-by-logstash-to-elasticsearch-is-not-working/125058)
**Author:** [@apetrovYa](https://discuss.elastic.co/u/apetrovYa)\ **Replies:** 3\ **Last updated:** [March 22, 2018, 8:06pm UTC](https://discuss.elastic.co/t/why-forwarding-logs-by-logstash-to-elasticsearch-is-not-working/125058 "2018-03-22T20:06:49Z")
Hi all, I am playing with Elastic stack through Docker containers (Filebeat -\> Logstash Reciever -\> Elasticsearch \<- Kibana). Everything works pretty fine, but I have one problem. Assuming the following logstash.conf f… --- ## [Can I get the current path of file being processed by Logstash file input when searching through subdirectories?](https://discuss.elastic.co/t/can-i-get-the-current-path-of-file-being-processed-by-logstash-file-input-when-searching-through-subdirectories/125046)
**Author:** [@daved](https://discuss.elastic.co/u/daved)\ **Replies:** 2\ **Last updated:** [March 22, 2018, 4:42pm UTC](https://discuss.elastic.co/t/can-i-get-the-current-path-of-file-being-processed-by-logstash-file-input-when-searching-through-subdirectories/125046 "2018-03-22T16:42:44Z")
Hey, I'm wondering if there is anyway to tell what subdirectory/path to file is currently getting processed by Logstash when using the file input to look through a directory tree? From what I understand I could parse m… --- ## [@timestamp one day earlier than Event timestamp](https://discuss.elastic.co/t/timestamp-one-day-earlier-than-event-timestamp/125041)
**Author:** [@bo9354](https://discuss.elastic.co/u/bo9354)\ **Replies:** 3\ **Last updated:** [March 22, 2018, 4:31pm UTC](https://discuss.elastic.co/t/timestamp-one-day-earlier-than-event-timestamp/125041 "2018-03-22T16:31:12Z")
strong text @timestamp one day earlier (20 hours) than Event timestamp Kibana is displaying the Document in CDT, the Event received is in EDT and @timestamp is in UTC. When Logstash collects the message below, the tim… --- ## [Page file size 0 different to configured page](https://discuss.elastic.co/t/page-file-size-0-different-to-configured-page/125226)
**Author:** [@rrs](https://discuss.elastic.co/u/rrs)\ **Replies:** 0\ **Last updated:** [March 22, 2018, 3:55pm UTC](https://discuss.elastic.co/t/page-file-size-0-different-to-configured-page/125226 "2018-03-22T15:55:10Z")
Hi, Our Logstash-5.6.3 suddenly stopped working and giving below error. Can somebody help me to fix it. \[ERROR\]\[logstash.agent \] Cannot create pipeline {:reason=\>"java.io.IOException: Page file size 0 differe… --- ## [Preserve original message from Beats for later usage](https://discuss.elastic.co/t/preserve-original-message-from-beats-for-later-usage/125214)
**Author:** [@admlko](https://discuss.elastic.co/u/admlko)\ **Replies:** 0\ **Last updated:** [March 22, 2018, 2:58pm UTC](https://discuss.elastic.co/t/preserve-original-message-from-beats-for-later-usage/125214 "2018-03-22T14:58:24Z")
Hi! I have set up Logstash to preserve original message in "message" field for archiving purposes. Every night, I run a pipeline, which reads everything from yesterday and outputs it to a file on disk. The idea is to … --- ## ["\_dateparsefailure" Logstash 6.2.1](https://discuss.elastic.co/t/dateparsefailure-logstash-6-2-1/125196)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 1\ **Last updated:** [March 22, 2018, 1:40pm UTC](https://discuss.elastic.co/t/dateparsefailure-logstash-6-2-1/125196 "2018-03-22T13:40:51Z")
Even though the datetime field is parsed properly by logstash pipeline, every document in elasticsearch has "tags": \[ "\_dateparsefailure" \], So wondering why the tags field is getting populated with \_dateparsefailure … --- ## [My conf file has a es ip address that can't be compiled correctly](https://discuss.elastic.co/t/my-conf-file-has-a-es-ip-address-that-cant-be-compiled-correctly/125199)
**Author:** [@Phoebe](https://discuss.elastic.co/u/Phoebe)\ **Replies:** 0\ **Last updated:** [March 22, 2018, 1:36pm UTC](https://discuss.elastic.co/t/my-conf-file-has-a-es-ip-address-that-cant-be-compiled-correctly/125199 "2018-03-22T13:36:10Z")
my file's address input { log4j { host =\> "192.168.22.75" port =\> 4560 } } output { stdout { codec =\> rubydebug } elasticsearch{ hosts =\> \[ "192.168.22.95:9200"\] index =\> "log4j-%{+YYYY.MM.dd}" } } and … --- ## [Parsing nested XML into logstash](https://discuss.elastic.co/t/parsing-nested-xml-into-logstash/125131)
**Author:** [@porkiz](https://discuss.elastic.co/u/porkiz)\ **Replies:** 2\ **Last updated:** [March 22, 2018, 1:18pm UTC](https://discuss.elastic.co/t/parsing-nested-xml-into-logstash/125131 "2018-03-22T13:18:45Z")
\ \ \test 1 \test 2 \ \ \test 3 \test 4 \ \ \ \ \Info1 \ **Author:** [@dao](https://discuss.elastic.co/u/dao)\ **Replies:** 4\ **Last updated:** [March 22, 2018, 12:57pm UTC](https://discuss.elastic.co/t/systemd-logstash-service/125083 "2018-03-22T12:57:48Z") Hello, I am surprised that the deb package of logstash does not come with a systemd service. Elasticsearch and kibana does. Did I miss something? if it is not in the package, can you provide an official way to start, r… --- ## [Why is it merging my indexes?](https://discuss.elastic.co/t/why-is-it-merging-my-indexes/125048)
**Author:** [@hgpit](https://discuss.elastic.co/u/hgpit)\ **Replies:** 4\ **Last updated:** [March 22, 2018, 12:31pm UTC](https://discuss.elastic.co/t/why-is-it-merging-my-indexes/125048 "2018-03-22T12:31:29Z")
Hello, I am experimenting with Logstash 6. I have two \*.conf files. One is sending results to index "sshd\_fail-%{+YYYY.MM}", the second is sending results to index "idx\_md-descriptions". I have installed Kibana, and I… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=794) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=796)