# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=524 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 525 --- ## [Log stash - fetching data from file - not getting the output in the console](https://discuss.elastic.co/t/log-stash-fetching-data-from-file-not-getting-the-output-in-the-console/183455)
**Author:** [@ashokchinna](https://discuss.elastic.co/u/ashokchinna)\ **Replies:** 16\ **Last updated:** [June 19, 2019, 10:29pm UTC](https://discuss.elastic.co/t/log-stash-fetching-data-from-file-not-getting-the-output-in-the-console/183455 "2019-06-19T22:29:03Z")
I am trying to use the file as input source. But I am not getting the file content in the stdout. Command ./logstash -f ../conf/file.conf Configuration File #sample configuration 1 #file.conf input { file{ path… --- ## [Message arrival](https://discuss.elastic.co/t/message-arrival/184916)
**Author:** [@Aviad\_Bitton](https://discuss.elastic.co/u/Aviad_Bitton)\ **Replies:** 3\ **Last updated:** [June 20, 2019, 9:21pm UTC](https://discuss.elastic.co/t/message-arrival/184916 "2019-06-20T21:21:45Z")
Hi, In my logstash configuration i got http as input, some filter, ES as output. Logstash will return status code 200 even when ES won't get it eventually. How can i know for sure that when i send a log, that log will… --- ## [Ruby Filter to Add Multiple Values to a Field](https://discuss.elastic.co/t/ruby-filter-to-add-multiple-values-to-a-field/186746)
**Author:** [@cappy](https://discuss.elastic.co/u/cappy)\ **Replies:** 2\ **Last updated:** [June 20, 2019, 9:13pm UTC](https://discuss.elastic.co/t/ruby-filter-to-add-multiple-values-to-a-field/186746 "2019-06-20T21:13:33Z")
Hello, Due to the fact that a field is getting interpreted as a single string and attempted to be indexed into Elasticsearch as type ip (and failing due to mapping, rightfully so), I am looking to take values separated … --- ## [Logstash Beats Pod stopped servicing our requests](https://discuss.elastic.co/t/logstash-beats-pod-stopped-servicing-our-requests/186762)
**Author:** [@Jacko-ski](https://discuss.elastic.co/u/Jacko-ski)\ **Replies:** 0\ **Last updated:** [June 20, 2019, 8:31pm UTC](https://discuss.elastic.co/t/logstash-beats-pod-stopped-servicing-our-requests/186762 "2019-06-20T20:31:19Z")
Logstash 6.6.1 running in Kubernetes 1.12, with only once ReplicaSet, which receives both Metricbeat and Filebeat data. This morning, we noticed a continuous gap in the incoming Metricbeat data. Looking at the Logsta… --- ## [Http\_poller input plugin Logstash solution](https://discuss.elastic.co/t/http-poller-input-plugin-logstash-solution/186581)
**Author:** [@EZprogramming](https://discuss.elastic.co/u/EZprogramming)\ **Replies:** 1\ **Last updated:** [June 20, 2019, 6:57pm UTC](https://discuss.elastic.co/t/http-poller-input-plugin-logstash-solution/186581 "2019-06-20T18:57:54Z")
Background info: The objective is to transfer release information of a hardware machine from Balena to Kibana using the following logic: Balena JSON data \> Logstash \> Elasticsearch \> Kibana I am stuck in the Logstash … --- ## [Problems with Drop Filter in Logstash](https://discuss.elastic.co/t/problems-with-drop-filter-in-logstash/186394)
**Author:** [@Ghostrider](https://discuss.elastic.co/u/Ghostrider)\ **Replies:** 1\ **Last updated:** [June 20, 2019, 8:22pm UTC](https://discuss.elastic.co/t/problems-with-drop-filter-in-logstash/186394 "2019-06-20T20:22:02Z")
Hey guys, I am a total amateur with logstash and I try to build something small, I think. I am currently using winlogbeat to get windows events, but I want to customize the filter a bit. The goal of my filter is that I… --- ## [Logstash 7.1.1 - codec multiline filter loses last line](https://discuss.elastic.co/t/logstash-7-1-1-codec-multiline-filter-loses-last-line/186754)
**Author:** [@Alexl](https://discuss.elastic.co/u/Alexl)\ **Replies:** 0\ **Last updated:** [June 20, 2019, 8:00pm UTC](https://discuss.elastic.co/t/logstash-7-1-1-codec-multiline-filter-loses-last-line/186754 "2019-06-20T20:00:14Z")
Hello all! I'm still trying to use so necessary examples from the internet pages but it looks like there is no fix of the bug in the community. Someone recommends use of auto\_flush\_interval =\> 1 which seems not working.… --- ## [Logstash 7.1.1 - How could I use http filter to get and post data via https?](https://discuss.elastic.co/t/logstash-7-1-1-how-could-i-use-http-filter-to-get-and-post-data-via-https/186753)
**Author:** [@Alexl](https://discuss.elastic.co/u/Alexl)\ **Replies:** 0\ **Last updated:** [June 20, 2019, 7:52pm UTC](https://discuss.elastic.co/t/logstash-7-1-1-how-could-i-use-http-filter-to-get-and-post-data-via-https/186753 "2019-06-20T19:52:14Z")
Hello all! I want this filter but still not found it's usage example. How to use it to read data and write data over the internet? Thank you. --- ## [Unable to capture the syslog events from port 514](https://discuss.elastic.co/t/unable-to-capture-the-syslog-events-from-port-514/186750)
**Author:** [@Narayanan\_Sukumar](https://discuss.elastic.co/u/Narayanan_Sukumar)\ **Replies:** 0\ **Last updated:** [June 20, 2019, 7:32pm UTC](https://discuss.elastic.co/t/unable-to-capture-the-syslog-events-from-port-514/186750 "2019-06-20T19:32:58Z")
I have entire devices logs that are pushing to my server on port 514 and I can able to see that in tcpdump as below: \[root@localhost config\]# tcpdump -i eno1 port 514 -vvvv 2:20:03.035727 IP (tos 0x0, ttl 28, id 22525,… --- ## [LogStash, GROK and a Versa Appliance Log -](https://discuss.elastic.co/t/logstash-grok-and-a-versa-appliance-log/186552)
**Author:** [@Zorkmid](https://discuss.elastic.co/u/Zorkmid)\ **Replies:** 2\ **Last updated:** [June 20, 2019, 5:03pm UTC](https://discuss.elastic.co/t/logstash-grok-and-a-versa-appliance-log/186552 "2019-06-20T17:03:28Z")
Hello All, I have the need to ingesting event logs from VersaNetworks appliances. In building the LogStash filters I've run into bit of a conundrum. I have so far the following built and have been checking it with the… --- ## [Date\_time parsing error in Elastic Stack 7.0. Worked fine in 6.7](https://discuss.elastic.co/t/date-time-parsing-error-in-elastic-stack-7-0-worked-fine-in-6-7/185658)
**Author:** [@Paul\_14](https://discuss.elastic.co/u/Paul_14)\ **Replies:** 7\ **Last updated:** [June 20, 2019, 3:02pm UTC](https://discuss.elastic.co/t/date-time-parsing-error-in-elastic-stack-7-0-worked-fine-in-6-7/185658 "2019-06-20T15:02:31Z")
Hello, Bit of a strange problem. I was using Elastic Stack 6.7 and have recently created a new 7.0 cluster. From Filebeat I feed the below log line into both of these versions of Logstash Log line: \[INFO \] \[2019-06-… --- ## [Delete document from index starting with fuzzy match of document id](https://discuss.elastic.co/t/delete-document-from-index-starting-with-fuzzy-match-of-document-id/186714)
**Author:** [@avinash9999](https://discuss.elastic.co/u/avinash9999)\ **Replies:** 0\ **Last updated:** [June 20, 2019, 2:42pm UTC](https://discuss.elastic.co/t/delete-document-from-index-starting-with-fuzzy-match-of-document-id/186714 "2019-06-20T14:42:19Z")
I am trying to delete documents whose document\_id starts with sid output { elasticsearch { hosts =\> " xyz" user =\> "xyz " password =\> "xyz " index =\> " xyz" document\_id =\> "%{sid}\*" action =\> 'delete' } } s… --- ## [Base64 decode](https://discuss.elastic.co/t/base64-decode/186564)
**Author:** [@bkasrai](https://discuss.elastic.co/u/bkasrai)\ **Replies:** 3\ **Last updated:** [June 20, 2019, 1:47pm UTC](https://discuss.elastic.co/t/base64-decode/186564 "2019-06-20T13:47:55Z")
Hello all, I am working on something I have never worked on before and I really do not know where to go from here and I am hoping someone might have some direction for me to attempt trying to get this parsed to Elastics… --- ## [Logstash ConfigurationError, trying to migrate data from mysql to ELK using JDBC driver](https://discuss.elastic.co/t/logstash-configurationerror-trying-to-migrate-data-from-mysql-to-elk-using-jdbc-driver/186587)
**Author:** [@ShashankAC](https://discuss.elastic.co/u/ShashankAC)\ **Replies:** 3\ **Last updated:** [June 20, 2019, 1:33pm UTC](https://discuss.elastic.co/t/logstash-configurationerror-trying-to-migrate-data-from-mysql-to-elk-using-jdbc-driver/186587 "2019-06-20T13:33:23Z")
Hi there, I am new ELK and having some trouble configuring Logstash. I tried to follow this tutorial the config file is as follows. input { jdbc { jdbc\_connection\_string =\> "jdbc:mysql://localhost/telecom1" # Th… --- ## [Logstash date filter parsing year that is greater than 4 digits](https://discuss.elastic.co/t/logstash-date-filter-parsing-year-that-is-greater-than-4-digits/186683)
**Author:** [@rsmoorthy](https://discuss.elastic.co/u/rsmoorthy)\ **Replies:** 0\ **Last updated:** [June 20, 2019, 12:29pm UTC](https://discuss.elastic.co/t/logstash-date-filter-parsing-year-that-is-greater-than-4-digits/186683 "2019-06-20T12:29:51Z")
With a logstash filter like this: date { match =\> \[ "logdate", "YYYY-MM-dd HH:mm:ss" \] } and the value of logdate being 12019-06-17 15:09:16 I would expect the date filter to fail the match, as there are 5 digits. B… --- ## [Import JSON String data as a Flat table](https://discuss.elastic.co/t/import-json-string-data-as-a-flat-table/186557)
**Author:** [@sxmalla](https://discuss.elastic.co/u/sxmalla)\ **Replies:** 4\ **Last updated:** [June 20, 2019, 12:18pm UTC](https://discuss.elastic.co/t/import-json-string-data-as-a-flat-table/186557 "2019-06-20T12:18:24Z")
Hi I imported Data from SQL server into elastic using Logstash jdbc plugin. The data contains a Varchar(4000) element which contains Json string in it. Is there any way that I can take that data in JSON format and parse… --- ## [Could not index event to Elasticsearch after upgrade to 7.0.1](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-after-upgrade-to-7-0-1/186641)
**Author:** [@strom18](https://discuss.elastic.co/u/strom18)\ **Replies:** 1\ **Last updated:** [June 20, 2019, 12:00pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-after-upgrade-to-7-0-1/186641 "2019-06-20T12:00:09Z")
Hello, I have this following error since upgrade from 6.7.1 to 7.0.1 : \[2019-06-20T11:21:04,154\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>n… --- ## [How to configure logging for helm? logging output different than manually running logstash](https://discuss.elastic.co/t/how-to-configure-logging-for-helm-logging-output-different-than-manually-running-logstash/186667)
**Author:** [@procyclinsur](https://discuss.elastic.co/u/procyclinsur)\ **Replies:** 0\ **Last updated:** [June 20, 2019, 11:20am UTC](https://discuss.elastic.co/t/how-to-configure-logging-for-helm-logging-output-different-than-manually-running-logstash/186667 "2019-06-20T11:20:50Z")
I have been trying to deploying logstash via helm with my setup described more HERE for two weeks now and I cannot get it to work, it keeps crashing.... I need to configure logging in such a way that I know what is going… --- ## [Logstash Date parsing (with AM/PM data) for YYYY-MM-dd HH:mm:ss,SSS format](https://discuss.elastic.co/t/logstash-date-parsing-with-am-pm-data-for-yyyy-mm-dd-hhss-sss-format/186094)
**Author:** [@abinashkd](https://discuss.elastic.co/u/abinashkd)\ **Replies:** 3\ **Last updated:** [June 20, 2019, 11:09am UTC](https://discuss.elastic.co/t/logstash-date-parsing-with-am-pm-data-for-yyyy-mm-dd-hhss-sss-format/186094 "2019-06-20T11:09:43Z")
Hi, I am new to logstash. With some R&D I have written a grok, but don't weather it validates or not. Input: ####\ Output: I need the date in the format of 2019-06-17T13:33:20.000Z in ES… --- ## [Dissector mapping, pattern not found](https://discuss.elastic.co/t/dissector-mapping-pattern-not-found/186509)
**Author:** [@wasder](https://discuss.elastic.co/u/wasder)\ **Replies:** 2\ **Last updated:** [June 20, 2019, 10:57am UTC](https://discuss.elastic.co/t/dissector-mapping-pattern-not-found/186509 "2019-06-20T10:57:01Z")
Can't understand why my pattern doesn't work. None of events are parsed with such error: \[2019-06-19T18:01:29,601\]\[WARN \]\[org.logstash.dissect.Dissector\] Dissector mapping, pattern not found { "field"=\>"message", "pat… --- ## [Regex \\d or \\w not working any longer in 7.1.1, but in 6.6.2: unmatched range specifier in char-class:](https://discuss.elastic.co/t/regex-d-or-w-not-working-any-longer-in-7-1-1-but-in-6-6-2-unmatched-range-specifier-in-char-class/186223)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 2\ **Last updated:** [June 20, 2019, 7:37am UTC](https://discuss.elastic.co/t/regex-d-or-w-not-working-any-longer-in-7-1-1-but-in-6-6-2-unmatched-range-specifier-in-char-class/186223 "2019-06-20T07:37:30Z")
Hi, I am getting following filter in LS 7.1.1: \[2019-06-18T09:31:39,304\]\[ERROR\]\[logstash.javapipeline \] Pipeline aborted due to error {:pipeline\_id=\>"auskunft\_json", :exception=\>#\ **Author:** [@Chang-Suk\_Kang](https://discuss.elastic.co/u/Chang-Suk_Kang)\ **Replies:** 3\ **Last updated:** [June 20, 2019, 7:08am UTC](https://discuss.elastic.co/t/how-to-drop-mutiple-ips-efficiently-in-using-grok-filter/186107 "2019-06-20T07:08:34Z") I'd like to drop multiple ips in messaging. I wrote config below. . Sample : I waana drop message including ip (10.10.10.31 ~ 10.10.10.49) =========================================================================== i… --- ## [Error messages from elastic search with \\r\\n instead of actual new lines](https://discuss.elastic.co/t/error-messages-from-elastic-search-with-r-n-instead-of-actual-new-lines/186603)
**Author:** [@gautiery](https://discuss.elastic.co/u/gautiery)\ **Replies:** 0\ **Last updated:** [June 20, 2019, 6:49am UTC](https://discuss.elastic.co/t/error-messages-from-elastic-search-with-r-n-instead-of-actual-new-lines/186603 "2019-06-20T06:49:13Z")
Hi I am using logstash to update a document in an elasticsearch index. When there is a problem (here within a painless script in my logstash conf file) I get errors similar to the one below. The loggstash logger does no… --- ## [Unable to receive events in lovebeat when sent from statsd](https://discuss.elastic.co/t/unable-to-receive-events-in-lovebeat-when-sent-from-statsd/186550)
**Author:** [@H\_C1](https://discuss.elastic.co/u/H_C1)\ **Replies:** 1\ **Last updated:** [June 20, 2019, 6:02am UTC](https://discuss.elastic.co/t/unable-to-receive-events-in-lovebeat-when-sent-from-statsd/186550 "2019-06-20T06:02:32Z")
HI I'm unable to receive events in lovebeat when sent from statsd while I can receive them in elasticsearch. The logstash output configuration is as follows : output { elasticsearch { hosts =\> \["localhost:9200"\] i… --- ## [Setting log time to @timestamp in logstash](https://discuss.elastic.co/t/setting-log-time-to-timestamp-in-logstash/186374)
**Author:** [@Vinutha\_A\_H](https://discuss.elastic.co/u/Vinutha_A_H)\ **Replies:** 6\ **Last updated:** [June 20, 2019, 5:44am UTC](https://discuss.elastic.co/t/setting-log-time-to-timestamp-in-logstash/186374 "2019-06-20T05:44:52Z")
Hi, In our logs, we have our date time as below and trying to set the same to timestamp field in logstash. But it's giving dateparsefailure. Any help in this is useful. Logs : the below data is mapped as DateTime usin… --- ## [Separate a field generated by the kv filter](https://discuss.elastic.co/t/separate-a-field-generated-by-the-kv-filter/186365)
**Author:** [@GGUERRA](https://discuss.elastic.co/u/GGUERRA)\ **Replies:** 2\ **Last updated:** [June 20, 2019, 2:18am UTC](https://discuss.elastic.co/t/separate-a-field-generated-by-the-kv-filter/186365 "2019-06-20T02:18:20Z")
Hello everyone, I tell you my situation. I'm filtering a fortigate log using the kv filter as follows: kv { source =\> "message" target =\> "kv" } (I would like to emphasize that the fields that come in … --- ## [Cisco ASA grok parse failure](https://discuss.elastic.co/t/cisco-asa-grok-parse-failure/186168)
**Author:** [@VamPikmin](https://discuss.elastic.co/u/VamPikmin)\ **Replies:** 6\ **Last updated:** [June 20, 2019, 2:14am UTC](https://discuss.elastic.co/t/cisco-asa-grok-parse-failure/186168 "2019-06-20T02:14:44Z")
When I test this pattern on http://grokdebug.herokuapp.com/ it works however when I run it through logstash I get a \_grokparsefailure \<166\>Jun 17 2019 07:13:33: %ASA-6-305011: Built dynamic TCP translation from any:192.… --- ## [Logstash 7.1.1 - Any quote symbol except ' and " for ruby code?](https://discuss.elastic.co/t/logstash-7-1-1-any-quote-symbol-except-and-for-ruby-code/186527)
**Author:** [@Alexl](https://discuss.elastic.co/u/Alexl)\ **Replies:** 3\ **Last updated:** [June 19, 2019, 10:35pm UTC](https://discuss.elastic.co/t/logstash-7-1-1-any-quote-symbol-except-and-for-ruby-code/186527 "2019-06-19T22:35:33Z")
Hello all! I'm trying to pass obtained by grok variable data into ruby code filter and I need to do that in double quotas of plain ruby code which contain 'somestring' that I would like to concatenate with my variable. … --- ## [Using cloudid in logstash](https://discuss.elastic.co/t/using-cloudid-in-logstash/186057)
**Author:** [@werowe](https://discuss.elastic.co/u/werowe)\ **Replies:** 1\ **Last updated:** [June 19, 2019, 7:11pm UTC](https://discuss.elastic.co/t/using-cloudid-in-logstash/186057 "2019-06-19T19:11:14Z")
This feature must be new because there are no examples. You are supposed to put cloud.id and cloud.auth in config/logstash.yml. My understanding is that would override the url provided in the logstash config file.But I… --- ## [\[ERROR\]\[logstash.filters.ruby \] Ruby exception occurred: undefined method \`split' for nil:NilClass](https://discuss.elastic.co/t/error-logstash-filters-ruby-ruby-exception-occurred-undefined-method-split-for-nil-nilclass/178329)
**Author:** [@mattsdevop](https://discuss.elastic.co/u/mattsdevop)\ **Replies:** 9\ **Last updated:** [June 19, 2019, 6:36pm UTC](https://discuss.elastic.co/t/error-logstash-filters-ruby-ruby-exception-occurred-undefined-method-split-for-nil-nilclass/178329 "2019-06-19T18:36:37Z")
Attempting to send syslog from a Barracuda firewall to logstash. Hoping somebody can help out here. Using logstash configuration from here (waf.conf) with the appropriate changes for my stack: https://campus.barracuda.co… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=523) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=525)