# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=467 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 468 --- ## [Transform input before multiline processing?](https://discuss.elastic.co/t/transform-input-before-multiline-processing/202242)
**Author:** [@dpr](https://discuss.elastic.co/u/dpr)\ **Replies:** 1\ **Last updated:** [October 3, 2019, 10:16pm UTC](https://discuss.elastic.co/t/transform-input-before-multiline-processing/202242 "2019-10-03T22:16:09Z")
Thanks for looking at my question. I am using the TCP input to receive XML data from a remote host, which I am parsing into XML stanzas using multiline. This works pretty well. Unfortunately, the remote host (which I … --- ## [Grok filter multiple match](https://discuss.elastic.co/t/grok-filter-multiple-match/202248)
**Author:** [@cactus](https://discuss.elastic.co/u/cactus)\ **Replies:** 3\ **Last updated:** [October 3, 2019, 10:54pm UTC](https://discuss.elastic.co/t/grok-filter-multiple-match/202248 "2019-10-03T22:54:16Z")
Hello, Can someone point me to a proper way with grok parsing? :slight\_smile: I want to parse syslog messages, for this purpose I created 2 syslog pattern within grok: filter { if \[type\] == 'syslog' { grok { … --- ## [Need help in using logstash with grok and fingerprint](https://discuss.elastic.co/t/need-help-in-using-logstash-with-grok-and-fingerprint/201109)
**Author:** [@Nara725](https://discuss.elastic.co/u/Nara725)\ **Replies:** 9\ **Last updated:** [October 3, 2019, 9:50pm UTC](https://discuss.elastic.co/t/need-help-in-using-logstash-with-grok-and-fingerprint/201109 "2019-10-03T21:50:30Z")
Hi, I'm new to logstash and i have the below requirement: My logs look like below and i dont want to send all 3 lines to Kibana, instead want to send only one line for a given time range. \[2019-09-25 17:17:33.153\] \[lo… --- ## [HTTP output plugin clarification](https://discuss.elastic.co/t/http-output-plugin-clarification/202231)
**Author:** [@nxns1](https://discuss.elastic.co/u/nxns1)\ **Replies:** 1\ **Last updated:** [October 3, 2019, 7:39pm UTC](https://discuss.elastic.co/t/http-output-plugin-clarification/202231 "2019-10-03T19:39:07Z")
Hi, Im sending a json format input to logstash, and I have it configured to post to a http endpoint. When I output the data on stdout I get something like: { "path" =\> "blabla", "@timestamp" =\> blabla, "host" =\> "bla… --- ## [How to create index as per file creation date?](https://discuss.elastic.co/t/how-to-create-index-as-per-file-creation-date/202206)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 3\ **Last updated:** [October 3, 2019, 7:35pm UTC](https://discuss.elastic.co/t/how-to-create-index-as-per-file-creation-date/202206 "2019-10-03T19:35:00Z")
Hi Experts , For now I am creating index in ES on daily basis with below logic output { elasticsearch { action =\> "index" hosts =\> \["elastic:1234"\] … --- ## [Migrar BD Mysql a otra BD Mysql remota con logstash](https://discuss.elastic.co/t/migrar-bd-mysql-a-otra-bd-mysql-remota-con-logstash/202234)
**Author:** [@alejandro\_del\_rey](https://discuss.elastic.co/u/alejandro_del_rey)\ **Replies:** 0\ **Last updated:** [October 3, 2019, 7:26pm UTC](https://discuss.elastic.co/t/migrar-bd-mysql-a-otra-bd-mysql-remota-con-logstash/202234 "2019-10-03T19:26:16Z")
Hola, esoy usando logstash, y estoy sincronizando mi BD local con una que esta en otro servidor, mediante el input y el ouput de logstash.... el problema me viene cuando pierdo la red, la conexion con el servidor remoto.… --- ## [Logstash monitoring not showing in Kibana](https://discuss.elastic.co/t/logstash-monitoring-not-showing-in-kibana/202056)
**Author:** [@drew0624](https://discuss.elastic.co/u/drew0624)\ **Replies:** 5\ **Last updated:** [October 3, 2019, 4:06pm UTC](https://discuss.elastic.co/t/logstash-monitoring-not-showing-in-kibana/202056 "2019-10-03T16:06:14Z")
Hello, I'm setting up a brand new ELK stack. All nodes are on 7.4.0. Xpack monitoring is turned on and I can see metrics for Elasticsearch and Kibana with no issues. However, nothing is showing for my Logstash nodes. It… --- ## [Is logstash required to parse url.query field into separate parts?](https://discuss.elastic.co/t/is-logstash-required-to-parse-url-query-field-into-separate-parts/202202)
**Author:** [@antonio84](https://discuss.elastic.co/u/antonio84)\ **Replies:** 0\ **Last updated:** [October 3, 2019, 3:23pm UTC](https://discuss.elastic.co/t/is-logstash-required-to-parse-url-query-field-into-separate-parts/202202 "2019-10-03T15:23:55Z")
ELK 7.3 and Filebeat 7.3 I'm using the IIS module currently, and everything is parsing great. I would like to further parse the url.query section of this line. It is highlighted in bold. 2019-09-24 17:14:04 10.202.225… --- ## [Metatrader, how parse this?](https://discuss.elastic.co/t/metatrader-how-parse-this/201982)
**Author:** [@immo](https://discuss.elastic.co/u/immo)\ **Replies:** 5\ **Last updated:** [October 3, 2019, 1:27pm UTC](https://discuss.elastic.co/t/metatrader-how-parse-this/201982 "2019-10-03T13:27:03Z")
Hello! Iam new there, and have some problems with log. Main problem - log have not only same lines, so i just cant grok it. Here is an example: EL 0 4 23:36:56.041 Glfx mt5 live\_capital\_m datafeed process start compl… --- ## [How to pass basic authentication details in http filter plugin](https://discuss.elastic.co/t/how-to-pass-basic-authentication-details-in-http-filter-plugin/201246)
**Author:** [@saroja](https://discuss.elastic.co/u/saroja)\ **Replies:** 17\ **Last updated:** [October 3, 2019, 12:50pm UTC](https://discuss.elastic.co/t/how-to-pass-basic-authentication-details-in-http-filter-plugin/201246 "2019-10-03T12:50:02Z")
Hi, I am using logstash HTTP filter plugin. I want to pass basic authentication details such as username, and password in the request. Could you please suggest, how we can post a request using header, body, and basic-… --- ## [Grok Custom Timestamp Filter](https://discuss.elastic.co/t/grok-custom-timestamp-filter/202141)
**Author:** [@Ghassan\_Zein](https://discuss.elastic.co/u/Ghassan_Zein)\ **Replies:** 5\ **Last updated:** [October 3, 2019, 12:24pm UTC](https://discuss.elastic.co/t/grok-custom-timestamp-filter/202141 "2019-10-03T12:24:52Z")
Am not able to filter the following date through grok, it doesnt fall under any ready made TIMESTAMP filters: 07 Aug 2019 13:54:27.463 I want to retrive it and replace it with @timestamp (or create a "Date" variable an… --- ## [Logstash 2.4.X performance check](https://discuss.elastic.co/t/logstash-2-4-x-performance-check/202105)
**Author:** [@n3RDY](https://discuss.elastic.co/u/n3RDY)\ **Replies:** 0\ **Last updated:** [October 3, 2019, 7:24am UTC](https://discuss.elastic.co/t/logstash-2-4-x-performance-check/202105 "2019-10-03T07:24:38Z")
Hi team, i am using old logstash version 2.4.X and planning to upgrade in near future, is there any possibility i can check logstash(2.4.X) performance via some API or plugin. any help will be appreciated. br, Rajni… --- ## [Logstash\_Parsing\_Issue](https://discuss.elastic.co/t/logstash-parsing-issue/202093)
**Author:** [@rkcharlie](https://discuss.elastic.co/u/rkcharlie)\ **Replies:** 0\ **Last updated:** [October 3, 2019, 6:13am UTC](https://discuss.elastic.co/t/logstash-parsing-issue/202093 "2019-10-03T06:13:58Z")
Hi Folks, Can you please help me in parsing below data in green fields , out of whole data (i don't wont data in red) --- ## [Logstash 400](https://discuss.elastic.co/t/logstash-400/201871)
**Author:** [@jtdepalm](https://discuss.elastic.co/u/jtdepalm)\ **Replies:** 5\ **Last updated:** [October 2, 2019, 8:53pm UTC](https://discuss.elastic.co/t/logstash-400/201871 "2019-10-02T20:53:55Z")
Hello all, Some background: My Elasticsearch cluster is in Azure. I used the Azure template ( https://www.elastic.co/blog/deploying-elasticsearch-on-microsoft-azure ) to initially create the cluster with a Kibana VM, L… --- ## [Anyone succeeded in Logstash output to Clickhouse in some way?](https://discuss.elastic.co/t/anyone-succeeded-in-logstash-output-to-clickhouse-in-some-way/172002)
**Author:** [@paulsbuxton](https://discuss.elastic.co/u/paulsbuxton)\ **Replies:** 1\ **Last updated:** [October 2, 2019, 8:15pm UTC](https://discuss.elastic.co/t/anyone-succeeded-in-logstash-output-to-clickhouse-in-some-way/172002 "2019-10-02T20:15:19Z")
For various reasons, I'm trying to use logstash to push to Clickhouse using the plugin at https://github.com/mikechris/logstash-output-clickhouse, with data being source from a SQLServer input. The SQLServer side is abs… --- ## [Substitution in metrics plugin's options only works for meter itself](https://discuss.elastic.co/t/substitution-in-metrics-plugins-options-only-works-for-meter-itself/201965)
**Author:** [@dmytrok](https://discuss.elastic.co/u/dmytrok)\ **Replies:** 2\ **Last updated:** [October 2, 2019, 7:43pm UTC](https://discuss.elastic.co/t/substitution-in-metrics-plugins-options-only-works-for-meter-itself/201965 "2019-10-02T19:43:47Z")
Hello, I wonder if what I see is a bug, or I have misunderstood something, would very appreciate some help. I have the next configuration of metrics plugin for Logstash: filter { metrics { meter =\> "loglines\_from\_%… --- ## [Conditional statement ignored in output module](https://discuss.elastic.co/t/conditional-statement-ignored-in-output-module/202030)
**Author:** [@erikk](https://discuss.elastic.co/u/erikk)\ **Replies:** 1\ **Last updated:** [October 2, 2019, 6:42pm UTC](https://discuss.elastic.co/t/conditional-statement-ignored-in-output-module/202030 "2019-10-02T18:42:03Z")
I'm having an issues with Logstash 7.2.1 where conditional statements within the output module are being ignored. Example, I made up \[blahblahblah\], so it is empty. output { elasticsearch { id =\> "output\_elastics… --- ## [Can't get Filebeat log data into Logstash container. Version 7.3.2](https://discuss.elastic.co/t/cant-get-filebeat-log-data-into-logstash-container-version-7-3-2/202003)
**Author:** [@Scott\_Moore](https://discuss.elastic.co/u/Scott_Moore)\ **Replies:** 4\ **Last updated:** [October 2, 2019, 6:28pm UTC](https://discuss.elastic.co/t/cant-get-filebeat-log-data-into-logstash-container-version-7-3-2/202003 "2019-10-02T18:28:06Z")
have a single CENTOS 7 server (let's call it DOCKERHOST with an IP address of 1.2.3.4 - but I am not using the actual IP addresses). It is hosting multiple Docker containers that includes almost all of the Elastic stack. … --- ## [Help parsing large CSV files](https://discuss.elastic.co/t/help-parsing-large-csv-files/201974)
**Author:** [@megajune](https://discuss.elastic.co/u/megajune)\ **Replies:** 2\ **Last updated:** [October 2, 2019, 5:57pm UTC](https://discuss.elastic.co/t/help-parsing-large-csv-files/201974 "2019-10-02T17:57:56Z")
Hello, I'm trying to write a logstash plugin for Symantec. JSON isn't an option for this product. Here's an example of the log I'm trying to parse. In actual product, this log will have many more fields. SymantecServer… --- ## [Troubleshoot logstash. Find dropped events by logs](https://discuss.elastic.co/t/troubleshoot-logstash-find-dropped-events-by-logs/202018)
**Author:** [@smerzlyakov](https://discuss.elastic.co/u/smerzlyakov)\ **Replies:** 0\ **Last updated:** [October 2, 2019, 5:24pm UTC](https://discuss.elastic.co/t/troubleshoot-logstash-find-dropped-events-by-logs/202018 "2019-10-02T17:24:06Z")
Hello! Resently i noticed, that i doesn't get some Windows logs. I use WEF collector - Winlogbeat - Logstash - Elasticsearch. I checked WEF, messages were on it and it was ok, than i check winlogbeat logs - there is no… --- ## [How to configure the jdbc input plugin for mysql](https://discuss.elastic.co/t/how-to-configure-the-jdbc-input-plugin-for-mysql/201991)
**Author:** [@alejandro\_del\_rey](https://discuss.elastic.co/u/alejandro_del_rey)\ **Replies:** 0\ **Last updated:** [October 2, 2019, 3:30pm UTC](https://discuss.elastic.co/t/how-to-configure-the-jdbc-input-plugin-for-mysql/201991 "2019-10-02T15:30:41Z")
HI. I work in windows, and i need use input plugin for mysql, i have this: jdbc { jdbc\_connection\_string =\> "jdbc:mysql://127.0.0.1:3306/ruby" jdbc\_user =\> "ale" jdbc\_password =\> "ale" #DRIVER jdbc\_driver\_library… --- ## [Logstash fails to start input.beats after elk 6.8 to 7.3 upgrade](https://discuss.elastic.co/t/logstash-fails-to-start-input-beats-after-elk-6-8-to-7-3-upgrade/201835)
**Author:** [@anelkuser](https://discuss.elastic.co/u/anelkuser)\ **Replies:** 1\ **Last updated:** [October 2, 2019, 2:52pm UTC](https://discuss.elastic.co/t/logstash-fails-to-start-input-beats-after-elk-6-8-to-7-3-upgrade/201835 "2019-10-02T14:52:17Z")
After 6.8.0 to 7.3.2 upgrade (no issues in 7.0 upgrade assistant in kibana) Elastic fails to start input.beats with the error below: \[2019-10-01T14:30:32,879\]\[WARN \]\[org.logstash.beats.Server\] Exception caught in channe… --- ## [How to remove double sign equal](https://discuss.elastic.co/t/how-to-remove-double-sign-equal/201973)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 2\ **Last updated:** [October 2, 2019, 2:49pm UTC](https://discuss.elastic.co/t/how-to-remove-double-sign-equal/201973 "2019-10-02T14:49:56Z")
Hello, I have logs like this : field1=value field2=value somedata==blablafoo field3=value I use kv to parse the logs. I have to remove double sign equal (==) because kv interpet this as a field, but when I use : mut… --- ## [Logstash XML Filter Issue](https://discuss.elastic.co/t/logstash-xml-filter-issue/201699)
**Author:** [@niall](https://discuss.elastic.co/u/niall)\ **Replies:** 3\ **Last updated:** [October 2, 2019, 1:22pm UTC](https://discuss.elastic.co/t/logstash-xml-filter-issue/201699 "2019-10-02T13:22:19Z")
Hi, I'm new to Logstash and i'm trying to parse the following sample XML file. I've tried using XPATH and Split filter plugins and still can't get it working. Can someone please advice or guide me please? Thanks a lot. … --- ## [5 logs for every audit event](https://discuss.elastic.co/t/5-logs-for-every-audit-event/201864)
**Author:** [@crazedmeph](https://discuss.elastic.co/u/crazedmeph)\ **Replies:** 2\ **Last updated:** [October 2, 2019, 1:19pm UTC](https://discuss.elastic.co/t/5-logs-for-every-audit-event/201864 "2019-10-02T13:19:16Z")
I am testing out how auditbeats work and upon logging into the box that auditbeat is enabled I get 5 authentication\_success messages. All 5 of the logs are exactly the same, the only value that is not the same is the \_id… --- ## [Migrating nested data from mysql to elasticsearch](https://discuss.elastic.co/t/migrating-nested-data-from-mysql-to-elasticsearch/201953)
**Author:** [@saifrehman](https://discuss.elastic.co/u/saifrehman)\ **Replies:** 1\ **Last updated:** [October 2, 2019, 1:05pm UTC](https://discuss.elastic.co/t/migrating-nested-data-from-mysql-to-elasticsearch/201953 "2019-10-02T13:05:50Z")
I have got following error when i run my script. ilsa@ILSA-LAPTOP-01:/usr/share/logstash$ sudo bin/logstash -f /etc/logstash/conf.d/mysql.conf WARNING: Could not find logstash.yml which is typically located in $LS\_HO… --- ## [Unable to ingest CSV files since Logstash 7.3](https://discuss.elastic.co/t/unable-to-ingest-csv-files-since-logstash-7-3/201895)
**Author:** [@vincent2mots](https://discuss.elastic.co/u/vincent2mots)\ **Replies:** 2\ **Last updated:** [October 2, 2019, 12:38pm UTC](https://discuss.elastic.co/t/unable-to-ingest-csv-files-since-logstash-7-3/201895 "2019-10-02T12:38:48Z")
Hi experts, I'm struggling with Logstash since the 7.3 release version. Here my logstash configuration file : input { file { path =\> \["C:\\samples\\sample.csv"\] start\_position =\> "beginning" sincedb\_path =\>… --- ## [Create custom pattern](https://discuss.elastic.co/t/create-custom-pattern/201948)
**Author:** [@kisuke](https://discuss.elastic.co/u/kisuke)\ **Replies:** 0\ **Last updated:** [October 2, 2019, 12:09pm UTC](https://discuss.elastic.co/t/create-custom-pattern/201948 "2019-10-02T12:09:04Z")
Hello, I try to create a custom pattern to include theses caracters \[;\] But I don't understand how can I include it in one custom pattern. Can you help me ? --- ## [Create multiple indexs with multiple input in logstash](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779)
**Author:** [@Asmaa\_Sarih](https://discuss.elastic.co/u/Asmaa_Sarih)\ **Replies:** 13\ **Last updated:** [October 2, 2019, 11:52am UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779 "2019-10-02T11:52:33Z")
Hi Team, Can anyone help me in confugiring multiple indexes from multiple input with logstash, I am unable to create multiple index in elastic (with multiple if conditions) . One index is getting created but not both, … --- ## [Syslog Cisco "illegal\_argument\_exception"](https://discuss.elastic.co/t/syslog-cisco-illegal-argument-exception/201939)
**Author:** [@DorianL](https://discuss.elastic.co/u/DorianL)\ **Replies:** 0\ **Last updated:** [October 2, 2019, 10:33am UTC](https://discuss.elastic.co/t/syslog-cisco-illegal-argument-exception/201939 "2019-10-02T10:33:08Z")
Hello, I want to export logs from my cisco router via logstash. Here is the filter that I put in place and which brings out the logs correctly: the filter ########################### filter { grok { patterns\_d… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=466) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=468)