# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=811 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 812 --- ## [Date filter dosen't work](https://discuss.elastic.co/t/date-filter-dosent-work/121870)
**Author:** [@akml\_kk](https://discuss.elastic.co/u/akml_kk)\ **Replies:** 10\ **Last updated:** [March 1, 2018, 7:48am UTC](https://discuss.elastic.co/t/date-filter-dosent-work/121870 "2018-03-01T07:48:08Z")
hello why my date filter dosen't work, i've seen a lot of question here and they helped me to optimise my configuration, but date filter still dosen't work : my column1 still of type String and i still got a @timestamp … --- ## [Json payload masking using logstash](https://discuss.elastic.co/t/json-payload-masking-using-logstash/118158)
**Author:** [@Suman\_Reddy1](https://discuss.elastic.co/u/Suman_Reddy1)\ **Replies:** 4\ **Last updated:** [March 1, 2018, 7:03am UTC](https://discuss.elastic.co/t/json-payload-masking-using-logstash/118158 "2018-03-01T07:03:08Z")
Team, I am a newbie here, we are using ELK stack for log-management. We configured logstash such that it will read log messages from our logs and push them to Elastisearch. We are good till this part, now the issue came… --- ## [Not able to run logstash on RHEL server](https://discuss.elastic.co/t/not-able-to-run-logstash-on-rhel-server/121854)
**Author:** [@Rahulkumar](https://discuss.elastic.co/u/Rahulkumar)\ **Replies:** 6\ **Last updated:** [March 1, 2018, 5:58am UTC](https://discuss.elastic.co/t/not-able-to-run-logstash-on-rhel-server/121854 "2018-03-01T05:58:17Z")
Hi, I have successfully installed logstash. I have also uploaded config file under "/etc/logstash/conf.d" and tried starting logstash with below command: \[root@localhost Softwares\]# sudo rpm -i logstash-6.0.0.rpm warni… --- ## [Duplicate entries for same data in multiple indices](https://discuss.elastic.co/t/duplicate-entries-for-same-data-in-multiple-indices/121722)
**Author:** [@a\_kasireddy](https://discuss.elastic.co/u/a_kasireddy)\ **Replies:** 2\ **Last updated:** [February 28, 2018, 10:14pm UTC](https://discuss.elastic.co/t/duplicate-entries-for-same-data-in-multiple-indices/121722 "2018-02-28T22:14:42Z")
Hello All, I'm using logstash (5.3) to parse data and send it to elastic search. I currently have 3 configuration files under config.d which parses 3 different types of data (all data is being collected from a database… --- ## [How to get specific pieces of log instead of entire log](https://discuss.elastic.co/t/how-to-get-specific-pieces-of-log-instead-of-entire-log/121944)
**Author:** [@Criviere](https://discuss.elastic.co/u/Criviere)\ **Replies:** 1\ **Last updated:** [February 28, 2018, 9:29pm UTC](https://discuss.elastic.co/t/how-to-get-specific-pieces-of-log-instead-of-entire-log/121944 "2018-02-28T21:29:23Z")
Hello, I was wondering on how I would go about taking only specific pieces of data from a log instead of passing the entire log to elasticsearch. Is there a way to currently do this? --- ## [Geoip Field mapping](https://discuss.elastic.co/t/geoip-field-mapping/121904)
**Author:** [@Catalin\_Liviu\_Vrince](https://discuss.elastic.co/u/Catalin_Liviu_Vrince)\ **Replies:** 1\ **Last updated:** [February 28, 2018, 6:07pm UTC](https://discuss.elastic.co/t/geoip-field-mapping/121904 "2018-02-28T18:07:17Z")
Hi guys, I have one issue parsing the nested geoip filed on Logstash 5.6.7. I'm not able to find the right path for this. Please can you help me with that? Input: 10.57.49.85 - catalin.vrinceanu \[28/Feb/2018:16:44:58… --- ## [Add multiple types in filebeat for logstash output](https://discuss.elastic.co/t/add-multiple-types-in-filebeat-for-logstash-output/121909)
**Author:** [@red888](https://discuss.elastic.co/u/red888)\ **Replies:** 3\ **Last updated:** [February 28, 2018, 6:01pm UTC](https://discuss.elastic.co/t/add-multiple-types-in-filebeat-for-logstash-output/121909 "2018-02-28T18:01:18Z")
Is there a way I can have multiple types with filebeat and logstash? I have multiple app logs I want to send to different indices. BUT they will be processed the same way I want to do something like this: filebeat.pros… --- ## [How to properly config Logstash for multiple Pipelines](https://discuss.elastic.co/t/how-to-properly-config-logstash-for-multiple-pipelines/121894)
**Author:** [@a\_nicewood](https://discuss.elastic.co/u/a_nicewood)\ **Replies:** 1\ **Last updated:** [February 28, 2018, 5:59pm UTC](https://discuss.elastic.co/t/how-to-properly-config-logstash-for-multiple-pipelines/121894 "2018-02-28T17:59:27Z")
Hello, im currently having an issue with creating a multiple pipelines in logstash. I can't get enough information out of the documentation to get the pipelines running. Where in the logstash.yml config should i refer… --- ## [Logstash - Receive event including line feeds](https://discuss.elastic.co/t/logstash-receive-event-including-line-feeds/121897)
**Author:** [@Vadus](https://discuss.elastic.co/u/Vadus)\ **Replies:** 1\ **Last updated:** [February 28, 2018, 5:53pm UTC](https://discuss.elastic.co/t/logstash-receive-event-including-line-feeds/121897 "2018-02-28T17:53:40Z")
Hi folks, I try to read an event in Logstash, received via tcp. This event contains line feeds and unfortunately at the logstash filter, where I try to remove the linefeeds via mutate/gsub, I already receive each line o… --- ## [Redis input SSL](https://discuss.elastic.co/t/redis-input-ssl/119804)
**Author:** [@Ryan\_Grannell](https://discuss.elastic.co/u/Ryan_Grannell)\ **Replies:** 2\ **Last updated:** [February 28, 2018, 5:46pm UTC](https://discuss.elastic.co/t/redis-input-ssl/119804 "2018-02-28T17:46:46Z")
As raised in the following posts: Redis input plugin -proxy and ssl Azure Redis Input plugin is it possible to read / write to Redis via TLS using the logstash input & output plugins? We're currently using Stunnel to … --- ## [Configuring Logstash 5.6.5 on DCOS](https://discuss.elastic.co/t/configuring-logstash-5-6-5-on-dcos/121907)
**Author:** [@aysala](https://discuss.elastic.co/u/aysala)\ **Replies:** 0\ **Last updated:** [February 28, 2018, 5:30pm UTC](https://discuss.elastic.co/t/configuring-logstash-5-6-5-on-dcos/121907 "2018-02-28T17:30:45Z")
Hi, I am currently running Elastic and Kibana of version 5.6.5 from Catalog on my DCOS cluster. I am trying to deploy Logstash as well. The default logstash service available in the catalog is of the version 2.4.5 and … --- ## [Gelf with log4j 1.2](https://discuss.elastic.co/t/gelf-with-log4j-1-2/121131)
**Author:** [@Mouglou](https://discuss.elastic.co/u/Mouglou)\ **Replies:** 3\ **Last updated:** [February 28, 2018, 4:59pm UTC](https://discuss.elastic.co/t/gelf-with-log4j-1-2/121131 "2018-02-28T16:59:17Z")
Hi all, First post on this forum, so hi all ! And long life to elastic ! :vulcan\_salute: I'm here cause I need some advice about GELF and log4j. As for now, we used "log4j" input to receive data from app using log4j 1… --- ## [Index Pattern issue](https://discuss.elastic.co/t/index-pattern-issue/121879)
**Author:** [@Bakkali\_Amine](https://discuss.elastic.co/u/Bakkali_Amine)\ **Replies:** 4\ **Last updated:** [February 28, 2018, 4:19pm UTC](https://discuss.elastic.co/t/index-pattern-issue/121879 "2018-02-28T16:19:39Z")
When I add an "if" in my output kibana don't know the pattern --- ## [Use script in Logstash](https://discuss.elastic.co/t/use-script-in-logstash/121004)
**Author:** [@Adren](https://discuss.elastic.co/u/Adren)\ **Replies:** 5\ **Last updated:** [February 28, 2018, 3:45pm UTC](https://discuss.elastic.co/t/use-script-in-logstash/121004 "2018-02-28T15:45:03Z")
Hello, I have a question, can we use a script in Logstash pipeline? I found that we can use ruby code in the pipeline but can we use another language or maybe execute the script file we want ? This is the pipeline I wa… --- ## [Logstash - xmx parameter from jvm.properties not read](https://discuss.elastic.co/t/logstash-xmx-parameter-from-jvm-properties-not-read/121871)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 1\ **Last updated:** [February 28, 2018, 3:10pm UTC](https://discuss.elastic.co/t/logstash-xmx-parameter-from-jvm-properties-not-read/121871 "2018-02-28T15:10:12Z")
Hi, seems if I made some dumb error, but I do not find it. I extracted the logstash tar to /usr/local/elastic/logstash/logstash-6.2.1/ I changed /usr/local/elastic/logstash/logstash-6.2.1/config/jvm.options to followi… --- ## [Can't create a new index on elastic with logstash](https://discuss.elastic.co/t/cant-create-a-new-index-on-elastic-with-logstash/121865)
**Author:** [@geppo](https://discuss.elastic.co/u/geppo)\ **Replies:** 4\ **Last updated:** [February 28, 2018, 3:04pm UTC](https://discuss.elastic.co/t/cant-create-a-new-index-on-elastic-with-logstash/121865 "2018-02-28T15:04:46Z")
Hi, is the first time for me that I use Logstash. I'm trying to index my apache log on localhost to a server. I can't see any error in /var/log/logstash/logstash-plain.log: \[2018-02-28T14:45:29,984\]\[INFO \]\[logstash.out… --- ## [Logstash Input - beats and azureblob](https://discuss.elastic.co/t/logstash-input-beats-and-azureblob/121866)
**Author:** [@charan.gandra](https://discuss.elastic.co/u/charan.gandra)\ **Replies:** 2\ **Last updated:** [February 28, 2018, 2:27pm UTC](https://discuss.elastic.co/t/logstash-input-beats-and-azureblob/121866 "2018-02-28T14:27:22Z")
Hi, I am using Filebeat to send the log files from remote host to logstash and also trying to grab NSG flow logs from Azure using logstash plugin. Here is my input.conf, how can I parse these two individual inputs sepa… --- ## [Data Import to Elasticsearch from MySQL](https://discuss.elastic.co/t/data-import-to-elasticsearch-from-mysql/121849)
**Author:** [@devil\_srj7](https://discuss.elastic.co/u/devil_srj7)\ **Replies:** 2\ **Last updated:** [February 28, 2018, 1:15pm UTC](https://discuss.elastic.co/t/data-import-to-elasticsearch-from-mysql/121849 "2018-02-28T13:15:35Z")
i use the following configuration file: input { jdbc { jdbc\_connection\_string =\> "jdbc:mysql://localhost:3306/live\_test" jdbc\_user =\> "teddy" jdbc\_password =\> "cdac" jdbc\_driver\_lib… --- ## [Json Parsing Issue](https://discuss.elastic.co/t/json-parsing-issue/121717)
**Author:** [@wjdavis5](https://discuss.elastic.co/u/wjdavis5)\ **Replies:** 3\ **Last updated:** [February 28, 2018, 11:45am UTC](https://discuss.elastic.co/t/json-parsing-issue/121717 "2018-02-28T11:45:43Z")
Given the following config: input { udp { type=\> "genericJson2" port =\> 9994 } } filter{ if \[type\] == "genericJson2" { json { source =\> "message" … --- ## [Logstash missing records in elastic search](https://discuss.elastic.co/t/logstash-missing-records-in-elastic-search/121842)
**Author:** [@vinaygarg30](https://discuss.elastic.co/u/vinaygarg30)\ **Replies:** 0\ **Last updated:** [February 28, 2018, 11:33am UTC](https://discuss.elastic.co/t/logstash-missing-records-in-elastic-search/121842 "2018-02-28T11:33:12Z")
Hi, I have used the configration for logstash to dump data from MYSQL 5.5 to Elastic Search 6.1.2. input { jdbc { jdbc\_driver\_library =\> "/home/ubuntu/mysql-connector-java-5.1.45/mysql-connector-java-5.1.45-bin.jar" … --- ## [Error starting logstash 6.2.2](https://discuss.elastic.co/t/error-starting-logstash-6-2-2/121779)
**Author:** [@lynx](https://discuss.elastic.co/u/lynx)\ **Replies:** 2\ **Last updated:** [February 28, 2018, 11:17am UTC](https://discuss.elastic.co/t/error-starting-logstash-6-2-2/121779 "2018-02-28T11:17:07Z")
Dear All I have installed logstash-6.2.2 and when trying to start the service with systemctl i get the below errors. Please advice NameError: cannot link Java class org.apache.logging.log4j.Level needs Java 7 (java.lan… --- ## [Creating logstash file](https://discuss.elastic.co/t/creating-logstash-file/121810)
**Author:** [@Nada\_I](https://discuss.elastic.co/u/Nada_I)\ **Replies:** 3\ **Last updated:** [February 28, 2018, 11:05am UTC](https://discuss.elastic.co/t/creating-logstash-file/121810 "2018-02-28T11:05:04Z")
How can i create the logstash file and how to run bin/logstash -f logstash.conf ? --- ## [How to extract timestamp from app logs](https://discuss.elastic.co/t/how-to-extract-timestamp-from-app-logs/121745)
**Author:** [@proximator](https://discuss.elastic.co/u/proximator)\ **Replies:** 4\ **Last updated:** [February 28, 2018, 10:57am UTC](https://discuss.elastic.co/t/how-to-extract-timestamp-from-app-logs/121745 "2018-02-28T10:57:02Z")
Hello All my app logs start with a timestamp having this format: yyyy-MM-dd HH:mm:SS Z Log messages can have a different format and I have some grok pattern to parse some of them, but what I want is to always extract t… --- ## [Logstash 1.5.4-9 sending incomplete logs to Elasticsearch](https://discuss.elastic.co/t/logstash-1-5-4-9-sending-incomplete-logs-to-elasticsearch/121815)
**Author:** [@Vanya\_Sharma](https://discuss.elastic.co/u/Vanya_Sharma)\ **Replies:** 0\ **Last updated:** [February 28, 2018, 9:30am UTC](https://discuss.elastic.co/t/logstash-1-5-4-9-sending-incomplete-logs-to-elasticsearch/121815 "2018-02-28T09:30:28Z")
Hi, I'm using logstash 1.5.4-9 with multiline filter to parse and send logs to Elasticsearch 1.7.4. My configuration looks something like: input { file { type =\> "abc" tags =\> "abc" … --- ## [Logstash ignore continuous newline](https://discuss.elastic.co/t/logstash-ignore-continuous-newline/121803)
**Author:** [@byungik](https://discuss.elastic.co/u/byungik)\ **Replies:** 0\ **Last updated:** [February 28, 2018, 8:07am UTC](https://discuss.elastic.co/t/logstash-ignore-continuous-newline/121803 "2018-02-28T08:07:50Z")
I use Logstash 6.1.1 version. I use stdin for input and multiline codec. and then I saw message. I input newline continuously 5 times but I there's only \\n one time. I don't want to ignore blank newline because I have… --- ## [Could not create the Java virtual mechine](https://discuss.elastic.co/t/could-not-create-the-java-virtual-mechine/121808)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 0\ **Last updated:** [February 28, 2018, 8:47am UTC](https://discuss.elastic.co/t/could-not-create-the-java-virtual-mechine/121808 "2018-02-28T08:47:08Z")
After running Logstash some time I am getting a below error. \[image\] Even I set JVM options in logstash and I increased java heap size Please, someone, help me how to resolve this. Thanks in Advance... --- ## [Where to place deserializer class jar file?](https://discuss.elastic.co/t/where-to-place-deserializer-class-jar-file/121162)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 0\ **Last updated:** [February 23, 2018, 4:40am UTC](https://discuss.elastic.co/t/where-to-place-deserializer-class-jar-file/121162 "2018-02-23T04:40:28Z")
Hello all, I am using ELK GA 6.0.0. I am using kafka input and below is the configuration; input { kafka{ group\_id =\> "avro\_test\_group\_1" topics =\> \["avro\_test\_topic"\] bootstrap\_servers =\> "192.168.0.1:9092" c… --- ## [Request for GROK pattern](https://discuss.elastic.co/t/request-for-grok-pattern/121780)
**Author:** [@Srikanth\_FT](https://discuss.elastic.co/u/Srikanth_FT)\ **Replies:** 0\ **Last updated:** [February 28, 2018, 6:08am UTC](https://discuss.elastic.co/t/request-for-grok-pattern/121780 "2018-02-28T06:08:45Z")
My log is "ABCDE-Empfänger gestartet am 14.03.2017 00:00:35 Uhr IJK\_20170314\_000010\_180.OUT: Datei empfangen um 00:00:39 Uhr IJK-Datei analysieren... IJK-Datei erfolgreich verarbeitet ABCDE-Empfänger gestoppt am 14… --- ## [Several questions about logstash and filebeat](https://discuss.elastic.co/t/several-questions-about-logstash-and-filebeat/120999)
**Author:** [@jaas](https://discuss.elastic.co/u/jaas)\ **Replies:** 3\ **Last updated:** [February 28, 2018, 6:05am UTC](https://discuss.elastic.co/t/several-questions-about-logstash-and-filebeat/120999 "2018-02-28T06:05:18Z")
1.I am dealing with a database export file like this Nokia^A20180101^A1999.99 ^A RM-356 ^A1 ^A/UCWEB8.9.0.253/50/999^A20171209 and ^A is the delimeter. I checked https://github.com/kkos/oniguruma/blob/master/doc/RE#L4… --- ## [Logstash config unable to load](https://discuss.elastic.co/t/logstash-config-unable-to-load/121392)
**Author:** [@abhivj7b](https://discuss.elastic.co/u/abhivj7b)\ **Replies:** 5\ **Last updated:** [February 28, 2018, 5:21am UTC](https://discuss.elastic.co/t/logstash-config-unable-to-load/121392 "2018-02-28T05:21:45Z")
While running logstash config on my windows server I am getting this error. -java.lqng.illegqlstateException:org.jruby.exceptions.Raiseexceptions: Please help to fix. [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=810) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=812)