# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=706 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 707 --- ## [Conditions and filters not applying](https://discuss.elastic.co/t/conditions-and-filters-not-applying/142637)
**Author:** [@pooch](https://discuss.elastic.co/u/pooch)\ **Replies:** 10\ **Last updated:** [August 7, 2018, 6:53pm UTC](https://discuss.elastic.co/t/conditions-and-filters-not-applying/142637 "2018-08-07T18:53:34Z")
Greetings All, I am new to ELK and stumbled all day yesterday trying to filter out logs that met a certain condition. Any insight as to how I should approach this is much appreciated. \*\*Failed attempt #1 input { bea… --- ## [Availiable actions for \_grokparsefailure (drop, mutate, ...) - need \_grokparsefailure events in a separate file](https://discuss.elastic.co/t/availiable-actions-for-grokparsefailure-drop-mutate-need-grokparsefailure-events-in-a-separate-file/143365)
**Author:** [@jiona](https://discuss.elastic.co/u/jiona)\ **Replies:** 2\ **Last updated:** [August 7, 2018, 4:48pm UTC](https://discuss.elastic.co/t/availiable-actions-for-grokparsefailure-drop-mutate-need-grokparsefailure-events-in-a-separate-file/143365 "2018-08-07T16:48:53Z")
For further analysis I need only \_grokparsefailure events in a separate file. I thought instead of "drop" I could use "file" (like in the output section) but it doesnt work. How can I do that or where can I find more i… --- ## [Combine fields into one field but from Grok match](https://discuss.elastic.co/t/combine-fields-into-one-field-but-from-grok-match/143370)
**Author:** [@waqark3389](https://discuss.elastic.co/u/waqark3389)\ **Replies:** 2\ **Last updated:** [August 7, 2018, 4:50pm UTC](https://discuss.elastic.co/t/combine-fields-into-one-field-but-from-grok-match/143370 "2018-08-07T16:50:11Z")
match =\> \["message", "(?\%{YEAR}.%{MONTHNUM}.%{MONTHDAY} %{TIME}Z) (?%{SYSLOG5424SD}):(?%{SYSLOG5424SD}):(?%{SYSLOG5424SD}) (?(.|\\r|\\n)\*)"\] break\_on\_match =\> false mutate { add\_field =\> {"lo… --- ## [Drop if message contains text from a List of Strings](https://discuss.elastic.co/t/drop-if-message-contains-text-from-a-list-of-strings/142102)
**Author:** [@Waleed\_Malik](https://discuss.elastic.co/u/Waleed_Malik)\ **Replies:** 1\ **Last updated:** [August 7, 2018, 4:30pm UTC](https://discuss.elastic.co/t/drop-if-message-contains-text-from-a-list-of-strings/142102 "2018-08-07T16:30:09Z")
Hi, I have a pre-defined set of Strings that i want to hard code in the conf file. if "some text " in \[message\] or "some text" in \[message\] { drop {} } I want to create an array of strings and loop through them a… --- ## [About config logstash for mongoDB](https://discuss.elastic.co/t/about-config-logstash-for-mongodb/143368)
**Author:** [@NIloufarVafaei](https://discuss.elastic.co/u/NIloufarVafaei)\ **Replies:** 0\ **Last updated:** [August 7, 2018, 4:03pm UTC](https://discuss.elastic.co/t/about-config-logstash-for-mongodb/143368 "2018-08-07T16:03:53Z")
hi i want to read data from mongoDB and i searched in official document of logstash and i did not find the any example configuration for mongoDB, so any one know about this kind of configuration? --- ## [Jdbc\_streaming filter help](https://discuss.elastic.co/t/jdbc-streaming-filter-help/142499)
**Author:** [@ripopo](https://discuss.elastic.co/u/ripopo)\ **Replies:** 1\ **Last updated:** [August 7, 2018, 3:53pm UTC](https://discuss.elastic.co/t/jdbc-streaming-filter-help/142499 "2018-08-07T15:53:31Z")
Hello, for example i have a database with columns id, a, b, c. here is my config file \< input { stdin { } } filter { jdbc\_streaming { jdbc\_driver\_library =\> "path/to/data.jar" jdbc\_driver\_class =\> "..." jdbc\_co… --- ## [Getting start (logstash 6.3.2-1 / filebeat 6.3.2) on debian 9.4 /](https://discuss.elastic.co/t/getting-start-logstash-6-3-2-1-filebeat-6-3-2-on-debian-9-4/143356)
**Author:** [@nonozi](https://discuss.elastic.co/u/nonozi)\ **Replies:** 2\ **Last updated:** [August 7, 2018, 3:32pm UTC](https://discuss.elastic.co/t/getting-start-logstash-6-3-2-1-filebeat-6-3-2-on-debian-9-4/143356 "2018-08-07T15:32:58Z")
Hello, New user, i'd like try the power of ELK on a debian 9.4 / openjdk version "1.8.0\_151" 2 days I try to run ELK with no success so I decided to start again from the beginning and I encountered a issue fairly quick… --- ## [Enriching winlogbeat data with the jdbc\_static filter](https://discuss.elastic.co/t/enriching-winlogbeat-data-with-the-jdbc-static-filter/142113)
**Author:** [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\ **Replies:** 1\ **Last updated:** [August 7, 2018, 3:14pm UTC](https://discuss.elastic.co/t/enriching-winlogbeat-data-with-the-jdbc-static-filter/142113 "2018-08-07T15:14:55Z")
Hi I have a MySQL database with the following table structure: ID, FQDN, IP, SERVICE I want to enrich the logs recevied from our Windows servers using winlogbeat. I'd like to add the following fields: ID =\> ID FQDN … --- ## [Netflow plugin installation error](https://discuss.elastic.co/t/netflow-plugin-installation-error/143209)
**Author:** [@thealy](https://discuss.elastic.co/u/thealy)\ **Replies:** 2\ **Last updated:** [August 7, 2018, 1:24pm UTC](https://discuss.elastic.co/t/netflow-plugin-installation-error/143209 "2018-08-07T13:24:39Z")
Setting up a test for the Netflow plugin, on freshly installed V6.3.2 on Red Hat Enterprise Linux Workstation release 6.10. Despite setting the no-verify flag, I get this result: /usr/share/logstash/bin/logstash-plugin… --- ## [Sending only Json format of log to Kafka server from logs containing both json and non json format in logstash config](https://discuss.elastic.co/t/sending-only-json-format-of-log-to-kafka-server-from-logs-containing-both-json-and-non-json-format-in-logstash-config/143168)
**Author:** [@Anshika](https://discuss.elastic.co/u/Anshika)\ **Replies:** 5\ **Last updated:** [August 7, 2018, 1:17pm UTC](https://discuss.elastic.co/t/sending-only-json-format-of-log-to-kafka-server-from-logs-containing-both-json-and-non-json-format-in-logstash-config/143168 "2018-08-07T13:17:31Z")
I am trying to send the Json format of the log from the logs containing both json and non json format to Kafka server. example of logs: 880 \<14\>1 2018-08-06T10:49:05.89677+00:00 dev.hello-world 24f9ade2-1bdb-46c2-bc5c-4… --- ## [How can i set two geoip source fields?](https://discuss.elastic.co/t/how-can-i-set-two-geoip-source-fields/143309)
**Author:** [@med\_amine](https://discuss.elastic.co/u/med_amine)\ **Replies:** 6\ **Last updated:** [August 7, 2018, 12:49pm UTC](https://discuss.elastic.co/t/how-can-i-set-two-geoip-source-fields/143309 "2018-08-07T12:49:30Z")
hello, i have two geoip sources the first is "clientip" and the second is "clientip\_e" so i use this syntax but nothing #it's error!!! geoip{ source =\> {"clientip" } } geoip{ source =\> {"clientip\_e" } } how ca… --- ## [How to configure conditional input-output from Filebeats](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121)
**Author:** [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)\ **Replies:** 10\ **Last updated:** [August 7, 2018, 12:48pm UTC](https://discuss.elastic.co/t/how-to-configure-conditional-input-output-from-filebeats/143121 "2018-08-07T12:48:42Z")
Hi there. I'm a newbie in ELK stack. I am trying to configure logstasth to gather data from filebeat and put it in different indices depending from sources' filenames. Filebeats config: filebeat.inputs: - type: log ena… --- ## [Logstash exceptionCaught() event](https://discuss.elastic.co/t/logstash-exceptioncaught-event/141814)
**Author:** [@Khaled\_Saidi](https://discuss.elastic.co/u/Khaled_Saidi)\ **Replies:** 12\ **Last updated:** [August 7, 2018, 12:18pm UTC](https://discuss.elastic.co/t/logstash-exceptioncaught-event/141814 "2018-08-07T12:18:51Z")
Hi, I use filebeat to send logs to logstash who is running in a docker's container on ubuntu. Since the gathered log files format was modified, i have a lot of grokparse failure. I don't know if there is a relation bu… --- ## [Updating data](https://discuss.elastic.co/t/updating-data/143186)
**Author:** [@Olha1](https://discuss.elastic.co/u/Olha1)\ **Replies:** 5\ **Last updated:** [August 7, 2018, 11:15am UTC](https://discuss.elastic.co/t/updating-data/143186 "2018-08-07T11:15:21Z")
Hi! I have a problem with updating my data in elasticsearch. I use a postgresql. When I transfer the data from database to elasticsearch through logstash, I have problem with update my data again and again. If I have, … --- ## [Simple GROK pattern not parsable](https://discuss.elastic.co/t/simple-grok-pattern-not-parsable/143262)
**Author:** [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Replies:** 4\ **Last updated:** [August 7, 2018, 8:04am UTC](https://discuss.elastic.co/t/simple-grok-pattern-not-parsable/143262 "2018-08-07T08:04:44Z")
Its weird that GROK pattern is not parsing the string : org.dozer.config.GlobalSettings.loadGlobalSettings %{JAVACLASS:class}\\.%{JAVAMETHOD:method} I see compile error at https://grokdebug.herokuapp.com/. Please sugge… --- ## [How to install and configure Logstash in Windows local machine? Is it Supporting?](https://discuss.elastic.co/t/how-to-install-and-configure-logstash-in-windows-local-machine-is-it-supporting/143252)
**Author:** [@Vignesh\_T](https://discuss.elastic.co/u/Vignesh_T)\ **Replies:** 1\ **Last updated:** [August 7, 2018, 8:55am UTC](https://discuss.elastic.co/t/how-to-install-and-configure-logstash-in-windows-local-machine-is-it-supporting/143252 "2018-08-07T08:55:15Z")
I am new to ELK stack. I am trying to create a Sample app using ELK stack. Right now I have downloaded Elastics Stack in my local Windows Machine from Elastics Website -\> Products Page. But, I can not find the Logstack .… --- ## [Logstash losing messages](https://discuss.elastic.co/t/logstash-losing-messages/139730)
**Author:** [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\ **Replies:** 11\ **Last updated:** [August 7, 2018, 8:54am UTC](https://discuss.elastic.co/t/logstash-losing-messages/139730 "2018-08-07T08:54:27Z")
Hello, everyone! I have a lot of syslog messages. And when I trying to filter them I found that logstash losing some messages. How can I investigate why? And how can I found howq many of messages are missed ? --- ## [How to convert datatype field from "string" to "integer" with mutate filter?](https://discuss.elastic.co/t/how-to-convert-datatype-field-from-string-to-integer-with-mutate-filter/143272)
**Author:** [@med\_amine](https://discuss.elastic.co/u/med_amine)\ **Replies:** 1\ **Last updated:** [August 7, 2018, 8:52am UTC](https://discuss.elastic.co/t/how-to-convert-datatype-field-from-string-to-integer-with-mutate-filter/143272 "2018-08-07T08:52:33Z")
hello , i'm using mutate filter to convert datatype filed from string to integer in logstash config file but no result it still showing string as type to "bytes\_sent" field , this is the filter ; mutate { convert =\> … --- ## [Logstash with a netflow module AND other config files](https://discuss.elastic.co/t/logstash-with-a-netflow-module-and-other-config-files/141763)
**Author:** [@samia](https://discuss.elastic.co/u/samia)\ **Replies:** 4\ **Last updated:** [August 7, 2018, 8:40am UTC](https://discuss.elastic.co/t/logstash-with-a-netflow-module-and-other-config-files/141763 "2018-08-07T08:40:26Z")
Hello, I'm wandering if it's possible to use logstash with an imported module along side with other configuration files. I'm using the netflow module. After a first setup (described here : https://www.elastic.co/guide… --- ## [Help with Ruby code in Logstash](https://discuss.elastic.co/t/help-with-ruby-code-in-logstash/142384)
**Author:** [@espogian](https://discuss.elastic.co/u/espogian)\ **Replies:** 5\ **Last updated:** [August 7, 2018, 8:18am UTC](https://discuss.elastic.co/t/help-with-ruby-code-in-logstash/142384 "2018-08-07T08:18:36Z")
Hi, I have to create a new field in a Logstash filter which should be the result of a math operation between two other fields. Given that I have defined in input section: convert =\> { "field1" =\> "integer" } convert =\> … --- ## [Aggregate problem](https://discuss.elastic.co/t/aggregate-problem/142572)
**Author:** [@Pierre2](https://discuss.elastic.co/u/Pierre2)\ **Replies:** 19\ **Last updated:** [August 7, 2018, 7:25am UTC](https://discuss.elastic.co/t/aggregate-problem/142572 "2018-08-07T07:25:19Z")
Hello ! My logstash config can deal with serveral log type. Two of them are "Appel" and "SVI". These two types have a similar field called "ID appelSVI". I want to add à field called "Passage par SVI" to my "Appel" eve… --- ## [Logstash grok break E0802 into E and 0802](https://discuss.elastic.co/t/logstash-grok-break-e0802-into-e-and-0802/143222)
**Author:** [@databeata](https://discuss.elastic.co/u/databeata)\ **Replies:** 2\ **Last updated:** [August 7, 2018, 12:04am UTC](https://discuss.elastic.co/t/logstash-grok-break-e0802-into-e-and-0802/143222 "2018-08-07T00:04:20Z")
How do I break E0802 into E and 0802? http://grokdebug.herokuapp.com/ E0802 05:01:29.979775 30003 logging.cc:121\] stderr will be logged to this file. %{NOTSPACE:log\_level} %{NOTSPACE:log\_time} %{NOTSPACE:log\_threa… --- ## [Mutate filter unable to convert date to string](https://discuss.elastic.co/t/mutate-filter-unable-to-convert-date-to-string/142518)
**Author:** [@ravian100](https://discuss.elastic.co/u/ravian100)\ **Replies:** 6\ **Last updated:** [August 6, 2018, 6:33pm UTC](https://discuss.elastic.co/t/mutate-filter-unable-to-convert-date-to-string/142518 "2018-08-06T18:33:05Z")
I have searched a lot regarding this issue and now I am posting this problem. here is my configuration file. input { file { path =\> "/home/Kunden2.csv" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } … --- ## [Import data from DB causes OutOfMemory](https://discuss.elastic.co/t/import-data-from-db-causes-outofmemory/142573)
**Author:** [@Isa](https://discuss.elastic.co/u/Isa)\ **Replies:** 4\ **Last updated:** [August 6, 2018, 6:32pm UTC](https://discuss.elastic.co/t/import-data-from-db-causes-outofmemory/142573 "2018-08-06T18:32:37Z")
Hi i am trying to import data from a postgreSQL DB however it never completes the imports becuase logstash runs out of Memory. If I limit the data in my select statement the data imports successfully. Current JVM Arg -… --- ## [ELK Server Failed](https://discuss.elastic.co/t/elk-server-failed/143179)
**Author:** [@Chris\_Billett](https://discuss.elastic.co/u/Chris_Billett)\ **Replies:** 8\ **Last updated:** [August 6, 2018, 4:53pm UTC](https://discuss.elastic.co/t/elk-server-failed/143179 "2018-08-06T16:53:03Z")
Hi, My ELK server failed 2 days again with an issue in Elasticsearch for 'Too Many Files Open' I managed to resolved this moving out old indexes. Now I can access Kibana, but im not seeing any new indexes being created… --- ## [Issues & questions about --path.config usage](https://discuss.elastic.co/t/issues-questions-about-path-config-usage/143184)
**Author:** [@somelifeDev](https://discuss.elastic.co/u/somelifeDev)\ **Replies:** 4\ **Last updated:** [August 6, 2018, 3:11pm UTC](https://discuss.elastic.co/t/issues-questions-about-path-config-usage/143184 "2018-08-06T15:11:56Z")
Greeting, To avoid having one big .conf receiving logs from filebeat on 5044, filtering through tons of ifs and elses, then forwarding to elastic, I've been trying to use the --path.config command lately as I understand… --- ## [Merge Multiline json into single line json using codec multiline plugin](https://discuss.elastic.co/t/merge-multiline-json-into-single-line-json-using-codec-multiline-plugin/143175)
**Author:** [@vinit](https://discuss.elastic.co/u/vinit)\ **Replies:** 1\ **Last updated:** [August 6, 2018, 2:08pm UTC](https://discuss.elastic.co/t/merge-multiline-json-into-single-line-json-using-codec-multiline-plugin/143175 "2018-08-06T14:08:58Z")
What pattern I need to apply in input plugin to merge multiline json? I am using below multiline json: { "end-time": "2018-07-18T07:43:01", "start-time": "2018-07-17T15:36:39", "status": "SUCCESS", "steps": \[ { "… --- ## [Unable to search with timestamp](https://discuss.elastic.co/t/unable-to-search-with-timestamp/143139)
**Author:** [@Vijay\_Chander](https://discuss.elastic.co/u/Vijay_Chander)\ **Replies:** 4\ **Last updated:** [August 6, 2018, 2:01pm UTC](https://discuss.elastic.co/t/unable-to-search-with-timestamp/143139 "2018-08-06T14:01:44Z")
I've created an index with two doc types(sas\_logs & pid\_logs) of different formats. The logstash config file is input { file { path =\> "E:/ElasticSearchWork/pid\_logs/.log" type =\> "pid\_logs" start\_position =\> "beg… --- ## [XML filter on Logstash](https://discuss.elastic.co/t/xml-filter-on-logstash/143174)
**Author:** [@Fram\_Souza](https://discuss.elastic.co/u/Fram_Souza)\ **Replies:** 3\ **Last updated:** [August 6, 2018, 1:52pm UTC](https://discuss.elastic.co/t/xml-filter-on-logstash/143174 "2018-08-06T13:52:37Z")
I'm indexing xml on Elasticsearch using xml filter, below a simple example (a original xml is very large): \\\\ **Author:** [@jmoffitt](https://discuss.elastic.co/u/jmoffitt)\ **Replies:** 7\ **Last updated:** [August 6, 2018, 12:07pm UTC](https://discuss.elastic.co/t/how-to-update-the-type-of-a-field/142641 "2018-08-06T12:07:10Z") I am having the same problem as listed at this link I first attempted to update the logstash template as described, but that appeared to completely wipe out everything but the field for location and caused writes from l… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=705) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=707)