# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=531 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 532 --- ## [Logstash filtering on eventId in ruby](https://discuss.elastic.co/t/logstash-filtering-on-eventid-in-ruby/185117)
**Author:** [@dries\_vanherpe](https://discuss.elastic.co/u/dries_vanherpe)\ **Replies:** 2\ **Last updated:** [June 11, 2019, 1:03pm UTC](https://discuss.elastic.co/t/logstash-filtering-on-eventid-in-ruby/185117 "2019-06-11T13:03:10Z")
So im trying to filter with the ruby filter plugin in Logstash. Im pretty new to ruby and Logstash but I have tried some code and it doesn't seem to work. It looks like it won't go inside the CSV.foreach. I want to give … --- ## [Centralized logging solution for custom application logs](https://discuss.elastic.co/t/centralized-logging-solution-for-custom-application-logs/185051)
**Author:** [@ygirouard\_stm](https://discuss.elastic.co/u/ygirouard_stm)\ **Replies:** 2\ **Last updated:** [June 11, 2019, 1:00pm UTC](https://discuss.elastic.co/t/centralized-logging-solution-for-custom-application-logs/185051 "2019-06-11T13:00:34Z")
We are looking for a solution that could help us centralize custom logs from different applications that are all part of a single system. The applications are all different in nature, some are .NET C# apps on IIS, some a… --- ## [Logstash Error - SystemExit](https://discuss.elastic.co/t/logstash-error-systemexit/185182)
**Author:** [@Gelinski](https://discuss.elastic.co/u/Gelinski)\ **Replies:** 0\ **Last updated:** [June 11, 2019, 12:13pm UTC](https://discuss.elastic.co/t/logstash-error-systemexit/185182 "2019-06-11T12:13:30Z")
Hi, Can anyone help me to fix the following error? \[2019-06-11T08:20:57,331\]\[ERROR\]\[org.logstash.execution.WorkerLoop\] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter c… --- ## [Logstash monitoring with basic license?](https://discuss.elastic.co/t/logstash-monitoring-with-basic-license/185165)
**Author:** [@nbglink](https://discuss.elastic.co/u/nbglink)\ **Replies:** 1\ **Last updated:** [June 11, 2019, 10:59am UTC](https://discuss.elastic.co/t/logstash-monitoring-with-basic-license/185165 "2019-06-11T10:59:03Z")
Hi there. Is it possible to visualize logstash monitoring in Kibana using basic license? I read documentation and set the settings as they are described for logstash but monitoring option doesn't appear in Kibana. Any… --- ## [How to make sum(field)-sum(field2) and get a single result for all documents](https://discuss.elastic.co/t/how-to-make-sum-field-sum-field2-and-get-a-single-result-for-all-documents/184979)
**Author:** [@Aymen\_Ben\_Moussa](https://discuss.elastic.co/u/Aymen_Ben_Moussa)\ **Replies:** 3\ **Last updated:** [June 11, 2019, 9:57am UTC](https://discuss.elastic.co/t/how-to-make-sum-field-sum-field2-and-get-a-single-result-for-all-documents/184979 "2019-06-11T09:57:24Z")
I have a mongodb collection that contains two fields, amount and type, in type i have either cash-in , cash-out or transfer. I want to add an index to elasticsearch with a new field which contains (the sum of all cash-in… --- ## [JDBC input and JDBC output in one config](https://discuss.elastic.co/t/jdbc-input-and-jdbc-output-in-one-config/184753)
**Author:** [@fesenych](https://discuss.elastic.co/u/fesenych)\ **Replies:** 2\ **Last updated:** [June 11, 2019, 7:43am UTC](https://discuss.elastic.co/t/jdbc-input-and-jdbc-output-in-one-config/184753 "2019-06-11T07:43:45Z")
Hi, I am trying to configure JDBC input and JDBC output in one configuration file. When in the configuration file is configured only on JDBC input or JDBC output, then everything works well when I try to combine them in … --- ## [Pattern for the following](https://discuss.elastic.co/t/pattern-for-the-following/185103)
**Author:** [@AtulxD123](https://discuss.elastic.co/u/AtulxD123)\ **Replies:** 0\ **Last updated:** [June 11, 2019, 6:39am UTC](https://discuss.elastic.co/t/pattern-for-the-following/185103 "2019-06-11T06:39:08Z")
Please help me with grokparse for the following : User@Host: db\_usr\_retaility\[db\_usr\_retaility\] currently i'm using match =\> { "message" =\> \["^# User@Host: %{USER:user}\[\[^\]+\]\]"\]} but it says no match found in grok deb… --- ## [Adding a new field in logstash](https://discuss.elastic.co/t/adding-a-new-field-in-logstash/185102)
**Author:** [@brendachi](https://discuss.elastic.co/u/brendachi)\ **Replies:** 0\ **Last updated:** [June 11, 2019, 6:35am UTC](https://discuss.elastic.co/t/adding-a-new-field-in-logstash/185102 "2019-06-11T06:35:57Z")
Hello guys. I would like to ask if it is possible to take a section of the path field in logstash and create a new field where i can put it . For example the path fiel come out like this: "path" : "/home/data/pr/number/p… --- ## [ELK Syslog - iptables filtering](https://discuss.elastic.co/t/elk-syslog-iptables-filtering/185091)
**Author:** [@hassaank97](https://discuss.elastic.co/u/hassaank97)\ **Replies:** 0\ **Last updated:** [June 11, 2019, 4:56am UTC](https://discuss.elastic.co/t/elk-syslog-iptables-filtering/185091 "2019-06-11T04:56:04Z")
Hi all, I am working on how to configure ELK stack to forward syslog file to ELK stack server using Filebeat. Everything looks fine. The logs are coming to Logstash in following format: input.type: log ecs.version… --- ## [Logstash multiple Pipeline with Kafka input and output plugin](https://discuss.elastic.co/t/logstash-multiple-pipeline-with-kafka-input-and-output-plugin/185094)
**Author:** [@Giridhar](https://discuss.elastic.co/u/Giridhar)\ **Replies:** 0\ **Last updated:** [June 11, 2019, 5:08am UTC](https://discuss.elastic.co/t/logstash-multiple-pipeline-with-kafka-input-and-output-plugin/185094 "2019-06-11T05:08:56Z")
my Logstash configuration files with kafka input and output plugin is working fine individually. when im trying to start logstash having single configuration file using pipeline it is also working. But if i have multipl… --- ## [Websocket input client threw expection](https://discuss.elastic.co/t/websocket-input-client-threw-expection/184917)
**Author:** [@Smack\_Alpha](https://discuss.elastic.co/u/Smack_Alpha)\ **Replies:** 2\ **Last updated:** [June 11, 2019, 5:02am UTC](https://discuss.elastic.co/t/websocket-input-client-threw-expection/184917 "2019-06-11T05:02:10Z")
I am using the websocket input plugin of logstash. I am using data from the websocket to be indexed on Elasticsearch. But it throws me a error. I tried some answers on your forum. Eventhough it fails. Help me with some s… --- ## [Changing log level for dissect filter](https://discuss.elastic.co/t/changing-log-level-for-dissect-filter/185083)
**Author:** [@Hari\_Prasad](https://discuss.elastic.co/u/Hari_Prasad)\ **Replies:** 0\ **Last updated:** [June 11, 2019, 4:02am UTC](https://discuss.elastic.co/t/changing-log-level-for-dissect-filter/185083 "2019-06-11T04:02:41Z")
In my Logstash, some of the pipelines have multiple Dissect filters. The dissect filter is expected to fail in many of the cases. The main problem that I am facing is that the logs for the Logstash is filled with Dissec… --- ## [Logstash 6.7.1 unable to start with error java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Hex.encodeHexString](https://discuss.elastic.co/t/logstash-6-7-1-unable-to-start-with-error-java-lang-nosuchmethoderror-org-apache-commons-codec-binary-hex-encodehexstring/185079)
**Author:** [@nagarjun1](https://discuss.elastic.co/u/nagarjun1)\ **Replies:** 0\ **Last updated:** [June 11, 2019, 2:38am UTC](https://discuss.elastic.co/t/logstash-6-7-1-unable-to-start-with-error-java-lang-nosuchmethoderror-org-apache-commons-codec-binary-hex-encodehexstring/185079 "2019-06-11T02:38:27Z")
JAVA VERSION : java version "1.8.0\_201" Java(TM) SE Runtime Environment (build 1.8.0\_201-b09) WINDOWS : 2008 R2 LOGSTASH : 6.7.1 COMMAND : logstash.bat --config.test\_and\_exit -f E:\\ELK\\logstash\\bin\\log… --- ## [Using csv file as dictionary for translate plugin](https://discuss.elastic.co/t/using-csv-file-as-dictionary-for-translate-plugin/185069)
**Author:** [@leeyu](https://discuss.elastic.co/u/leeyu)\ **Replies:** 6\ **Last updated:** [June 11, 2019, 2:26am UTC](https://discuss.elastic.co/t/using-csv-file-as-dictionary-for-translate-plugin/185069 "2019-06-11T02:26:48Z")
Hi all, I am new to elastic. Today I was trying to perform translate on a csv file. Including the dictionary in the conf works fine but logstash always shut down if I try to load the dictionary from a csv file. Can a… --- ## [How to set the period at which the Elasticsearch filter in logstash](https://discuss.elastic.co/t/how-to-set-the-period-at-which-the-elasticsearch-filter-in-logstash/185076)
**Author:** [@Yungyoung\_Ok](https://discuss.elastic.co/u/Yungyoung_Ok)\ **Replies:** 4\ **Last updated:** [June 11, 2019, 1:28am UTC](https://discuss.elastic.co/t/how-to-set-the-period-at-which-the-elasticsearch-filter-in-logstash/185076 "2019-06-11T01:28:33Z")
I'd like to set the period at which the Elasticsearch filter in logstash flies the query. When data is received through the input, the value of the id field is dropped into the query to obtain the name corresponding to … --- ## [Filter unito too large](https://discuss.elastic.co/t/filter-unito-too-large/185067)
**Author:** [@df5732f0e4bff6b59465](https://discuss.elastic.co/u/df5732f0e4bff6b59465)\ **Replies:** 3\ **Last updated:** [June 11, 2019, 12:38am UTC](https://discuss.elastic.co/t/filter-unito-too-large/185067 "2019-06-11T00:38:06Z")
Hello. Now, in my filter block, there are 480 lines of code, is it possible to separate it? --- ## [How to disable specific log output using log4j.properties?](https://discuss.elastic.co/t/how-to-disable-specific-log-output-using-log4j-properties/184787)
**Author:** [@bschaeffer](https://discuss.elastic.co/u/bschaeffer)\ **Replies:** 2\ **Last updated:** [June 10, 2019, 10:45pm UTC](https://discuss.elastic.co/t/how-to-disable-specific-log-output-using-log4j-properties/184787 "2019-06-10T22:45:26Z")
I am seeing a ton of org.logstash.beats.BeatHandler logs in logstash's debug out, but I would love to disable that so I can better debug our pipeline. How would I go about that in the log4j.properties file? Currently we… --- ## [Sflow supported by ELK?](https://discuss.elastic.co/t/sflow-supported-by-elk/184986)
**Author:** [@hohooss](https://discuss.elastic.co/u/hohooss)\ **Replies:** 0\ **Last updated:** [June 10, 2019, 12:40pm UTC](https://discuss.elastic.co/t/sflow-supported-by-elk/184986 "2019-06-10T12:40:10Z")
Hello, can logstash support the SFLOW protocol? If not, which ELK element can do that? thank you. --- ## [Logstash not work after enable xpack.security](https://discuss.elastic.co/t/logstash-not-work-after-enable-xpack-security/183631)
**Author:** [@Mrcow](https://discuss.elastic.co/u/Mrcow)\ **Replies:** 1\ **Last updated:** [June 10, 2019, 8:48pm UTC](https://discuss.elastic.co/t/logstash-not-work-after-enable-xpack-security/183631 "2019-06-10T20:48:50Z")
Recently i enabled xpack.security en elasticsearch but logstash stop working and i got this error: \[2019-05-30T17:16:08,636\]\[WARN \]\[logstash.outputs.elasticsearch\] Attempted to resurrect connection to dead ES instance, … --- ## [How to add root level field in logstash json output](https://discuss.elastic.co/t/how-to-add-root-level-field-in-logstash-json-output/185010)
**Author:** [@rahulkothanath](https://discuss.elastic.co/u/rahulkothanath)\ **Replies:** 1\ **Last updated:** [June 10, 2019, 5:41pm UTC](https://discuss.elastic.co/t/how-to-add-root-level-field-in-logstash-json-output/185010 "2019-06-10T17:41:15Z")
My logstash output is like this { "a":"b", "c":"d" } I want to add a new field called parent as a root to this existing json like below: { "parent":{ "a":"b", "c":"d" … --- ## [Logstash find a date/time in a log entry](https://discuss.elastic.co/t/logstash-find-a-date-time-in-a-log-entry/184995)
**Author:** [@paiz1556](https://discuss.elastic.co/u/paiz1556)\ **Replies:** 2\ **Last updated:** [June 10, 2019, 5:30pm UTC](https://discuss.elastic.co/t/logstash-find-a-date-time-in-a-log-entry/184995 "2019-06-10T17:30:33Z")
I've just started using logstash and I'm working on trying to parse a log and push it into solr using the solr\_http plugin. I successfully used a grok filter to pull the severity out as a separate field, but I can't see… --- ## [Bucket option must not contain a forward-slash (/)](https://discuss.elastic.co/t/bucket-option-must-not-contain-a-forward-slash/185035)
**Author:** [@Manishvsaraf](https://discuss.elastic.co/u/Manishvsaraf)\ **Replies:** 0\ **Last updated:** [June 10, 2019, 5:16pm UTC](https://discuss.elastic.co/t/bucket-option-must-not-contain-a-forward-slash/185035 "2019-06-10T17:16:45Z")
My logstash.conf is as below, know that something wrong with bucket path. but don't how to define. Every hour I am receiving log files at S3 bucket : abcd/logs/aa.com/r/year=2019/month=06/day=10/hour=16 Next hour loca… --- ## [Custom Date format with brackets](https://discuss.elastic.co/t/custom-date-format-with-brackets/185029)
**Author:** [@paiz1556](https://discuss.elastic.co/u/paiz1556)\ **Replies:** 1\ **Last updated:** [June 10, 2019, 5:11pm UTC](https://discuss.elastic.co/t/custom-date-format-with-brackets/185029 "2019-06-10T17:11:31Z")
Here is my sample log file: 20:54:50,406 TRACE \[com.test.my.process.module.GARBAGE06\] (Thread-15 (ActiveMQ-client-global-threads-1006309389)) \[2019-06-07\]\[20:04:12\]\[5239354\]\[TESTTRN \]\[GARBAGE06 \]\[corePr… --- ## [Grok pattern problem](https://discuss.elastic.co/t/grok-pattern-problem/184908)
**Author:** [@mmk1995](https://discuss.elastic.co/u/mmk1995)\ **Replies:** 1\ **Last updated:** [June 10, 2019, 4:39pm UTC](https://discuss.elastic.co/t/grok-pattern-problem/184908 "2019-06-10T16:39:24Z")
Hi all, I am facing a log pattern like this xxxxx.com on 2017-5-15 16:38:18 from 142.xx.xx.xx And I tried to use grok like this: grok { match =\> { "message" =\> \[ "Timestamp: %{DATESTAMP:timestamp}", "IP: %{IP:ip}… --- ## [Something is wrong with you config](https://discuss.elastic.co/t/something-is-wrong-with-you-config/184972)
**Author:** [@AtulxD123](https://discuss.elastic.co/u/AtulxD123)\ **Replies:** 3\ **Last updated:** [June 10, 2019, 4:16pm UTC](https://discuss.elastic.co/t/something-is-wrong-with-you-config/184972 "2019-06-10T16:16:33Z")
Following is my config file for the following log file. input : input { file { path =\> "/Users/atul.maurya/Desktop/Data1/squery1.txt" start\_position =\> "beginning" since\_db =\> "NUL" codec =\> multiline { pattern … --- ## [Logstash assigns fields to all filters](https://discuss.elastic.co/t/logstash-assigns-fields-to-all-filters/184818)
**Author:** [@johnbchron](https://discuss.elastic.co/u/johnbchron)\ **Replies:** 2\ **Last updated:** [June 10, 2019, 4:07pm UTC](https://discuss.elastic.co/t/logstash-assigns-fields-to-all-filters/184818 "2019-06-10T16:07:05Z")
Hello all, I can't get my non-beats index to have the correct fields. This may have come from a massive misunderstanding of how logstash/elasticsearch works, but in my understanding, when you have an input/filter/outpu… --- ## [Using basic auth for kafka output plugin logstash 6.4.2](https://discuss.elastic.co/t/using-basic-auth-for-kafka-output-plugin-logstash-6-4-2/185015)
**Author:** [@preddy75](https://discuss.elastic.co/u/preddy75)\ **Replies:** 0\ **Last updated:** [June 10, 2019, 3:46pm UTC](https://discuss.elastic.co/t/using-basic-auth-for-kafka-output-plugin-logstash-6-4-2/185015 "2019-06-10T15:46:06Z")
Hi, I see from documentation that the logstash kafka output plugin does not support basic auth. Unfortunately the security environment we are provided only allows basic auth. Is there anyway to use basic auth with logst… --- ## [Trim the file name remove last 2 characters and extension using grok](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631)
**Author:** [@Mohit\_Ruke](https://discuss.elastic.co/u/Mohit_Ruke)\ **Replies:** 9\ **Last updated:** [June 10, 2019, 2:30pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631 "2019-06-10T14:30:58Z")
HI Everyone, I am trying to push data from s3 to Es using logstash on Ec2. this is my config file for logstash input { s3 { bucket =\> "" access\_key\_id =\> "" secret\_access\_key =\> "\*" region =\> "us-east-1" codec =\>… --- ## [Load Multiple xml file from logstash into Elasticsearch](https://discuss.elastic.co/t/load-multiple-xml-file-from-logstash-into-elasticsearch/184886)
**Author:** [@rameshkr1994](https://discuss.elastic.co/u/rameshkr1994)\ **Replies:** 5\ **Last updated:** [June 10, 2019, 2:25pm UTC](https://discuss.elastic.co/t/load-multiple-xml-file-from-logstash-into-elasticsearch/184886 "2019-06-10T14:25:41Z")
Dear All. I am New Here please help me for below:- iHow can load multiple xml from logstash into ES. --- ## [Jmx metrics using logstash jmx input plugin](https://discuss.elastic.co/t/jmx-metrics-using-logstash-jmx-input-plugin/184889)
**Author:** [@460](https://discuss.elastic.co/u/460)\ **Replies:** 4\ **Last updated:** [June 10, 2019, 12:55pm UTC](https://discuss.elastic.co/t/jmx-metrics-using-logstash-jmx-input-plugin/184889 "2019-06-10T12:55:30Z")
Hello team I have installed jmx plugin to logstash 6.6.2 How do this plugin work. When I start the logstash with jmx input and jmx config as the input file.....It is giving an error....no such file or directory found. … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=530) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=532)