# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1046 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1047 --- ## [Logstash on kubernetes: not seeing any output](https://discuss.elastic.co/t/logstash-on-kubernetes-not-seeing-any-output/72962)
**Author:** [@ronakg](https://discuss.elastic.co/u/ronakg)\ **Replies:** 4\ **Last updated:** [January 29, 2017, 10:25pm UTC](https://discuss.elastic.co/t/logstash-on-kubernetes-not-seeing-any-output/72962 "2017-01-29T22:25:36Z")
Hi, I'm trying to bring up Logstash on our kubernetes cluster. I'm using the 5.1.2-alpine Docker image and bringing up the Logstash pod. I'm seeing that the input/output configuration is being parsed properly, but I do… --- ## [If \_jsonparsefailure in \[tags\]](https://discuss.elastic.co/t/if--jsonparsefailure-in-tags/73062)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 3\ **Last updated:** [January 29, 2017, 7:01pm UTC](https://discuss.elastic.co/t/if--jsonparsefailure-in-tags/73062 "2017-01-29T19:01:03Z")
My filter: filter { json { source =\> "message" } } sometimes, I see events tagged as "\_jsonparsefailure", however some events (NON-JSON) are being dropped completely. How do I output events to elasticsearch with… --- ## [UDP listener died \>\>IOError: closed stream](https://discuss.elastic.co/t/udp-listener-died-ioerror-closed-stream/72884)
**Author:** [@dmcn\_es](https://discuss.elastic.co/u/dmcn_es)\ **Replies:** 3\ **Last updated:** [January 29, 2017, 6:55pm UTC](https://discuss.elastic.co/t/udp-listener-died-ioerror-closed-stream/72884 "2017-01-29T18:55:44Z")
hi, the disk filled up in ELK server on 23/1 , I dealt with that by deleteing indices and then installed curator so the same situation would not arise again. However since 23/1 no new indices are created, even thou… --- ## [Logstash UDP input only processing ~1000 events/sec](https://discuss.elastic.co/t/logstash-udp-input-only-processing-1000-events-sec/72887)
**Author:** [@sip](https://discuss.elastic.co/u/sip)\ **Replies:** 1\ **Last updated:** [January 29, 2017, 6:53pm UTC](https://discuss.elastic.co/t/logstash-udp-input-only-processing-1000-events-sec/72887 "2017-01-29T18:53:05Z")
I have an 8core 32GB Windows VM running LS 2.3.2 ES 2.3.3 Kibana 4.5.1 and I max out the CPUs with 4 UDP workers and process ~1,000 UDP netflow events/sec. This seems really low compared to what I have seen other rates.… --- ## [Logstash service does not start: logstash: unrecognized service](https://discuss.elastic.co/t/logstash-service-does-not-start-logstash-unrecognized-service/72827)
**Author:** [@Soham\_Navadiya](https://discuss.elastic.co/u/Soham_Navadiya)\ **Replies:** 1\ **Last updated:** [January 29, 2017, 6:49pm UTC](https://discuss.elastic.co/t/logstash-service-does-not-start-logstash-unrecognized-service/72827 "2017-01-29T18:49:12Z")
I installed successfully logstash. but when I am trying to fire below command: sudo service logstash start I am getting: \> logstash: unrecognized service During installation my last output was like below. Using p… --- ## [Logstash string equality test weirdness](https://discuss.elastic.co/t/logstash-string-equality-test-weirdness/73115)
**Author:** [@nick-george](https://discuss.elastic.co/u/nick-george)\ **Replies:** 4\ **Last updated:** [January 29, 2017, 4:17pm UTC](https://discuss.elastic.co/t/logstash-string-equality-test-weirdness/73115 "2017-01-29T16:17:25Z")
Hi, Logstash version: logstash-5.1.2-1.noarch Java version: Oracle JRE 1.8.0\_111 OS: Oracle Linux 7.2 I am trying to use an environment variable to control a beat input configuration. To illustrate the issue below, … --- ## [Enrich existing elasticsearch index with new fields from logstash](https://discuss.elastic.co/t/enrich-existing-elasticsearch-index-with-new-fields-from-logstash/72749)
**Author:** [@Maher.Glenza](https://discuss.elastic.co/u/Maher.Glenza)\ **Replies:** 1\ **Last updated:** [January 29, 2017, 4:26am UTC](https://discuss.elastic.co/t/enrich-existing-elasticsearch-index-with-new-fields-from-logstash/72749 "2017-01-29T04:26:52Z")
hi , if any one know if i can and how to receive for the first time a log in logstash filter it then save it in ES , next step is to receive another type of logs filter it then add its fields to the first one saved in E… --- ## [Parsing Apache Tomcat application Logs](https://discuss.elastic.co/t/parsing-apache-tomcat-application-logs/72746)
**Author:** [@pDonS](https://discuss.elastic.co/u/pDonS)\ **Replies:** 1\ **Last updated:** [January 29, 2017, 4:26am UTC](https://discuss.elastic.co/t/parsing-apache-tomcat-application-logs/72746 "2017-01-29T04:26:23Z")
Hi , I am trying to parse below Apache Tomcar Application log , 2017-01-23 00:00:00,234 INFO \[FTP\_Channel-0\] aps.pistribution (PhannelImpl.java:205) - FTP\_Channel-0Refill completed\> Using below pattern , 20%{YEAR}-%{… --- ## [Split array without generating new event](https://discuss.elastic.co/t/split-array-without-generating-new-event/73097)
**Author:** [@paketecuento](https://discuss.elastic.co/u/paketecuento)\ **Replies:** 1\ **Last updated:** [January 28, 2017, 8:46pm UTC](https://discuss.elastic.co/t/split-array-without-generating-new-event/73097 "2017-01-28T20:46:13Z")
Hi, I'm trying to parse some JSON from scopus database. I'm almost done using mutate and split plugins, but I'm unable to get a final step. Using json plugin and rubydebug output, I get: : { "search-results": { "aut… --- ## [Logstash - client\_cert](https://discuss.elastic.co/t/logstash-client-cert/72759)
**Author:** [@avivc](https://discuss.elastic.co/u/avivc)\ **Replies:** 2\ **Last updated:** [January 28, 2017, 7:18pm UTC](https://discuss.elastic.co/t/logstash-client-cert/72759 "2017-01-28T19:18:21Z")
Hi, I'm new with Elastic I'm trying to add a client\_cert property to my configuration file (output of course) I'm nesting this property under elasticsearch curl bracket When trying to run Logstash machine, it pop an erro… --- ## [Input http dont know service](https://discuss.elastic.co/t/input-http-dont-know-service/73073)
**Author:** [@yoshua](https://discuss.elastic.co/u/yoshua)\ **Replies:** 2\ **Last updated:** [January 28, 2017, 4:42pm UTC](https://discuss.elastic.co/t/input-http-dont-know-service/73073 "2017-01-28T16:42:46Z")
\[2017-01-27T20:34:31,943\]\[ERROR\]\[logstash.agent \] Pipeline aborted due to error {:exception=\>#, :backtrace=\>\["org/jruby/ext/socket/RubyTCPServer.java:115:in initialize'", "org/jruby/RubyIO.java:871:innew'", "/u… --- ## [Can not find the multine plug-in in my logstash 5.1.2?](https://discuss.elastic.co/t/can-not-find-the-multine-plug-in-in-my-logstash-5-1-2/73081)
**Author:** [@rlv\_praveen](https://discuss.elastic.co/u/rlv_praveen)\ **Replies:** 4\ **Last updated:** [January 28, 2017, 10:57am UTC](https://discuss.elastic.co/t/can-not-find-the-multine-plug-in-in-my-logstash-5-1-2/73081 "2017-01-28T10:57:31Z")
Hi , I was getting a error with logstash multi line plug in not found.Could you guys please help me out? Thanks in advance. PFB Error- reason-"Couldn't find any filter plugin named 'multiline'. Are you sure this is c… --- ## [Forwarding logs from Logstash to another external location i.e SIEM](https://discuss.elastic.co/t/forwarding-logs-from-logstash-to-another-external-location-i-e-siem/39381)
**Author:** [@Ohmygord](https://discuss.elastic.co/u/Ohmygord)\ **Replies:** 6\ **Last updated:** [January 27, 2017, 11:40pm UTC](https://discuss.elastic.co/t/forwarding-logs-from-logstash-to-another-external-location-i-e-siem/39381 "2017-01-27T23:40:41Z")
Hi, I'm currently setting up an ELK stack and will be forwarding my logs to Logstash. However I also have a requirement to send these logs top a separate SIEM. Is it possible to do this with logstash? I cant find any… --- ## [What happens when using Logstash and Filebeat in production and any of them fails?](https://discuss.elastic.co/t/what-happens-when-using-logstash-and-filebeat-in-production-and-any-of-them-fails/73027)
**Author:** [@Svetoslav\_Anastasov](https://discuss.elastic.co/u/Svetoslav_Anastasov)\ **Replies:** 1\ **Last updated:** [January 27, 2017, 11:03pm UTC](https://discuss.elastic.co/t/what-happens-when-using-logstash-and-filebeat-in-production-and-any-of-them-fails/73027 "2017-01-27T23:03:28Z")
Hello, I am currently testing the ELK. My system contains of a few machines producing logs with Filebeat on them. One machine with centralized Logstash and from there pushing logs into AWS ES. When I push this into produ… --- ## [Logstash Remove Field Output Issue](https://discuss.elastic.co/t/logstash-remove-field-output-issue/72959)
**Author:** [@kopacko](https://discuss.elastic.co/u/kopacko)\ **Replies:** 10\ **Last updated:** [January 27, 2017, 11:02pm UTC](https://discuss.elastic.co/t/logstash-remove-field-output-issue/72959 "2017-01-27T23:02:56Z")
I was cleaning up some code that I use to generate emails for alerts based on some criteria in Logstash. These alerts send an email as well as create a new document for insertion into Elasticsearch. This gives a histor… --- ## [Can I just upgrade logstash?](https://discuss.elastic.co/t/can-i-just-upgrade-logstash/73059)
**Author:** [@Jimmy\_Kang](https://discuss.elastic.co/u/Jimmy_Kang)\ **Replies:** 3\ **Last updated:** [January 27, 2017, 9:07pm UTC](https://discuss.elastic.co/t/can-i-just-upgrade-logstash/73059 "2017-01-27T21:07:52Z")
Currently I am running logstash 1.5 with kibana 1.7 and Elasticsearch 2.3. If i were to upgrade logstash to 2.3, would I need to upgrade kibana and elasticsearch? Or does it only possibly affect some of the plugins b… --- ## [Issue with remove\_field](https://discuss.elastic.co/t/issue-with-remove-field/72978)
**Author:** [@blahnik](https://discuss.elastic.co/u/blahnik)\ **Replies:** 5\ **Last updated:** [January 27, 2017, 6:32pm UTC](https://discuss.elastic.co/t/issue-with-remove-field/72978 "2017-01-27T18:32:48Z")
I am having a hard time understanding why my filter to remove some fields is not working. I grabbed a sample of the JSON from kibana { "\_index": "logstash-2017.01.27", "\_type": "json", "\_score": 1, "\_source": { … --- ## [Logstash kafka output: Failed to construct kafka producer](https://discuss.elastic.co/t/logstash-kafka-output-failed-to-construct-kafka-producer/73049)
**Author:** [@ixonok](https://discuss.elastic.co/u/ixonok)\ **Replies:** 0\ **Last updated:** [January 27, 2017, 5:26pm UTC](https://discuss.elastic.co/t/logstash-kafka-output-failed-to-construct-kafka-producer/73049 "2017-01-27T17:26:48Z")
i have a problem connection to kafka cluster with Logstash. Version: logstash 5.1.2 Plugin Version logstash-output-kafka (5.1.1) Operating System: Debian GNU/Linux 7.5 (wheezy) ruby version: ruby 1.9.3p194 (2012-04-20 … --- ## [Using cidr plugin for tagging logs](https://discuss.elastic.co/t/using-cidr-plugin-for-tagging-logs/72878)
**Author:** [@vdsm](https://discuss.elastic.co/u/vdsm)\ **Replies:** 27\ **Last updated:** [January 27, 2017, 4:36pm UTC](https://discuss.elastic.co/t/using-cidr-plugin-for-tagging-logs/72878 "2017-01-27T16:36:37Z")
Hi everyone, Before I start I have to warn you that i'm French and my English is kind limited. I just started with ELK but I want to so something like this : host A --\> logs -- LANA | … --- ## [How to set Kv filter plugin to keep spaces in value](https://discuss.elastic.co/t/how-to-set-kv-filter-plugin-to-keep-spaces-in-value/73032)
**Author:** [@Darrenliu](https://discuss.elastic.co/u/Darrenliu)\ **Replies:** 0\ **Last updated:** [January 27, 2017, 3:53pm UTC](https://discuss.elastic.co/t/how-to-set-kv-filter-plugin-to-keep-spaces-in-value/73032 "2017-01-27T15:53:54Z")
Hi everyone : I am new to learn ELK, then I'm challenges in using kv filter now. Let me make it brief, the following is one part of my dataset: s1Label=Rule cs2Label=URL Category cs3Label=Virtual System then my confi… --- ## [Question about Logstash and SNMP](https://discuss.elastic.co/t/question-about-logstash-and-snmp/72938)
**Author:** [@rcaropreso](https://discuss.elastic.co/u/rcaropreso)\ **Replies:** 5\ **Last updated:** [January 27, 2017, 3:41pm UTC](https://discuss.elastic.co/t/question-about-logstash-and-snmp/72938 "2017-01-27T15:41:59Z")
Hello, I'm new on logstash applications and I have a scenario here like this: it is necessary to perform a polling over a bunch of sensors through SNMP requests. The snmp trap is not applicable for this situation si… --- ## [How to install Logstash mongo db plug in?](https://discuss.elastic.co/t/how-to-install-logstash-mongo-db-plug-in/73013)
**Author:** [@rlv\_praveen](https://discuss.elastic.co/u/rlv_praveen)\ **Replies:** 5\ **Last updated:** [January 27, 2017, 2:03pm UTC](https://discuss.elastic.co/t/how-to-install-logstash-mongo-db-plug-in/73013 "2017-01-27T14:03:15Z")
Hi team, I have a configuration in which after parsing my data i want to send the data to mongo db. But i am getting the error as :reason=\>"Couldn't find any output plugin named \*\* 'mongodb'. Are you sure this is cor… --- ## ['T' data from logstash](https://discuss.elastic.co/t/t-data-from-logstash/73004)
**Author:** [@Alistair\_Hardy](https://discuss.elastic.co/u/Alistair_Hardy)\ **Replies:** 1\ **Last updated:** [January 27, 2017, 12:09pm UTC](https://discuss.elastic.co/t/t-data-from-logstash/73004 "2017-01-27T12:09:48Z")
Morning all I've been asked setup the elastic stack for the systems we ship. Though for now it's possibly outside the scope of what I've been asked to do, is it possible to take info from metricbeat and effectively T it … --- ## [Grok timestamp](https://discuss.elastic.co/t/grok-timestamp/72995)
**Author:** [@Maher.Glenza](https://discuss.elastic.co/u/Maher.Glenza)\ **Replies:** 14\ **Last updated:** [January 27, 2017, 11:07am UTC](https://discuss.elastic.co/t/grok-timestamp/72995 "2017-01-27T11:07:04Z")
20170118154716225 plz what is its format in grok ? --- ## [Grok debugger(online) works, but ELK stack doesn't](https://discuss.elastic.co/t/grok-debugger-online-works-but-elk-stack-doesnt/72897)
**Author:** [@Joseph](https://discuss.elastic.co/u/Joseph)\ **Replies:** 6\ **Last updated:** [January 27, 2017, 10:29am UTC](https://discuss.elastic.co/t/grok-debugger-online-works-but-elk-stack-doesnt/72897 "2017-01-27T10:29:20Z")
I am new to logstash and grok filters. My sample logs look like this {"level":"INFO", "date":"01/24/2017", "time":"14:22:05", "text":"\[Audit\]:RBAC Bundle Started"}, {"level":"INFO", "date":"01/24/2017", "time":"14:22:0… --- ## [Custom log parsing that has xml contents in ELK](https://discuss.elastic.co/t/custom-log-parsing-that-has-xml-contents-in-elk/72992)
**Author:** [@prosenjit](https://discuss.elastic.co/u/prosenjit)\ **Replies:** 2\ **Last updated:** [January 27, 2017, 9:40am UTC](https://discuss.elastic.co/t/custom-log-parsing-that-has-xml-contents-in-elk/72992 "2017-01-27T09:40:50Z")
I have a custom log file that has plain text as well as xml tags. How do i capture these in separate fields. Here is how it looks like: 1/10/2017 4:16:35 AM : Error thrown is: No Error Request sent is: \ **Author:** [@Maher.Glenza](https://discuss.elastic.co/u/Maher.Glenza)\ **Replies:** 2\ **Last updated:** [January 27, 2017, 9:08am UTC](https://discuss.elastic.co/t/logstash-input-multiple-files-output-one-index-elasticsearch-failure/72898 "2017-01-27T09:08:39Z") hi , plz i m trying to receive multiple logs then after filteriing i want to have one index containing all filtered logs but i failed this is my config file input { file { path =\>"C:\\elk\\pvin" type =\> "pvin" s… --- ## [Read xml log files which has no line breaks and everything is written in one single line](https://discuss.elastic.co/t/read-xml-log-files-which-has-no-line-breaks-and-everything-is-written-in-one-single-line/72821)
**Author:** [@elkinsights](https://discuss.elastic.co/u/elkinsights)\ **Replies:** 1\ **Last updated:** [January 27, 2017, 8:14am UTC](https://discuss.elastic.co/t/read-xml-log-files-which-has-no-line-breaks-and-everything-is-written-in-one-single-line/72821 "2017-01-27T08:14:18Z")
Hi, I have used mutline filter with other log formats and had no issues and i was able succesffuly parse the and get the output. But, I have an XML log file that the application is genration and when ever an event h… --- ## [Parsing log file with logs of different patterns](https://discuss.elastic.co/t/parsing-log-file-with-logs-of-different-patterns/72933)
**Author:** [@Prasad\_Zambare](https://discuss.elastic.co/u/Prasad_Zambare)\ **Replies:** 1\ **Last updated:** [January 27, 2017, 1:38am UTC](https://discuss.elastic.co/t/parsing-log-file-with-logs-of-different-patterns/72933 "2017-01-27T01:38:04Z")
Hi All, I am new to logstash in that matter ELK stack. A log file is having different processes logging data to it. Each process writes logs with different patterns. I want to parse this log file. Each log in this log … --- ## [Convert JSON Object to string](https://discuss.elastic.co/t/convert-json-object-to-string/72952)
**Author:** [@patrick.wall](https://discuss.elastic.co/u/patrick.wall)\ **Replies:** 2\ **Last updated:** [January 26, 2017, 9:42pm UTC](https://discuss.elastic.co/t/convert-json-object-to-string/72952 "2017-01-26T21:42:37Z")
Hi All, I am struggling to grasp something here. My use case is this - i am using logstash and elasticsearch to log every request that comes through my API. Whenever my api processes a request, I log the headers, and t… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1045) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1047)