# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=557 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 558 --- ## [Logstash 6.5.0 stops shipping logs to ES](https://discuss.elastic.co/t/logstash-6-5-0-stops-shipping-logs-to-es/175477)
**Author:** [@stash2logs](https://discuss.elastic.co/u/stash2logs)\ **Replies:** 22\ **Last updated:** [April 27, 2019, 2:15pm UTC](https://discuss.elastic.co/t/logstash-6-5-0-stops-shipping-logs-to-es/175477 "2019-04-27T14:15:07Z")
We are running into an issue where the Logstash service is running but the logs are not being shipped to ES. This issue is quite strange as it is random, we would start the logstash instance and after a few hours or som… --- ## [Mutate filter Join question](https://discuss.elastic.co/t/mutate-filter-join-question/178705)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 1\ **Last updated:** [April 27, 2019, 12:21pm UTC](https://discuss.elastic.co/t/mutate-filter-join-question/178705 "2019-04-27T12:21:58Z")
https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-join Does this only work for arrays of strings? What would happen is I tried to Join an array of objects? or an array o… --- ## [How to extract the data?](https://discuss.elastic.co/t/how-to-extract-the-data/178148)
**Author:** [@yuva\_rajan](https://discuss.elastic.co/u/yuva_rajan)\ **Replies:** 7\ **Last updated:** [April 27, 2019, 10:47am UTC](https://discuss.elastic.co/t/how-to-extract-the-data/178148 "2019-04-27T10:47:34Z")
Below is the data in json :-1: { "\_source": { "notification": { ""eventDescription": "\[Monitored alert priority \>=4\] \[\*\*\*\] \[https://\*\*\*\] \[The application powershell.exe is executing a fileless script or command.\] \[In… --- ## [Logstash Lumberjack Cert](https://discuss.elastic.co/t/logstash-lumberjack-cert/174885)
**Author:** [@kharvey](https://discuss.elastic.co/u/kharvey)\ **Replies:** 9\ **Last updated:** [April 26, 2019, 9:58pm UTC](https://discuss.elastic.co/t/logstash-lumberjack-cert/174885 "2019-04-26T21:58:36Z")
I have a pipeline that is designed like: Filebeat -\> Logstash A -\> Logstash B -\> Elastic Search It took me a while but I figured out how to get communication up and running between Logstash A and Logstash B. But now I … --- ## [Logstash to logstash cluster with or without load balancer](https://discuss.elastic.co/t/logstash-to-logstash-cluster-with-or-without-load-balancer/178694)
**Author:** [@kharvey](https://discuss.elastic.co/u/kharvey)\ **Replies:** 1\ **Last updated:** [April 26, 2019, 9:53pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-cluster-with-or-without-load-balancer/178694 "2019-04-26T21:53:42Z")
My current setup is: Local Filebeats Logstash Output =\> Local Logstash TCP Input =\> Local Logstash TCP Output =\> Cloud Load Balancer =\> Cloud Cluster (Logstash1 TCP Input, Logstash2 TCP Input) =\> Cloud Elastic Search … --- ## [Writing my own filter plugin. What is the best way to implement the regex matching?](https://discuss.elastic.co/t/writing-my-own-filter-plugin-what-is-the-best-way-to-implement-the-regex-matching/178328)
**Author:** [@jcaballero](https://discuss.elastic.co/u/jcaballero)\ **Replies:** 1\ **Last updated:** [April 26, 2019, 8:46pm UTC](https://discuss.elastic.co/t/writing-my-own-filter-plugin-what-is-the-best-way-to-implement-the-regex-matching/178328 "2019-04-26T20:46:58Z")
I am learning how to write my own custom filter plugin. Just started. I got the basics. For my case, the input lines (or events) are like this key1 = value1 key2 = value2 ... ====== key6 = value6 key7 = value7 ... ====… --- ## [Cipher filter not working on numeric data](https://discuss.elastic.co/t/cipher-filter-not-working-on-numeric-data/177830)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 1\ **Last updated:** [April 26, 2019, 6:01pm UTC](https://discuss.elastic.co/t/cipher-filter-not-working-on-numeric-data/177830 "2019-04-26T18:01:30Z")
I am using cipher filter and I am providing numeric value as a source in the filter. Getting below error. It's working if I give the source as a string value. \*\* "error": "undefined method \`empty?' for 1:Fixnum \*\* … --- ## [Apply API to collect data from meetup.com with logstash](https://discuss.elastic.co/t/apply-api-to-collect-data-from-meetup-com-with-logstash/178581)
**Author:** [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 8:07am UTC](https://discuss.elastic.co/t/apply-api-to-collect-data-from-meetup-com-with-logstash/178581 "2019-04-26T08:07:54Z")
Hey, Can someone guide me how to collect data with logstash with the help of API. I am trying but unable to get. Can someone POST the input config. Regards --- ## [Test your Logstash configuration](https://discuss.elastic.co/t/test-your-logstash-configuration/178697)
**Author:** [@Labidi\_Ayoub](https://discuss.elastic.co/u/Labidi_Ayoub)\ **Replies:** 2\ **Last updated:** [April 26, 2019, 5:35pm UTC](https://discuss.elastic.co/t/test-your-logstash-configuration/178697 "2019-04-26T17:35:38Z")
Hi, Please, can anyone help me in this issue Test Logstash configuration : sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash -t --- ## [Logstash-plugin install logstash-output-statsd does not work](https://discuss.elastic.co/t/logstash-plugin-install-logstash-output-statsd-does-not-work/178486)
**Author:** [@Rodrigo\_Bernardo](https://discuss.elastic.co/u/Rodrigo_Bernardo)\ **Replies:** 3\ **Last updated:** [April 26, 2019, 5:14pm UTC](https://discuss.elastic.co/t/logstash-plugin-install-logstash-output-statsd-does-not-work/178486 "2019-04-26T17:14:50Z")
logstash-plugin install logstash-output-statsd does not work... because the URL https://artifacts.elastic.co/downloads/logstash-plugins/logstash-output-statsd/logstash-output-statsd-6.3.2.zip does not work... --- ## [MariaDB JDBC issue](https://discuss.elastic.co/t/mariadb-jdbc-issue/178674)
**Author:** [@cehau37](https://discuss.elastic.co/u/cehau37)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 3:30pm UTC](https://discuss.elastic.co/t/mariadb-jdbc-issue/178674 "2019-04-26T15:30:58Z")
Hi, I've been trying to find a solution for my problem without any success... I've been getting the error message below... Please help... Plugin: \"root", jdbc\_password=\>, statement=\>"… --- ## [Input elasticseach - aggregate data](https://discuss.elastic.co/t/input-elasticseach-aggregate-data/178671)
**Author:** [@Francisca\_Lima](https://discuss.elastic.co/u/Francisca_Lima)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 3:16pm UTC](https://discuss.elastic.co/t/input-elasticseach-aggregate-data/178671 "2019-04-26T15:16:29Z")
I am trying to read aggregated data of elasticsearch to do some calculations and create fields, and then add to an index of aggregated data. I tried the exec plugin, but this only present limited results. Exec plugin … --- ## [Logstash : error No configuration found in the configured sources](https://discuss.elastic.co/t/logstash-error-no-configuration-found-in-the-configured-sources/178624)
**Author:** [@medaly\_ahmed](https://discuss.elastic.co/u/medaly_ahmed)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 10:23am UTC](https://discuss.elastic.co/t/logstash-error-no-configuration-found-in-the-configured-sources/178624 "2019-04-26T10:23:37Z")
I install elasticsearch, kibana and logstash on ubuntu but when i try to load the file i find an error: bin/logstash -f medicacno\_montant.conf Sending Logstash logs to /home/ubuntu/elk/logstash-7.0.0/logs which is now … --- ## [Logstash needs tmp folder to be mounted with exec, can this be done through CFT](https://discuss.elastic.co/t/logstash-needs-tmp-folder-to-be-mounted-with-exec-can-this-be-done-through-cft/178641)
**Author:** [@Mick\_O\_Gorman](https://discuss.elastic.co/u/Mick_O_Gorman)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 12:04pm UTC](https://discuss.elastic.co/t/logstash-needs-tmp-folder-to-be-mounted-with-exec-can-this-be-done-through-cft/178641 "2019-04-26T12:04:40Z")
Hi, I am launching an ec2 through cloudformation template and installing logstash on it, one thing i have found is that unless i run sudo mount -o remount,exec /tmp i cannot get logstash working..(https://github.com/ela… --- ## [How to parse multipart json in logstash](https://discuss.elastic.co/t/how-to-parse-multipart-json-in-logstash/178400)
**Author:** [@kunal\_p](https://discuss.elastic.co/u/kunal_p)\ **Replies:** 2\ **Last updated:** [April 26, 2019, 6:06am UTC](https://discuss.elastic.co/t/how-to-parse-multipart-json-in-logstash/178400 "2019-04-26T06:06:00Z")
We are having sample json logs as below { "responseBody": { "Info": \[ { "supplierBookReferences": \[ { "field1": "NA", "field2": "NA", "field3": "", "field4": "NA", "field5": "", "field6": "" }\]… --- ## [Failed to execute action logstash](https://discuss.elastic.co/t/failed-to-execute-action-logstash/178629)
**Author:** [@Aleksey\_Rumjncev](https://discuss.elastic.co/u/Aleksey_Rumjncev)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 11:11am UTC](https://discuss.elastic.co/t/failed-to-execute-action-logstash/178629 "2019-04-26T11:11:37Z")
Hello, what is wrong in my config? filter { grok { match =\> \[ "message", "%{IP:client} - - %{GREEDYATA:data} "%{WORD:http\_type} %{NOTSPACE} HTTP/%{NUMBER:httpversion}" %{NUMBER:response}" \] overwrite =\> \[ "message" \] … --- ## [Date parsing with am:pm?](https://discuss.elastic.co/t/date-parsing-with-am-pm/178626)
**Author:** [@Jeongwoo\_Park](https://discuss.elastic.co/u/Jeongwoo_Park)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 10:40am UTC](https://discuss.elastic.co/t/date-parsing-with-am-pm/178626 "2019-04-26T10:40:56Z")
The date in my logs looks like that: 6월 15, 2018 4:10:20 오후 I tried to parse it like that: grok { match =\> {"message" =\> "%{NUMBER:mon\_num}%{NOTSPACE:month} %{NUMBER:day}, %{YEAR:year} %{TIME:time} %{NOTSPACE:hour\_cl… --- ## [Scale Logstash on Openshift](https://discuss.elastic.co/t/scale-logstash-on-openshift/178594)
**Author:** [@Emanuel9](https://discuss.elastic.co/u/Emanuel9)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 8:55am UTC](https://discuss.elastic.co/t/scale-logstash-on-openshift/178594 "2019-04-26T08:55:03Z")
Hello, I am running Logstash 6.4.3 on Openshift with only 1 Pod atm. I actually added quite a few servers within the last couple of days which all send logs via Filebeat to my Logstash instance. Today I realized that m… --- ## [Logstash is creating weird files](https://discuss.elastic.co/t/logstash-is-creating-weird-files/178620)
**Author:** [@keysersozee](https://discuss.elastic.co/u/keysersozee)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 10:13am UTC](https://discuss.elastic.co/t/logstash-is-creating-weird-files/178620 "2019-04-26T10:13:57Z")
Hello, I'm using Logstash 6.2.2 to ingest data coming from another system and then to send them to a REST endpoint via the HTTP output plugin. When I have a large volume to handle (about 1 000 000 events in few minutes… --- ## [Fault tolerance not working when multiple hosts](https://discuss.elastic.co/t/fault-tolerance-not-working-when-multiple-hosts/178619)
**Author:** [@Igorr](https://discuss.elastic.co/u/Igorr)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 10:11am UTC](https://discuss.elastic.co/t/fault-tolerance-not-working-when-multiple-hosts/178619 "2019-04-26T10:11:24Z")
Another host has been added to the logstash output to provide fault tolerance and load balancing. Output: elasticsearch { hosts =\> \["192.168.1.13:9200","192.168.1.19:9200"\] sniffing =\> "true" index =\> "stat… --- ## [How to apply a filter by "String in field"](https://discuss.elastic.co/t/how-to-apply-a-filter-by-string-in-field/178571)
**Author:** [@fabueno](https://discuss.elastic.co/u/fabueno)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 7:08am UTC](https://discuss.elastic.co/t/how-to-apply-a-filter-by-string-in-field/178571 "2019-04-26T07:08:02Z")
Hi, im using SNMP input, with "walk". When I got the OID walk result, I have a lot of field name like : ifTable.ifEntry.ifDescr.23, ifTable.ifEntry.ifDescr.24, ifTable.ifEntry.ifDescr.25, and I want to use something lik… --- ## [Logstash problem](https://discuss.elastic.co/t/logstash-problem/177636)
**Author:** [@bkrishna](https://discuss.elastic.co/u/bkrishna)\ **Replies:** 2\ **Last updated:** [April 22, 2019, 5:23am UTC](https://discuss.elastic.co/t/logstash-problem/177636 "2019-04-22T05:23:58Z")
input { tcp { port =\> 5004 codec =\> multiline { pattern =\> "^index::" negate =\> true what =\> "previous" } } } filter { if "index::" in \[message\] { grok { match =\> { "message" =\> "%{}" } } } else if "index… --- ## [Cgroup files on windows](https://discuss.elastic.co/t/cgroup-files-on-windows/178606)
**Author:** [@kaligoer](https://discuss.elastic.co/u/kaligoer)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 9:28am UTC](https://discuss.elastic.co/t/cgroup-files-on-windows/178606 "2019-04-26T09:28:34Z")
Hello, is it possible to bypass the cgroup files searching on logstash? I'm using Logstash7.0 on windows 7 and I am stuck in an infinite loop. i have this type of debug log: \[DEBUG\]\[logstash.instrument.periodicpoller… --- ## [Graphite input plugin client mode not fetching data](https://discuss.elastic.co/t/graphite-input-plugin-client-mode-not-fetching-data/174483)
**Author:** [@Jehutywong](https://discuss.elastic.co/u/Jehutywong)\ **Replies:** 1\ **Last updated:** [April 26, 2019, 8:39am UTC](https://discuss.elastic.co/t/graphite-input-plugin-client-mode-not-fetching-data/174483 "2019-04-26T08:39:24Z")
Logstash: 6.7.0 Graphite: 0.9.15 pipeline config: input { graphite { port =\> 8080 mode =\> "client" host =\> "127.0.0.1" type =\> "graphite" codec =\> graphite } } filter { } output { file { path =\> "/tmp/x" … --- ## [Writing multiline grok](https://discuss.elastic.co/t/writing-multiline-grok/178214)
**Author:** [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\ **Replies:** 7\ **Last updated:** [April 26, 2019, 5:46am UTC](https://discuss.elastic.co/t/writing-multiline-grok/178214 "2019-04-26T05:46:39Z")
HI I am trying to write grok for apache mod security logs. GET /images/new2.gif HTTP/1.1 Accept: / Referer: http://www.rrcat.gov.in/ Accept-Language: en-US User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; Trident/7.0… --- ## [Logstash Patterns help needed](https://discuss.elastic.co/t/logstash-patterns-help-needed/178487)
**Author:** [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Replies:** 2\ **Last updated:** [April 26, 2019, 4:48am UTC](https://discuss.elastic.co/t/logstash-patterns-help-needed/178487 "2019-04-26T04:48:12Z")
Hi grok-coders, I need a little bit help. I want to get some grok patterns for the following lines. They are coming via Filebeats from a CentOS 7 Server (Odoo Access Logs). How can I start with getting this message pro… --- ## [Issues consuming information from elasticsearch and sending to an http servicie via logstash http output](https://discuss.elastic.co/t/issues-consuming-information-from-elasticsearch-and-sending-to-an-http-servicie-via-logstash-http-output/178540)
**Author:** [@mgaspereyra](https://discuss.elastic.co/u/mgaspereyra)\ **Replies:** 0\ **Last updated:** [April 25, 2019, 9:18pm UTC](https://discuss.elastic.co/t/issues-consuming-information-from-elasticsearch-and-sending-to-an-http-servicie-via-logstash-http-output/178540 "2019-04-25T21:18:59Z")
Hi Everybody! I'm newbie with the ELK stack, but I found it's quite versatile and I'm love with it! Unfortunately, I was trying to replace some ecosystem based on other tool, and despite I was successful on the first s… --- ## [How to handle message with \value\\value\](https://discuss.elastic.co/t/how-to-handle-message-with-f1-value-f1-f3-value-f3/178521)
**Author:** [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Replies:** 1\ **Last updated:** [April 25, 2019, 7:47pm UTC](https://discuss.elastic.co/t/how-to-handle-message-with-f1-value-f1-f3-value-f3/178521 "2019-04-25T19:47:02Z")
We have some logs where fields are separated with html like tags, value or phrase. The problem is that some messages have different fields and/or they may be in different orders. Our SME's assistance was "Splunk just d… --- ## [How do I add new Logstash pipelines that do not combine filters with other configurations?](https://discuss.elastic.co/t/how-do-i-add-new-logstash-pipelines-that-do-not-combine-filters-with-other-configurations/178491)
**Author:** [@snowboardcto](https://discuss.elastic.co/u/snowboardcto)\ **Replies:** 1\ **Last updated:** [April 25, 2019, 7:15pm UTC](https://discuss.elastic.co/t/how-do-i-add-new-logstash-pipelines-that-do-not-combine-filters-with-other-configurations/178491 "2019-04-25T19:15:30Z")
When I add a new configuration to a Logstash pipeline it adds it to the other configurations defined in that pipeline. Is there a setting I can use to ensure all of the configurations are treated independently? I would… --- ## [Config reload didn't remove pipelines](https://discuss.elastic.co/t/config-reload-didnt-remove-pipelines/178507)
**Author:** [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Replies:** 0\ **Last updated:** [April 25, 2019, 4:28pm UTC](https://discuss.elastic.co/t/config-reload-didnt-remove-pipelines/178507 "2019-04-25T16:28:34Z")
We're running 6.7.1 and frequently add new pipelines, both to pipelines.yml and to config directories. We also update pipeline configs. The configuration reloads as expected. However, we removed some pipelines from pip… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=556) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=558)