# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=837 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 838 --- ## [Logstash filter TLD fails on .goog](https://discuss.elastic.co/t/logstash-filter-tld-fails-on-goog/115893)
**Author:** [@reswob](https://discuss.elastic.co/u/reswob)\ **Replies:** 3\ **Last updated:** [January 22, 2018, 9:19pm UTC](https://discuss.elastic.co/t/logstash-filter-tld-fails-on-goog/115893 "2018-01-22T21:19:14Z")
We are using the logstash filter plugin tld version 3.0.2. We have a number of DNS queries for domains with the tld of .goog and it fails on all of them. I was told that the tld filter has a discreet list of tlds and i… --- ## [Possible to run modules and pipelines on one server?](https://discuss.elastic.co/t/possible-to-run-modules-and-pipelines-on-one-server/108496)
**Author:** [@dison4linux](https://discuss.elastic.co/u/dison4linux)\ **Replies:** 7\ **Last updated:** [January 22, 2018, 7:01pm UTC](https://discuss.elastic.co/t/possible-to-run-modules-and-pipelines-on-one-server/108496 "2018-01-22T19:01:28Z")
I've seen several mentions to "..when 6.0 comes out you can do multiple pipelines" but can we use modules and pipelines? I successfully enabled the netflow module but now when Logstash starts up I see this message print… --- ## [Logstash on k8s](https://discuss.elastic.co/t/logstash-on-k8s/116515)
**Author:** [@konstantino](https://discuss.elastic.co/u/konstantino)\ **Replies:** 0\ **Last updated:** [January 22, 2018, 5:24pm UTC](https://discuss.elastic.co/t/logstash-on-k8s/116515 "2018-01-22T17:24:06Z")
hi everyone, this is more of a design question: what would be the best way to handle logstash as it is monitoring several different files (different logs) in a kubernetes environment? Say we want to monitor 6 differen… --- ## [Translate filter not working properly](https://discuss.elastic.co/t/translate-filter-not-working-properly/115297)
**Author:** [@cskowronnek](https://discuss.elastic.co/u/cskowronnek)\ **Replies:** 2\ **Last updated:** [January 22, 2018, 1:52pm UTC](https://discuss.elastic.co/t/translate-filter-not-working-properly/115297 "2018-01-22T13:52:02Z")
Hi, I'm using Logstash 6.1.1 to parse some Apache logs and ship them to ES. I use the translate filter in my config to translate software IDs to filenames. Logstash config: if \[request\] =~ /sdc/ and \[product\]\[softwar… --- ## [Good grok debugger needed / benefit grok vs plain regex?](https://discuss.elastic.co/t/good-grok-debugger-needed-benefit-grok-vs-plain-regex/116092)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 1\ **Last updated:** [January 22, 2018, 11:49am UTC](https://discuss.elastic.co/t/good-grok-debugger-needed-benefit-grok-vs-plain-regex/116092 "2018-01-22T11:49:19Z")
Hi all, since I love regex101.com I used plain regex in my grok match clauses until now. Unfortunately there you cannot use the patterns which are supported by grok. Are there any grok debuggers which are same as user … --- ## [How to manually install Avro input plugin offline?](https://discuss.elastic.co/t/how-to-manually-install-avro-input-plugin-offline/116455)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 0\ **Last updated:** [January 22, 2018, 9:39am UTC](https://discuss.elastic.co/t/how-to-manually-install-avro-input-plugin-offline/116455 "2018-01-22T09:39:41Z")
Hi, I am using ELK GA 6.0.0. I have messages in my Kafka topic serialized using Avro, and I would like to push them into my ES using Logstash. From this link; https://www.elastic.co/guide/en/logstash/6.0/plugins-codecs-… --- ## [Issue with logstash persistence queue for multiple pipelines](https://discuss.elastic.co/t/issue-with-logstash-persistence-queue-for-multiple-pipelines/116446)
**Author:** [@Hemanth\_Gowda](https://discuss.elastic.co/u/Hemanth_Gowda)\ **Replies:** 0\ **Last updated:** [January 22, 2018, 8:54am UTC](https://discuss.elastic.co/t/issue-with-logstash-persistence-queue-for-multiple-pipelines/116446 "2018-01-22T08:54:40Z")
Hi Team, I have 2 configuration files /conf/test.config, /conf/test1.config When i run my logstash instance with first configuration the queue is getting created and i don't want the queue data to be utilized by any … --- ## [Logstash PIpeline for Filebeat-auditd-(redis)-LS-Elastic](https://discuss.elastic.co/t/logstash-pipeline-for-filebeat-auditd-redis-ls-elastic/116440)
**Author:** [@Karthik\_Ramachandran](https://discuss.elastic.co/u/Karthik_Ramachandran)\ **Replies:** 0\ **Last updated:** [January 22, 2018, 8:15am UTC](https://discuss.elastic.co/t/logstash-pipeline-for-filebeat-auditd-redis-ls-elastic/116440 "2018-01-22T08:15:09Z")
Dear All, I'm sending Filebeat auditd data to redis, and use Logstash to send the same to ES. Why FB to Redis to LS to ES, could be a question as FB can directly sent to ES. It is because we are adding more data as part… --- ## [Logstash installation error](https://discuss.elastic.co/t/logstash-installation-error/116367)
**Author:** [@Joe3](https://discuss.elastic.co/u/Joe3)\ **Replies:** 3\ **Last updated:** [January 22, 2018, 6:37am UTC](https://discuss.elastic.co/t/logstash-installation-error/116367 "2018-01-22T06:37:26Z")
This is frst time trying to install Logstash on linux/ubuntu. I keep running into an error that logstash can't start. I have no clue what I did wrong. So I would be grateful if anyone can point me what I can do correc… --- ## [Trying to use Logstash to index from AWS cloudwatch logs and Inject to ElasticSearch](https://discuss.elastic.co/t/trying-to-use-logstash-to-index-from-aws-cloudwatch-logs-and-inject-to-elasticsearch/116416)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 0\ **Last updated:** [January 22, 2018, 1:49am UTC](https://discuss.elastic.co/t/trying-to-use-logstash-to-index-from-aws-cloudwatch-logs-and-inject-to-elasticsearch/116416 "2018-01-22T01:49:28Z")
Hello elastic team! Good day to you. May I ask about logstash index from CWL and inject to ES I have application logs stored in Amazon CloudWatch Logs. Eg: Under Test-Log-Group, App1-Log-Stream, App2-Log-Stream, App3… --- ## [Breaking Changes With Logstash 6.0](https://discuss.elastic.co/t/breaking-changes-with-logstash-6-0/116398)
**Author:** [@grantcurell](https://discuss.elastic.co/u/grantcurell)\ **Replies:** 1\ **Last updated:** [January 21, 2018, 7:04pm UTC](https://discuss.elastic.co/t/breaking-changes-with-logstash-6-0/116398 "2018-01-21T19:04:34Z")
I just managed to get my Logstash config working with v6.0... but I don't understand why it's working and when it doesn't. Original config (does not work): elasticsearch { hosts =\> \["rockserver1.lan"\] index =\> "bro… --- ## [Kafka Logstash Plugin Not Working 6.12](https://discuss.elastic.co/t/kafka-logstash-plugin-not-working-6-12/116297)
**Author:** [@grantcurell](https://discuss.elastic.co/u/grantcurell)\ **Replies:** 1\ **Last updated:** [January 21, 2018, 6:26pm UTC](https://discuss.elastic.co/t/kafka-logstash-plugin-not-working-6-12/116297 "2018-01-21T18:26:06Z")
I'm new to Kafka and Logstash, but have done a fair amount of studying and have encountered a series of errors which I can't figure out and have no idea how to fix. ... had to cut because character limit pipeline.RUBY$… --- ## [How to restart or kill logstash process when kafka output plugin failed](https://discuss.elastic.co/t/how-to-restart-or-kill-logstash-process-when-kafka-output-plugin-failed/116395)
**Author:** [@berlin](https://discuss.elastic.co/u/berlin)\ **Replies:** 0\ **Last updated:** [January 21, 2018, 5:37pm UTC](https://discuss.elastic.co/t/how-to-restart-or-kill-logstash-process-when-kafka-output-plugin-failed/116395 "2018-01-21T17:37:02Z")
I am using docker.elastic.co/logstash/logstash-oss:6.0.0 with kafka output plugin, Logstash output kafka plugin is not pushing data into kafka when one of the kafka nodes go down or get different broker id. 1/19/2018 1… --- ## [Logstash input plugin errors](https://discuss.elastic.co/t/logstash-input-plugin-errors/116382)
**Author:** [@Ondro\_Tadanai](https://discuss.elastic.co/u/Ondro_Tadanai)\ **Replies:** 0\ **Last updated:** [January 21, 2018, 12:55pm UTC](https://discuss.elastic.co/t/logstash-input-plugin-errors/116382 "2018-01-21T12:55:45Z")
Hi all, i am developing logstash input plugin which will scrape data from facebook, but i have problems testing it in logstash, with that plugin, logstash fail to run, my config: input { fb-scrape { limit =\> … --- ## [Logstash: \_dateparsefailure](https://discuss.elastic.co/t/logstash-dateparsefailure/115235)
**Author:** [@trelo](https://discuss.elastic.co/u/trelo)\ **Replies:** 3\ **Last updated:** [January 21, 2018, 12:12pm UTC](https://discuss.elastic.co/t/logstash-dateparsefailure/115235 "2018-01-21T12:12:21Z")
Hello, I'm unsuccessfully struggling with \_dateparsefailure issue. While debugging everything works fine, but in Elastic every document comes with \_dateparsefailure tag. Here is piece of conf: ..... ruby { cod… --- ## [Logstash Filter error](https://discuss.elastic.co/t/logstash-filter-error/115655)
**Author:** [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Replies:** 2\ **Last updated:** [January 21, 2018, 12:00pm UTC](https://discuss.elastic.co/t/logstash-filter-error/115655 "2018-01-21T12:00:31Z")
My Log 198.0.200.105 - - \[14/Jan/2014:09:36:50 -0800\] "GET /svds.com/rockandroll HTTP/1.1" 301 241 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10\_9\_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537… --- ## [Logstash codecs (multiline,json) uncomplete messages from file lines](https://discuss.elastic.co/t/logstash-codecs-multiline-json-uncomplete-messages-from-file-lines/115286)
**Author:** [@Ged](https://discuss.elastic.co/u/Ged)\ **Replies:** 3\ **Last updated:** [January 20, 2018, 5:18pm UTC](https://discuss.elastic.co/t/logstash-codecs-multiline-json-uncomplete-messages-from-file-lines/115286 "2018-01-20T17:18:33Z")
Hi, Elasticsearch 5.5.0 Logstash 5.5.0 logstash-codec-multiline 3.0.5 logstash-codec-json 3.0.3. I'm reading file (NFS) containing JSON lines using above codecs and from time to time receiving only part of line as m… --- ## [New Install ELK 6.1 with Filebeat 6.1 for Tomcat Catalina Logs](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559)
**Author:** [@vvsvinu](https://discuss.elastic.co/u/vvsvinu)\ **Replies:** 13\ **Last updated:** [January 20, 2018, 10:56am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559 "2018-01-20T10:56:27Z")
Hai, I am a newbie to ELK Stack. I have tried to configure the ELK Stack in a test machine in our LAB environment and was not able to succeed in parsing Apache Tomcat Logs. It would be great help if any one could assist… --- ## [Logstash "WMI" input plugin Error ==\> Error registering plugin](https://discuss.elastic.co/t/logstash-wmi-input-plugin-error-error-registering-plugin/116333)
**Author:** [@fsaas](https://discuss.elastic.co/u/fsaas)\ **Replies:** 1\ **Last updated:** [January 20, 2018, 9:40am UTC](https://discuss.elastic.co/t/logstash-wmi-input-plugin-error-error-registering-plugin/116333 "2018-01-20T09:40:45Z")
Hello What I'm trying to do I used to use ELK in a ubuntu Virtual Machine. but I need to use "WMI" input plugin that I can get windows OS information. So I decided to let logstash installed in windwos, and set logst… --- ## [How to define custom grock to get only 3 levels deep url](https://discuss.elastic.co/t/how-to-define-custom-grock-to-get-only-3-levels-deep-url/116317)
**Author:** [@gokhalep](https://discuss.elastic.co/u/gokhalep)\ **Replies:** 0\ **Last updated:** [January 20, 2018, 1:57am UTC](https://discuss.elastic.co/t/how-to-define-custom-grock-to-get-only-3-levels-deep-url/116317 "2018-01-20T01:57:04Z")
Hello I am new to the grocks . I am using a %{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:resp… --- ## [Please Help, I can't get logstash 6.1.1 to listen on port on windows](https://discuss.elastic.co/t/please-help-i-cant-get-logstash-6-1-1-to-listen-on-port-on-windows/116306)
**Author:** [@treynolds](https://discuss.elastic.co/u/treynolds)\ **Replies:** 0\ **Last updated:** [January 19, 2018, 9:20pm UTC](https://discuss.elastic.co/t/please-help-i-cant-get-logstash-6-1-1-to-listen-on-port-on-windows/116306 "2018-01-19T21:20:07Z")
I believe I have followed all the setup procedures/configurations mentioned but no matter what I try there is nothing listening on the specified port (5044). I have tried to follow everything I can find in this forum, b… --- ## [Want to use logtimestamp as elastic timestamp](https://discuss.elastic.co/t/want-to-use-logtimestamp-as-elastic-timestamp/116269)
**Author:** [@aviationfan](https://discuss.elastic.co/u/aviationfan)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 6:22pm UTC](https://discuss.elastic.co/t/want-to-use-logtimestamp-as-elastic-timestamp/116269 "2018-01-19T18:22:01Z")
I am new to ELK and am experimenting to learn how to use the tools. I have a config file for logstash setup and have gotten data into Elastic. I noticed that it ignores the data and time in the log entries and uses the c… --- ## [Logstash-Filter-Elasticsearch installation](https://discuss.elastic.co/t/logstash-filter-elasticsearch-installation/116276)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 0\ **Last updated:** [January 19, 2018, 4:23pm UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-installation/116276 "2018-01-19T16:23:32Z")
hi, on Centos / I'm trying to install plugin Logstash-Filter-Elasticsearch. Steps already done: installed ruby using yum (ruby 2.0) installed rubygems downloaded from official site. when I try to install Logstash-Fi… --- ## [Mutate filter doesn't work with error.message (heartbeat) field](https://discuss.elastic.co/t/mutate-filter-doesnt-work-with-error-message-heartbeat-field/116261)
**Author:** [@Bozzerreaneaux](https://discuss.elastic.co/u/Bozzerreaneaux)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 3:20pm UTC](https://discuss.elastic.co/t/mutate-filter-doesnt-work-with-error-message-heartbeat-field/116261 "2018-01-19T15:20:53Z")
Hello, trying to do a quick filter to replace a given heartbeat field value based on the value of other field but it seems when the field has a dot in the name t error.message, the filter doesn't work: works: if \[ip\] … --- ## [Netflow influxdb - parse error](https://discuss.elastic.co/t/netflow-influxdb-parse-error/116046)
**Author:** [@Sviks](https://discuss.elastic.co/u/Sviks)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 2:47pm UTC](https://discuss.elastic.co/t/netflow-influxdb-parse-error/116046 "2018-01-19T14:47:36Z")
I get this error \[2018-01-18T12:23:11,400\]\[WARN \]\[logstash.outputs.influxdb\] 28.999Z"} 1516278190000': invalid boolean\\nunable to parse 'logstash,host=xxx.xxx.xxx.xxx netflow={"ingressVRFID"=\\u003e1610612736, "output\_s… --- ## [TCP input: no cipher suites in common](https://discuss.elastic.co/t/tcp-input-no-cipher-suites-in-common/116253)
**Author:** [@bakurin](https://discuss.elastic.co/u/bakurin)\ **Replies:** 0\ **Last updated:** [January 19, 2018, 2:27pm UTC](https://discuss.elastic.co/t/tcp-input-no-cipher-suites-in-common/116253 "2018-01-19T14:27:58Z")
I'm trying to forward docker log to logstash using syslog docker driver and logstash tcp (ssl with self-signed certificate) input: tcp { port =\> 5050 type =\> "syslog" mode =\> "server" ssl\_cert =\> "/path/to/cert" ss… --- ## [Filebeat time in log writed randomly](https://discuss.elastic.co/t/filebeat-time-in-log-writed-randomly/116209)
**Author:** [@skylinetux](https://discuss.elastic.co/u/skylinetux)\ **Replies:** 0\ **Last updated:** [January 19, 2018, 10:26am UTC](https://discuss.elastic.co/t/filebeat-time-in-log-writed-randomly/116209 "2018-01-19T10:26:32Z")
Hi! Logstash write a log into file randomly. For example log file: \[2018-01-10 10:59:09,792\] \[DEBUG\] \[http-bio-3010-exec-102\] etc \[2018-01-10 10:58:49,440\] \[DEBUG\] \[http-bio-3010-exec-104\] etc \[2018-01-10 10:59:39,165\]… --- ## [Install plugin using gem. GemNotFound](https://discuss.elastic.co/t/install-plugin-using-gem-gemnotfound/116123)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 1\ **Last updated:** [January 19, 2018, 10:09am UTC](https://discuss.elastic.co/t/install-plugin-using-gem-gemnotfound/116123 "2018-01-19T10:09:30Z")
Hi, I'm using logstash 5.6.5 on Centos7 and I need to use elasticsearch filter so I'm trying to install this plugin. If I try the command l logstash-plugin install elasticsearch-filter I receive the warning to chec… --- ## [Split array in Logstash](https://discuss.elastic.co/t/split-array-in-logstash/116197)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 0\ **Last updated:** [January 19, 2018, 9:26am UTC](https://discuss.elastic.co/t/split-array-in-logstash/116197 "2018-01-19T09:26:09Z")
I have data coming in, in following format via Logstash { test =\> \[ \[0\] { "data"= "xyz" "field2"=\> "1234" } \[1\] { "data"= "xyz" "field2"=\> "1234" } \[2\] { "data"= "xyz" "field2"=\> "1234" } I want to split my fi… --- ## [Unable to start Logstash via systemd on RHEL - can't find Elapsed Plugin](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828)
**Author:** [@stevedearl](https://discuss.elastic.co/u/stevedearl)\ **Replies:** 8\ **Last updated:** [January 19, 2018, 9:09am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-systemd-on-rhel-cant-find-elapsed-plugin/114828 "2018-01-19T09:09:34Z")
Hi ELK-bods, I'm going through my first install of ELK on Linux/RHEL and am not able to run Logstash via systemd as it seems it is not able to find/read the 'logstash-input-elapsed' plugin. This plugin has been instal… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=836) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=838)