# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=739 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 740 --- ## [Is it ok to run Logstash in Docker for Production?](https://discuss.elastic.co/t/is-it-ok-to-run-logstash-in-docker-for-production/135998)
**Author:** [@jbrack](https://discuss.elastic.co/u/jbrack)\ **Replies:** 2\ **Last updated:** [June 18, 2018, 4:03pm UTC](https://discuss.elastic.co/t/is-it-ok-to-run-logstash-in-docker-for-production/135998 "2018-06-18T16:03:15Z")
I don't see much writing about this out on the web. Is it recommended to run Logstash in Docker for a production environment? What are some things to look out for? I am looking to run multiple logstash services in Kubern… --- ## [Beautifier for Logstash .conf files](https://discuss.elastic.co/t/beautifier-for-logstash-conf-files/136335)
**Author:** [@james\_mtl](https://discuss.elastic.co/u/james_mtl)\ **Replies:** 0\ **Last updated:** [June 18, 2018, 3:53pm UTC](https://discuss.elastic.co/t/beautifier-for-logstash-conf-files/136335 "2018-06-18T15:53:52Z")
Is there a logstash .conf beautifier utility online? --- ## [Logstash won't forward to Elasticsearch via output plugin](https://discuss.elastic.co/t/logstash-wont-forward-to-elasticsearch-via-output-plugin/136133)
**Author:** [@james\_mtl](https://discuss.elastic.co/u/james_mtl)\ **Replies:** 2\ **Last updated:** [June 18, 2018, 3:52pm UTC](https://discuss.elastic.co/t/logstash-wont-forward-to-elasticsearch-via-output-plugin/136133 "2018-06-18T15:52:33Z")
I am trying to modify an existing ELK configuration. Logs are being sent to Logstash via syslog and then forwarded to an Elasticsearch cluster. I am trying to switch our Elasticsearch endpoints to AWS' ES service. I hav… --- ## [Kibana showing two values instead of one](https://discuss.elastic.co/t/kibana-showing-two-values-instead-of-one/136187)
**Author:** [@Harsh\_Sharma](https://discuss.elastic.co/u/Harsh_Sharma)\ **Replies:** 4\ **Last updated:** [June 18, 2018, 3:38pm UTC](https://discuss.elastic.co/t/kibana-showing-two-values-instead-of-one/136187 "2018-06-18T15:38:14Z")
Hi, My Grok parsed successfully while I'm creating an index in Kibana I'm getting value twice for a field name. Please see the attachment. \[11%20AM\] Any suggestion how to fix it? Thanks. --- ## [Logstash Zabbix Output Plugin Doesn't Use PSKs](https://discuss.elastic.co/t/logstash-zabbix-output-plugin-doesnt-use-psks/136141)
**Author:** [@adamkasztenny](https://discuss.elastic.co/u/adamkasztenny)\ **Replies:** 2\ **Last updated:** [June 18, 2018, 3:23pm UTC](https://discuss.elastic.co/t/logstash-zabbix-output-plugin-doesnt-use-psks/136141 "2018-06-18T15:23:16Z")
We were debugging the code for the logstash-output-zabbix plugin, and found it doesn't use zabbix\_sender to communicate with Zabbix. Our Zabbix instance did not authorize the plugin to communicate with it. zabbix\_sender… --- ## [Lostash Plugins Failing with some Dependency error](https://discuss.elastic.co/t/lostash-plugins-failing-with-some-dependency-error/136258)
**Author:** [@prakash1243](https://discuss.elastic.co/u/prakash1243)\ **Replies:** 2\ **Last updated:** [June 18, 2018, 8:28am UTC](https://discuss.elastic.co/t/lostash-plugins-failing-with-some-dependency-error/136258 "2018-06-18T08:28:43Z")
Hello- I cloned many logstash plugins and executed the specs, but I get the jars dependency issues. An error occurred while loading ./spec/unit/outputs/elasticsearch\_spec.rb. Failure/Error: require "logstash/devutils/r… --- ## [How to efficiently parse log files in logstash?](https://discuss.elastic.co/t/how-to-efficiently-parse-log-files-in-logstash/135361)
**Author:** [@clash\_of\_clanss](https://discuss.elastic.co/u/clash_of_clanss)\ **Replies:** 2\ **Last updated:** [June 18, 2018, 8:00am UTC](https://discuss.elastic.co/t/how-to-efficiently-parse-log-files-in-logstash/135361 "2018-06-18T08:00:55Z")
Hi, I have to parse huge amount of log files in logstash. I stored all log files in one directory with .gz format. when I try to extract those logs the size becomes very high. Let's say the log file size is 50MB when it… --- ## [Logstash multiline codec plugin](https://discuss.elastic.co/t/logstash-multiline-codec-plugin/136236)
**Author:** [@bobbytan](https://discuss.elastic.co/u/bobbytan)\ **Replies:** 3\ **Last updated:** [June 18, 2018, 7:10am UTC](https://discuss.elastic.co/t/logstash-multiline-codec-plugin/136236 "2018-06-18T07:10:40Z")
hello, Im trying to use the multiline codec plugin to merge input log that contains several lines into one line within a textfile. config file: input { file { path =\> "C:\\Users\\bob\\Downloads\\data\\error\_test.txt" st… --- ## [Why am I geting error \[logstash.outputs.elasticsearch\] Action?](https://discuss.elastic.co/t/why-am-i-geting-error-logstash-outputs-elasticsearch-action/135342)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 4\ **Last updated:** [June 18, 2018, 5:40am UTC](https://discuss.elastic.co/t/why-am-i-geting-error-logstash-outputs-elasticsearch-action/135342 "2018-06-18T05:40:36Z")
I am using ELK GA 5.0.0. I am using Logstash to fetch data from Kafka topic. My config i like below; input { kafka{ group\_id =\> "mygroup" topics =\> \["mytopic"\] bootstrap\_servers =\> "192.168.0.1:9092,192.168.0.2:9… --- ## [Not able to start logstash 6.3.0](https://discuss.elastic.co/t/not-able-to-start-logstash-6-3-0/136157)
**Author:** [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Replies:** 2\ **Last updated:** [June 18, 2018, 5:00am UTC](https://discuss.elastic.co/t/not-able-to-start-logstash-6-3-0/136157 "2018-06-18T05:00:00Z")
Hi All, I recently downloaded logstash 6.3.0 and i am seeing the below issue on trying to start logstash, \[2018-06-16T06:27:43,664\]\[ERROR\]\[logstash.pipeline \] Pipeline aborted due to error {:pipeline\_id=\>"main",… --- ## [Updating the @timestamp is not working](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192)
**Author:** [@Pradana](https://discuss.elastic.co/u/Pradana)\ **Replies:** 18\ **Last updated:** [June 17, 2018, 6:26pm UTC](https://discuss.elastic.co/t/updating-the-timestamp-is-not-working/136192 "2018-06-17T18:26:34Z")
I am trying to updating the timestamp using below : date { match =\> \[ "smsdate", "dd/MMM/yyyy:HH:mm:ss Z" \] target =\> \["@timestamp"\] } but in kibana still showing the current date. How to make it right ? my objectiv… --- ## [Logstash takes a long time to load](https://discuss.elastic.co/t/logstash-takes-a-long-time-to-load/136216)
**Author:** [@shiranbi07](https://discuss.elastic.co/u/shiranbi07)\ **Replies:** 1\ **Last updated:** [June 17, 2018, 5:23pm UTC](https://discuss.elastic.co/t/logstash-takes-a-long-time-to-load/136216 "2018-06-17T17:23:20Z")
From first line in main function until printing of "Sending Logstash's logs to.." takes 35 seconds on: windows 7 java version "1.8.0\_161" Java(TM) SE Runtime Environment (build 1.8.0\_161-b12) Java HotSpot(TM) 64-Bit … --- ## [Logstash in Docker Container to ES Cloud](https://discuss.elastic.co/t/logstash-in-docker-container-to-es-cloud/135060)
**Author:** [@jenyphur](https://discuss.elastic.co/u/jenyphur)\ **Replies:** 3\ **Last updated:** [June 16, 2018, 10:15pm UTC](https://discuss.elastic.co/t/logstash-in-docker-container-to-es-cloud/135060 "2018-06-16T22:15:10Z")
Hi there, new so please go easy. I have logstash deployed in docker container and need it to send logs to our ElasticSearch in the Cloud account. In the logstash config, I see where you can specify the output as such: … --- ## [What are people using for orchestration & scheduling?](https://discuss.elastic.co/t/what-are-people-using-for-orchestration-scheduling/134980)
**Author:** [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\ **Replies:** 1\ **Last updated:** [June 16, 2018, 1:36pm UTC](https://discuss.elastic.co/t/what-are-people-using-for-orchestration-scheduling/134980 "2018-06-16T13:36:52Z")
Hello Everyone, Just wanted to see what people are using out of interest to schedule and manage orchestration of logstash configurations? Yes, I know logstash has a schedule future with cron like syntax but wanted to m… --- ## [Logstash Secret Management](https://discuss.elastic.co/t/logstash-secret-management/134981)
**Author:** [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\ **Replies:** 1\ **Last updated:** [June 16, 2018, 1:36pm UTC](https://discuss.elastic.co/t/logstash-secret-management/134981 "2018-06-16T13:36:02Z")
Hi, In 6.2 it was great to see the keystore addition - but wondering is it possible to use existing secret stores such as Vault? I am also wondering how keystore can be exposed over to EFS or something to ensure your ke… --- ## [Elasticsearch not getting data from logstash](https://discuss.elastic.co/t/elasticsearch-not-getting-data-from-logstash/136166)
**Author:** [@rusty\_22](https://discuss.elastic.co/u/rusty_22)\ **Replies:** 8\ **Last updated:** [June 16, 2018, 12:40pm UTC](https://discuss.elastic.co/t/elasticsearch-not-getting-data-from-logstash/136166 "2018-06-16T12:40:25Z")
07:37:08.569 \[\[main\]\>worker8\] ERROR logstash.outputs.elasticsearch - Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error\_message=\>"Elasticsearch Unreachable: \[h… --- ## [Ruby Scripts in Logstash CONF file](https://discuss.elastic.co/t/ruby-scripts-in-logstash-conf-file/135322)
**Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 6\ **Last updated:** [June 16, 2018, 11:35am UTC](https://discuss.elastic.co/t/ruby-scripts-in-logstash-conf-file/135322 "2018-06-16T11:35:43Z")
Hi- I have did the logs transformation using the Ruby code on Logstash pipelines and running the CONF file from a Virtual machine. The ruby code is getting bigger, and found its risky to have the code on a Virtual machi… --- ## [How can i get "Elasticsearch Filter Plugin" result in json list in C#](https://discuss.elastic.co/t/how-can-i-get-elasticsearch-filter-plugin-result-in-json-list-in-c/133788)
**Author:** [@yaser](https://discuss.elastic.co/u/yaser)\ **Replies:** 3\ **Last updated:** [June 16, 2018, 7:02am UTC](https://discuss.elastic.co/t/how-can-i-get-elasticsearch-filter-plugin-result-in-json-list-in-c/133788 "2018-06-16T07:02:26Z")
hi every one , i have a query like this in my logstash "elasticsearch plugin " : here is the logstash config file :: if\[type\] == "mytype" { elasticsearch { hosts =\> \["192.168.1.1:9200"\] index =\> "my\_index… --- ## [UNEXPECTED POOL ERROR but logstash is pushing to Elastic perfectly fine?](https://discuss.elastic.co/t/unexpected-pool-error-but-logstash-is-pushing-to-elastic-perfectly-fine/136144)
**Author:** [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\ **Replies:** 1\ **Last updated:** [June 15, 2018, 9:41pm UTC](https://discuss.elastic.co/t/unexpected-pool-error-but-logstash-is-pushing-to-elastic-perfectly-fine/136144 "2018-06-15T21:41:17Z")
Hi, running Logstash 6.2.4 in docker container with DC/OS marathon. I have logstash ingesting logs from Kafka and pushing to my cluster perfectly fine!!! But in the logs I see these every few seconds and it also seems t… --- ## [Rejecting mapping update to \[\] as the final mapping would have more than 1 type: \[\_doc, doc\]](https://discuss.elastic.co/t/rejecting-mapping-update-to-as-the-final-mapping-would-have-more-than-1-type-doc-doc/135985)
**Author:** [@dmongit](https://discuss.elastic.co/u/dmongit)\ **Replies:** 3\ **Last updated:** [June 15, 2018, 8:03pm UTC](https://discuss.elastic.co/t/rejecting-mapping-update-to-as-the-final-mapping-would-have-more-than-1-type-doc-doc/135985 "2018-06-15T20:03:15Z")
I have my own mapping that I dump to my running localhost:9200/abc. In my elasticsearch.yml I have action.auto\_create\_index: false I have a docker-compose that spins up elasticsearch, kibana and logstash. Logstash inp… --- ## [How to handle events rejected by kafka output plugin](https://discuss.elastic.co/t/how-to-handle-events-rejected-by-kafka-output-plugin/136130)
**Author:** [@wfhartford](https://discuss.elastic.co/u/wfhartford)\ **Replies:** 1\ **Last updated:** [June 15, 2018, 7:34pm UTC](https://discuss.elastic.co/t/how-to-handle-events-rejected-by-kafka-output-plugin/136130 "2018-06-15T19:34:25Z")
I recently encountered a situation in which my Kafka broker was unable to accept an event from Logstash (the event was too large). The default configuration of the Kafka output plugin is to retry forever. Unfortunately, … --- ## [Aggregate filter plugin throwing undefined method error](https://discuss.elastic.co/t/aggregate-filter-plugin-throwing-undefined-method-error/136027)
**Author:** [@elasticuser1](https://discuss.elastic.co/u/elasticuser1)\ **Replies:** 3\ **Last updated:** [June 15, 2018, 6:07pm UTC](https://discuss.elastic.co/t/aggregate-filter-plugin-throwing-undefined-method-error/136027 "2018-06-15T18:07:46Z")
Hi, I am trying to get the difference between startDt and endDt but I am getting the below error - Aggregate exception occurred {:error=\>#\ **Author:** [@Andras\_Klambauer](https://discuss.elastic.co/u/Andras_Klambauer)\ **Replies:** 14\ **Last updated:** [June 15, 2018, 4:22pm UTC](https://discuss.elastic.co/t/erro-after-install/135923 "2018-06-15T16:22:23Z") I just installed the logstash next to graylog and I got this erros all the time: 2018-06-14T13:33:05,268\]\[INFO \]\[logstash.runner \] Starting Logstash {"logstash.version"=\>"6.3.0"} \[2018-06-14T13:33:06,013\]\[ERRO… --- ## [Parse firewall logs collected by logstash through syslog](https://discuss.elastic.co/t/parse-firewall-logs-collected-by-logstash-through-syslog/132812)
**Author:** [@Dora](https://discuss.elastic.co/u/Dora)\ **Replies:** 11\ **Last updated:** [June 15, 2018, 2:55pm UTC](https://discuss.elastic.co/t/parse-firewall-logs-collected-by-logstash-through-syslog/132812 "2018-06-15T14:55:26Z")
Hello, I am using ELK (version 6.2.4), and I would like to collect firewall logs (Fortigate) from another SIEM, so I followed the following steps: I configured the other SIEM to forward these logs to ELK via the UDP p… --- ## [Logstash parsing issue](https://discuss.elastic.co/t/logstash-parsing-issue/136097)
**Author:** [@motts](https://discuss.elastic.co/u/motts)\ **Replies:** 2\ **Last updated:** [June 15, 2018, 2:43pm UTC](https://discuss.elastic.co/t/logstash-parsing-issue/136097 "2018-06-15T14:43:20Z")
I am currently running into some parsing issues within LogStash. Here is my conf file. filter { if \[type\] == "srr" { kv { trim =\> "\\{\\}\\"\\," trimkey =\> "\\{\\}\\"\\," … --- ## [X-pack logstash authentication issues (response code 401)](https://discuss.elastic.co/t/x-pack-logstash-authentication-issues-response-code-401/135987)
**Author:** [@snewman](https://discuss.elastic.co/u/snewman)\ **Replies:** 4\ **Last updated:** [June 15, 2018, 2:25pm UTC](https://discuss.elastic.co/t/x-pack-logstash-authentication-issues-response-code-401/135987 "2018-06-15T14:25:18Z")
I've read a number of forum posts on this subject, but none seem to hold the answer for me. I have x-pack installed on elasticsearch and kibana, and I have them both running. I can start logstash and see it appear in the… --- ## [Create new field with metadata content](https://discuss.elastic.co/t/create-new-field-with-metadata-content/136052)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 4\ **Last updated:** [June 15, 2018, 2:05pm UTC](https://discuss.elastic.co/t/create-new-field-with-metadata-content/136052 "2018-06-15T14:05:31Z")
Hey guys, Thanks to my pipeline, I can extract data and store it to a metadata field : if (\[ID2\] == "000003") { grok { match =\> { "DESCRIPTION" =\> \[ "%{DATA:TEST}/%{WORD:NOM\_BATCH}-%{BASE16NUM:DATE\_BAT… --- ## [Help with parsing JSON file](https://discuss.elastic.co/t/help-with-parsing-json-file/136065)
**Author:** [@JohnD76](https://discuss.elastic.co/u/JohnD76)\ **Replies:** 1\ **Last updated:** [June 15, 2018, 1:42pm UTC](https://discuss.elastic.co/t/help-with-parsing-json-file/136065 "2018-06-15T13:42:30Z")
Hi, I am still learning how to use Logstash and need some assistance in how to reference fields inside a field and also inside an array please. Using the example below, how do I reference the MONITORING.responseStatus … --- ## [Add\_field never replace my field](https://discuss.elastic.co/t/add-field-never-replace-my-field/136074)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 3\ **Last updated:** [June 15, 2018, 1:39pm UTC](https://discuss.elastic.co/t/add-field-never-replace-my-field/136074 "2018-06-15T13:39:20Z")
Hello I would to like to know why my "add\_field" option doesn"t replace my "NEWFIELD" field ? : if (\[ID2\] == "000003") { grok { match =\> { "DESCRIPTION" =\> \[ "%{DATA:TEST}/%{WORD:NOM\_BATCH}-%{BASE16NU… --- ## [Install offline plugins downloaded from Github](https://discuss.elastic.co/t/install-offline-plugins-downloaded-from-github/136101)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 1\ **Last updated:** [June 15, 2018, 1:38pm UTC](https://discuss.elastic.co/t/install-offline-plugins-downloaded-from-github/136101 "2018-06-15T13:38:09Z")
Hello everyone, I try to install a plugin found on github which may solve some of my problems but i'm new on ELK and i'm unable to install this plugin : I have no internet connection on my ELK server so that's why I … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=738) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=740)