# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=696 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 697 --- ## [Add and cut field on specific characters based on index name](https://discuss.elastic.co/t/add-and-cut-field-on-specific-characters-based-on-index-name/145584)
**Author:** [@sbienert](https://discuss.elastic.co/u/sbienert)\ **Replies:** 1\ **Last updated:** [August 25, 2018, 2:07pm UTC](https://discuss.elastic.co/t/add-and-cut-field-on-specific-characters-based-on-index-name/145584 "2018-08-25T14:07:29Z")
Hello, I want to add a field and fill it with the value of the index name but cut it after the first underscore and before the third underscore. I have an Elasticsearch in- and output and want to reindex with wildcard i… --- ## [How to prevent logging logstash logs in /var/log/messages](https://discuss.elastic.co/t/how-to-prevent-logging-logstash-logs-in-var-log-messages/145560)
**Author:** [@Bharadwaj](https://discuss.elastic.co/u/Bharadwaj)\ **Replies:** 1\ **Last updated:** [August 25, 2018, 2:04pm UTC](https://discuss.elastic.co/t/how-to-prevent-logging-logstash-logs-in-var-log-messages/145560 "2018-08-25T14:04:23Z")
Hi all logstash data is written to / var / log / messages. This causes the server capacity to continue to grow. Is there a way to prevent this data from being written to / var / log / messages? Thank you --- ## [Docker - Logstash - Error =\> "Elasticsearch Unreachable"](https://discuss.elastic.co/t/docker-logstash-error-elasticsearch-unreachable/145987)
**Author:** [@VietCong](https://discuss.elastic.co/u/VietCong)\ **Replies:** 0\ **Last updated:** [August 24, 2018, 10:10pm UTC](https://discuss.elastic.co/t/docker-logstash-error-elasticsearch-unreachable/145987 "2018-08-24T22:10:06Z")
I am trying to install ELK using Docker. I have kibana and Elasticsearch both running just fine. I just to push some logs to logstash and keep getting the error below: \[2018-08-24T22:01:24,787\]\[WARN \]\[logstash.licensech… --- ## [Configure Input Logstash more one path](https://discuss.elastic.co/t/configure-input-logstash-more-one-path/145955)
**Author:** [@marcos.vbp](https://discuss.elastic.co/u/marcos.vbp)\ **Replies:** 2\ **Last updated:** [August 24, 2018, 8:04pm UTC](https://discuss.elastic.co/t/configure-input-logstash-more-one-path/145955 "2018-08-24T20:04:32Z")
Hello, I have 10 managedServer Weblogic , and I would like add , application\_log , access.log and etc , logs for the Logstash. But I have one problem, how to make this , i try create file access.log using : input { f… --- ## [Data can not be indexed](https://discuss.elastic.co/t/data-can-not-be-indexed/137470)
**Author:** [@Yong\_Rhee](https://discuss.elastic.co/u/Yong_Rhee)\ **Replies:** 14\ **Last updated:** [August 24, 2018, 7:17pm UTC](https://discuss.elastic.co/t/data-can-not-be-indexed/137470 "2018-08-24T19:17:05Z")
I am trying to play with toy data example from the book. I can not load data because of the following problem. \[2018-06-26T11:58:16,824\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:s… --- ## [CloudFlare Log Ingest (Grok?)](https://discuss.elastic.co/t/cloudflare-log-ingest-grok/145782)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 3\ **Last updated:** [August 24, 2018, 6:48pm UTC](https://discuss.elastic.co/t/cloudflare-log-ingest-grok/145782 "2018-08-24T18:48:16Z")
Anybody ever setup Logstash to ingest CloudFlare API pulls? I'm not sure how to go about setting up grok patterns to match the output. Additionally, the returned data is in NDJson format, which I think is a double edge… --- ## [Log not show in kibana after filter in logstash](https://discuss.elastic.co/t/log-not-show-in-kibana-after-filter-in-logstash/145952)
**Author:** [@Tuyen\_Quang](https://discuss.elastic.co/u/Tuyen_Quang)\ **Replies:** 0\ **Last updated:** [August 24, 2018, 5:37pm UTC](https://discuss.elastic.co/t/log-not-show-in-kibana-after-filter-in-logstash/145952 "2018-08-24T17:37:26Z")
Hi all, I'm using filter for parsing docker log of python app. but after use grok for matching fields log no longer show in kibana dashboard. if \[type\] == "log" { grok { match =\> { "log" =\> "%{TIMESTAMP\_ISO8601:tim… --- ## [Fail to install Logstash-Forwarder on ubuntu 1804](https://discuss.elastic.co/t/fail-to-install-logstash-forwarder-on-ubuntu-1804/145820)
**Author:** [@cheng1688](https://discuss.elastic.co/u/cheng1688)\ **Replies:** 3\ **Last updated:** [August 24, 2018, 3:01pm UTC](https://discuss.elastic.co/t/fail-to-install-logstash-forwarder-on-ubuntu-1804/145820 "2018-08-24T15:01:44Z")
I have successfully installed logstash-forwarder on ubuntu 1404, and 1604. Now the same method failed on ubuntu 1804. This is the steps I do: echo 'deb http://packages.elasticsearch.org/logstashforwarder/debian stabl… --- ## [Getting an error in the logs of logstash](https://discuss.elastic.co/t/getting-an-error-in-the-logs-of-logstash/145908)
**Author:** [@kranthi\_851](https://discuss.elastic.co/u/kranthi_851)\ **Replies:** 3\ **Last updated:** [August 24, 2018, 2:38pm UTC](https://discuss.elastic.co/t/getting-an-error-in-the-logs-of-logstash/145908 "2018-08-24T14:38:19Z")
\[2018-08-24T17:34:51,894\]\[ERROR\]\[org.logstash.Logstash \] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit \[2018-08-24T17:35:11,964\]\[FATAL\]\[logstash.runner \]… --- ## [Email output with heartbeat input](https://discuss.elastic.co/t/email-output-with-heartbeat-input/145879)
**Author:** [@cowensel](https://discuss.elastic.co/u/cowensel)\ **Replies:** 1\ **Last updated:** [August 24, 2018, 2:21pm UTC](https://discuss.elastic.co/t/email-output-with-heartbeat-input/145879 "2018-08-24T14:21:32Z")
We have a heartbeat input setup on logstash. Is there a way to use the email output when the heartbeat does not generate any events to send an email. We have a scenario where logstash stops collecting new events but we … --- ## [How to drop event in logstash](https://discuss.elastic.co/t/how-to-drop-event-in-logstash/145854)
**Author:** [@ashok9177](https://discuss.elastic.co/u/ashok9177)\ **Replies:** 3\ **Last updated:** [August 24, 2018, 2:19pm UTC](https://discuss.elastic.co/t/how-to-drop-event-in-logstash/145854 "2018-08-24T14:19:22Z")
I want drop logfile mean shoudn't export to elasticsearch, like if any log message contains "monitoring" keyword i want to drop that event. can any anyone suggest me how to do that ? filter { if \[loglevel\] == "debug" … --- ## [Multiple pipeline configuration on logstash configuration](https://discuss.elastic.co/t/multiple-pipeline-configuration-on-logstash-configuration/145923)
**Author:** [@ramesh87](https://discuss.elastic.co/u/ramesh87)\ **Replies:** 1\ **Last updated:** [August 24, 2018, 2:18pm UTC](https://discuss.elastic.co/t/multiple-pipeline-configuration-on-logstash-configuration/145923 "2018-08-24T14:18:55Z")
HI , I have configured multiple pipeline configuration in logstash (version 6.3) in the pipeline.yml file. My pipeline configure is given below. pipeline.id: logstash path.config: "/etc/logstash/conf.d/logstash.conf… --- ## [Can we form a cluster for logstash?](https://discuss.elastic.co/t/can-we-form-a-cluster-for-logstash/145534)
**Author:** [@yash\_mangla](https://discuss.elastic.co/u/yash_mangla)\ **Replies:** 5\ **Last updated:** [August 24, 2018, 2:18pm UTC](https://discuss.elastic.co/t/can-we-form-a-cluster-for-logstash/145534 "2018-08-24T14:18:15Z")
Hello folks, Is it possible to make a logstash cluster to increase it's capacity in terms of logs processing, transformation and forwarding. i think we need this kind of setup for larger scale because when we correlati… --- ## [Elasticsearch input plugin index date](https://discuss.elastic.co/t/elasticsearch-input-plugin-index-date/145786)
**Author:** [@Scoyle](https://discuss.elastic.co/u/Scoyle)\ **Replies:** 3\ **Last updated:** [August 24, 2018, 2:14pm UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-index-date/145786 "2018-08-24T14:14:57Z")
Hello, im trying to get the elasticsearch input plugin to look at the previous days index. it seems to only like the wildcard or a full index name in the field. has anyone been able to change the date on the index for t… --- ## [Logstash http poller "\_http\_request\_failure" : The server failed to respond with a valid HTTP response](https://discuss.elastic.co/t/logstash-http-poller-http-request-failure-the-server-failed-to-respond-with-a-valid-http-response/145927)
**Author:** [@Harish\_Malu](https://discuss.elastic.co/u/Harish_Malu)\ **Replies:** 0\ **Last updated:** [August 24, 2018, 1:55pm UTC](https://discuss.elastic.co/t/logstash-http-poller-http-request-failure-the-server-failed-to-respond-with-a-valid-http-response/145927 "2018-08-24T13:55:38Z")
I have a http page that returns JSON data. If you enter the link in browser, it will just show data in JSON format. I used this web link in logstash http poller conf. However, execution gives error - { "@version" =\> "… --- ## [Logstash monitoring not show](https://discuss.elastic.co/t/logstash-monitoring-not-show/145272)
**Author:** [@Ilyas\_Ahsan](https://discuss.elastic.co/u/Ilyas_Ahsan)\ **Replies:** 1\ **Last updated:** [August 24, 2018, 1:42pm UTC](https://discuss.elastic.co/t/logstash-monitoring-not-show/145272 "2018-08-24T13:42:16Z")
Hello, i have issue on monitoring logstash. i have configure my logstash.yml xpack.monitoring.enabled: "true" xpack.monitoring.elasticsearch.url: "http://localhost:9200" xpack.monitoring.elasticsearch.username: "user… --- ## [System.auth.user grok results in string not object after index readonly event](https://discuss.elastic.co/t/system-auth-user-grok-results-in-string-not-object-after-index-readonly-event/145812)
**Author:** [@joer](https://discuss.elastic.co/u/joer)\ **Replies:** 1\ **Last updated:** [August 24, 2018, 1:20pm UTC](https://discuss.elastic.co/t/system-auth-user-grok-results-in-string-not-object-after-index-readonly-event/145812 "2018-08-24T13:20:25Z")
I had an event where the indices were set to read-only due to diskspace. After increasing space and setting them for read/write, the grok for syslog-auth appears to be parsing the system.auth.user wrong or somehow the i… --- ## [Logstash monitoring was not displayed on the kibana dash board](https://discuss.elastic.co/t/logstash-monitoring-was-not-displayed-on-the-kibana-dash-board/145921)
**Author:** [@ramesh87](https://discuss.elastic.co/u/ramesh87)\ **Replies:** 0\ **Last updated:** [August 24, 2018, 1:15pm UTC](https://discuss.elastic.co/t/logstash-monitoring-was-not-displayed-on-the-kibana-dash-board/145921 "2018-08-24T13:15:08Z")
HI, I am using kibana 6.3 and logstash version 6.3. When i am trying to monitoring logstash overview on kibana dash board , its not shown on the dash board. Can anyone help me how to enable the setting and where i ne… --- ## [Logstash date formatting](https://discuss.elastic.co/t/logstash-date-formatting/145767)
**Author:** [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Replies:** 2\ **Last updated:** [August 24, 2018, 9:15am UTC](https://discuss.elastic.co/t/logstash-date-formatting/145767 "2018-08-24T09:15:10Z")
Hello. I have following loglines: Aug 22 10:19:48 s1435 postfix/smtp\[17330\]: 28C1328124A4: to=jacobsd@gmail.com, relay=smtp2.google.com\[194.14.9.134\]:25, delay=1100, delays=0/1100/0.13/0.26, dsn=2.0.0, status=sent (25… --- ## [Logstash works well with an error](https://discuss.elastic.co/t/logstash-works-well-with-an-error/145841)
**Author:** [@Chunbo\_Liao](https://discuss.elastic.co/u/Chunbo_Liao)\ **Replies:** 4\ **Last updated:** [August 24, 2018, 8:41am UTC](https://discuss.elastic.co/t/logstash-works-well-with-an-error/145841 "2018-08-24T08:41:51Z")
I used the logstash with the version of 5.6.3, after a few time later. there is an error message in logs/logstash-plain.log. the error is :\[ERROR\]\[logstash.filters.ruby \] Ruby exception occurred: Regexp Interrupted. b… --- ## [If condition not working in logstash output](https://discuss.elastic.co/t/if-condition-not-working-in-logstash-output/145671)
**Author:** [@ashok9177](https://discuss.elastic.co/u/ashok9177)\ **Replies:** 8\ **Last updated:** [August 24, 2018, 6:37am UTC](https://discuss.elastic.co/t/if-condition-not-working-in-logstash-output/145671 "2018-08-24T06:37:56Z")
Here is my logstash config file, the issue is logstash not forwarding logs to all outputs, only 2 outputs are working output { if \[fields\]\[component\] == "cat" { file { path =\> "/data/cat/cat.%{+yyyy.MM.dd}" }… --- ## [How to write a Multiple input and logstash filter for hostname based on message field](https://discuss.elastic.co/t/how-to-write-a-multiple-input-and-logstash-filter-for-hostname-based-on-message-field/145843)
**Author:** [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\ **Replies:** 0\ **Last updated:** [August 24, 2018, 6:13am UTC](https://discuss.elastic.co/t/how-to-write-a-multiple-input-and-logstash-filter-for-hostname-based-on-message-field/145843 "2018-08-24T06:13:00Z")
As a newbie to logstash i would like to understand as i have two types of logs one is Linux system logs and another i have CISCO switches logs , now i'm looking forward to create the diffrent input and filter's for both. … --- ## [Logstash as a service on linux](https://discuss.elastic.co/t/logstash-as-a-service-on-linux/145128)
**Author:** [@Goyal.vikas87](https://discuss.elastic.co/u/Goyal.vikas87)\ **Replies:** 4\ **Last updated:** [August 24, 2018, 6:04am UTC](https://discuss.elastic.co/t/logstash-as-a-service-on-linux/145128 "2018-08-24T06:04:08Z")
Please help how we can execute logstash as a service on Linux (i have not installed logstash as packages and just extracted from tar file). Also please help how we can stop logstash service. If multiple config files are… --- ## [How to refer to an object's sub field in a filter](https://discuss.elastic.co/t/how-to-refer-to-an-objects-sub-field-in-a-filter/144809)
**Author:** [@Jing\_Wang](https://discuss.elastic.co/u/Jing_Wang)\ **Replies:** 2\ **Last updated:** [August 24, 2018, 1:36am UTC](https://discuss.elastic.co/t/how-to-refer-to-an-objects-sub-field-in-a-filter/144809 "2018-08-24T01:36:18Z")
I have 3 datetime string fileds, and i want to parse them by date filter and make the last one as the @timestamp of the doc date { match =\> \["syslog.timestamp", "MMM dd HH:mm:ss"\] target =\> "syslog.timestamp… --- ## [Kafka output plugin java.lang.long error](https://discuss.elastic.co/t/kafka-output-plugin-java-lang-long-error/145398)
**Author:** [@nich](https://discuss.elastic.co/u/nich)\ **Replies:** 3\ **Last updated:** [August 23, 2018, 9:53pm UTC](https://discuss.elastic.co/t/kafka-output-plugin-java-lang-long-error/145398 "2018-08-23T21:53:58Z")
When I try to use the Kafka output plugin I get this variable type error: \[ERROR\]\[logstash.outputs.kafka \] Unable to create Kafka producer from given configuration {:kafka\_error\_message=\>org.apache.kafka.common.config.C… --- ## [Mutate -\> convert filter is not working as expected](https://discuss.elastic.co/t/mutate-convert-filter-is-not-working-as-expected/145431)
**Author:** [@Malli](https://discuss.elastic.co/u/Malli)\ **Replies:** 6\ **Last updated:** [August 23, 2018, 9:38pm UTC](https://discuss.elastic.co/t/mutate-convert-filter-is-not-working-as-expected/145431 "2018-08-23T21:38:46Z")
I am trying to creating a new field using mutate-\> add\_field filter for a key coming from json data. I need to convert that filed to a integer/number. i am using the below syntax mutate { convert =\> {"timeTaken" =\> "i… --- ## [Logstash grok parser](https://discuss.elastic.co/t/logstash-grok-parser/145801)
**Author:** [@chandra0651](https://discuss.elastic.co/u/chandra0651)\ **Replies:** 0\ **Last updated:** [August 23, 2018, 8:21pm UTC](https://discuss.elastic.co/t/logstash-grok-parser/145801 "2018-08-23T20:21:01Z")
hello i am facing a problem to parse the log file and include the data into DATA field. The first grok parser with type DATA is not working where as the type GREEDYDATA is working, can any one help me out here \[2018-08-… --- ## [Logstash and filebeat configuration for ports](https://discuss.elastic.co/t/logstash-and-filebeat-configuration-for-ports/145576)
**Author:** [@rohitg](https://discuss.elastic.co/u/rohitg)\ **Replies:** 4\ **Last updated:** [August 23, 2018, 7:07pm UTC](https://discuss.elastic.co/t/logstash-and-filebeat-configuration-for-ports/145576 "2018-08-23T19:07:59Z")
I have setup one elk cluster (Elasticsearch + Logstash + Kibana) on a windows server. And installed beats(filebeat+winlogbeat) on 3 windows servers. Question about ports that need to be configure in beats and logstash.… --- ## [Logstash filter to include specific keys only](https://discuss.elastic.co/t/logstash-filter-to-include-specific-keys-only/145452)
**Author:** [@Harshad\_Velapure](https://discuss.elastic.co/u/Harshad_Velapure)\ **Replies:** 3\ **Last updated:** [August 23, 2018, 6:27pm UTC](https://discuss.elastic.co/t/logstash-filter-to-include-specific-keys-only/145452 "2018-08-23T18:27:15Z")
Hello Team, I have one use case related to logstash filters I need to parse nested json data with specific fields included and drop rest data Data format: { "\_index": "index-name", "\_source": { "level": "INFO", "… --- ## [Rename field after extracting it from another field](https://discuss.elastic.co/t/rename-field-after-extracting-it-from-another-field/145772)
**Author:** [@erezgby](https://discuss.elastic.co/u/erezgby)\ **Replies:** 1\ **Last updated:** [August 23, 2018, 6:24pm UTC](https://discuss.elastic.co/t/rename-field-after-extracting-it-from-another-field/145772 "2018-08-23T18:24:24Z")
Hi, I'm using kv filter in order to extract several fields out of "attributes" field, which works just fine. But then, I need to apply a mutate filter (rename in my case on one of this key-value pair that was extracted… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=695) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=697)