# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1111 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1112 --- ## [Logstash template file does not exist or cannot be opened](https://discuss.elastic.co/t/logstash-template-file-does-not-exist-or-cannot-be-opened/58829)
**Author:** [@daiyue](https://discuss.elastic.co/u/daiyue)\ **Replies:** 3\ **Last updated:** [August 24, 2016, 8:14pm UTC](https://discuss.elastic.co/t/logstash-template-file-does-not-exist-or-cannot-be-opened/58829 "2016-08-24T20:14:40Z")
Hi, I set up a template file in logstash.conf output like this, output { elasticsearch { hosts =\> \["172.17.0.2:9200"\] manage\_template =\> false template =\> "/home/user\_name/docker-elk/logstash/core\_map… --- ## [Best Match & Exact Match](https://discuss.elastic.co/t/best-match-exact-match/58675)
**Author:** [@uppaljs](https://discuss.elastic.co/u/uppaljs)\ **Replies:** 6\ **Last updated:** [August 24, 2016, 8:07pm UTC](https://discuss.elastic.co/t/best-match-exact-match/58675 "2016-08-24T20:07:26Z")
Hi, I have input coming in from a database , specifically call detail records , I need to parse the calling party number to identify the country from a dictionary file. The problem I am facing is that unless the inpu… --- ## [Logstash plugin logstash-input-google-cloud-pubsub and how to change jruby version](https://discuss.elastic.co/t/logstash-plugin-logstash-input-google-cloud-pubsub-and-how-to-change-jruby-version/58855)
**Author:** [@Jingxuan\_Sun](https://discuss.elastic.co/u/Jingxuan_Sun)\ **Replies:** 0\ **Last updated:** [August 24, 2016, 5:36pm UTC](https://discuss.elastic.co/t/logstash-plugin-logstash-input-google-cloud-pubsub-and-how-to-change-jruby-version/58855 "2016-08-24T17:36:17Z")
I am trying to add plugin logstash-input-google-cloud-pubsub, When I built the gem file, It only support gem and ruby version\>2.2. I could install this plugin successfully using bin/plugin install. But when i run the tes… --- ## [Fetching logs form remote host?](https://discuss.elastic.co/t/fetching-logs-form-remote-host/2295)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 9\ **Last updated:** [August 24, 2016, 5:11pm UTC](https://discuss.elastic.co/t/fetching-logs-form-remote-host/2295 "2016-08-24T17:11:31Z")
Hi, I am using LSF as forwarder, logstash (as shipper and indexer on remote server) through redis. We have some windows machines which have some log files to read. Is there any way so that I can groke those log files thr… --- ## [Sending metrics to InfluxDB](https://discuss.elastic.co/t/sending-metrics-to-influxdb/51591)
**Author:** [@aodj](https://discuss.elastic.co/u/aodj)\ **Replies:** 1\ **Last updated:** [August 24, 2016, 3:24pm UTC](https://discuss.elastic.co/t/sending-metrics-to-influxdb/51591 "2016-08-24T15:24:48Z")
I'm attempting to use the metrics filter to parse incoming logs, and to dispatch the results to influxdb using the Logstash output for Influx. The influx plugin requires a hard coded list of data points to send to Influx… --- ## [Metrics filter and InfluxDB output](https://discuss.elastic.co/t/metrics-filter-and-influxdb-output/58834)
**Author:** [@elvarb](https://discuss.elastic.co/u/elvarb)\ **Replies:** 0\ **Last updated:** [August 24, 2016, 2:50pm UTC](https://discuss.elastic.co/t/metrics-filter-and-influxdb-output/58834 "2016-08-24T14:50:46Z")
It seems such a good fit, but I'm not seeing how these fit I have a metric filter like this metrics { meter =\> "winevent\_%{Channel}" add\_tag =\> "metric" } But how can … --- ## [Docker S3 log error](https://discuss.elastic.co/t/docker-s3-log-error/58634)
**Author:** [@luisgzvs](https://discuss.elastic.co/u/luisgzvs)\ **Replies:** 5\ **Last updated:** [August 24, 2016, 2:44pm UTC](https://discuss.elastic.co/t/docker-s3-log-error/58634 "2016-08-24T14:44:09Z")
I'm trying to analice an asset log from AWS S3 bucket, I have a docker container with the ELK stack integrated, my logstash config file is this one: input { file { path =\> "/home/\*" start\_position =\> … --- ## [Grok Expression in If condition Filter](https://discuss.elastic.co/t/grok-expression-in-if-condition-filter/58615)
**Author:** [@kamesh](https://discuss.elastic.co/u/kamesh)\ **Replies:** 4\ **Last updated:** [August 24, 2016, 2:14pm UTC](https://discuss.elastic.co/t/grok-expression-in-if-condition-filter/58615 "2016-08-24T14:14:52Z")
Is it possible to use Grok Expression with in If condition i.e. filter { if (\[message\] = /^\\%{TIMESTAMP\_ISO8601}) (i.e. if line starts with ISO8601 Timestamp it needs to execute different condition) { grok (some co… --- ## [Logstash - Rabbitmq - same message repeatedly being loaded](https://discuss.elastic.co/t/logstash-rabbitmq-same-message-repeatedly-being-loaded/58533)
**Author:** [@jaykumar](https://discuss.elastic.co/u/jaykumar)\ **Replies:** 4\ **Last updated:** [August 24, 2016, 2:11pm UTC](https://discuss.elastic.co/t/logstash-rabbitmq-same-message-repeatedly-being-loaded/58533 "2016-08-24T14:11:03Z")
Hi There, I have configured rabbitmq as a broker for reliability, below is my configuration: input { udp { port =\> 5114 type =\> "syslog.imq" } } output { if \[type\] == "… --- ## ["uuid" and "fingerprint" fields](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794)
**Author:** [@gayathri](https://discuss.elastic.co/u/gayathri)\ **Replies:** 7\ **Last updated:** [August 24, 2016, 11:35am UTC](https://discuss.elastic.co/t/uuid-and-fingerprint-fields/58794 "2016-08-24T11:35:04Z")
Hi Team, I am using below configuration in logstash.conf for avoiding duplicate log entries: filter{ uuid { target =\> "@uuid" overwrite =\> true } fingerprint { source =\> \["message"\] target =\> "fingerp… --- ## [Logstash log4j input gets stuck after a while](https://discuss.elastic.co/t/logstash-log4j-input-gets-stuck-after-a-while/58803)
**Author:** [@jonasdaugalas](https://discuss.elastic.co/u/jonasdaugalas)\ **Replies:** 0\ **Last updated:** [August 24, 2016, 10:38am UTC](https://discuss.elastic.co/t/logstash-log4j-input-gets-stuck-after-a-while/58803 "2016-08-24T10:38:41Z")
I think I have a problem with log4j input plugin. OS: Scientific Linux CERN SLC release 6.6 (Carbon) Logstash: 2.3.4 Config pipeline.conf: input { log4j { mode =\> "client" host =\> log\_output\_host port =\> 26021 … --- ## [Update document with Logstash without updating the @timestamp](https://discuss.elastic.co/t/update-document-with-logstash-without-updating-the-timestamp/58792)
**Author:** [@rdev](https://discuss.elastic.co/u/rdev)\ **Replies:** 0\ **Last updated:** [August 24, 2016, 9:32am UTC](https://discuss.elastic.co/t/update-document-with-logstash-without-updating-the-timestamp/58792 "2016-08-24T09:32:15Z")
When I update a document through the Update API, I can change and add fields without updating the timestamp. I am now trying to achieve this with Logstash using the Elasticsearch output action "update" with no success. … --- ## [Add\_field](https://discuss.elastic.co/t/add-field/58663)
**Author:** [@Lucifer\_hbp](https://discuss.elastic.co/u/Lucifer_hbp)\ **Replies:** 4\ **Last updated:** [August 24, 2016, 7:11am UTC](https://discuss.elastic.co/t/add-field/58663 "2016-08-24T07:11:16Z")
Hi again, i hav written following 2 pattern inside single match because i hav log files with 2 different patterns: grok { match =\> \["message","%{BR1}%{SYSTEMDATE:timestamp}%{BR2} %{GREEDYDATA:Msg}", … --- ## [Logstash -t option returning false config ok message](https://discuss.elastic.co/t/logstash-t-option-returning-false-config-ok-message/58733)
**Author:** [@GardenMWM](https://discuss.elastic.co/u/GardenMWM)\ **Replies:** 3\ **Last updated:** [August 24, 2016, 6:15am UTC](https://discuss.elastic.co/t/logstash-t-option-returning-false-config-ok-message/58733 "2016-08-24T06:15:15Z")
I've run into an unexpected behaviour of the -t test option and if statements. If I use the following invalid config, input { stdin {} } filter { if somefield == "somevalue" { drop {}} } output { stdout {} } and … --- ## [Logstash "UDP listener died" with exception: Unable to establish loopback connection](https://discuss.elastic.co/t/logstash-udp-listener-died-with-exception-unable-to-establish-loopback-connection/57308)
**Author:** [@venkaty](https://discuss.elastic.co/u/venkaty)\ **Replies:** 4\ **Last updated:** [August 24, 2016, 6:15am UTC](https://discuss.elastic.co/t/logstash-udp-listener-died-with-exception-unable-to-establish-loopback-connection/57308 "2016-08-24T06:15:05Z")
Hi, I configured logstash version 2.1.1 to receive logs on UDP port. After running successfully few hours following exception shown in diagnostic log {:timestamp=\>"2016-07-26T18:30:10.976000+0530", :message=\>"Rec… --- ## [How to get the timestamp of the log event and assign it to a new date field?](https://discuss.elastic.co/t/how-to-get-the-timestamp-of-the-log-event-and-assign-it-to-a-new-date-field/58721)
**Author:** [@Kulasangar\_Gowrisang](https://discuss.elastic.co/u/Kulasangar_Gowrisang)\ **Replies:** 6\ **Last updated:** [August 24, 2016, 6:12am UTC](https://discuss.elastic.co/t/how-to-get-the-timestamp-of-the-log-event-and-assign-it-to-a-new-date-field/58721 "2016-08-24T06:12:41Z")
I have the following filter criteria of logstash conf file and my customized pattern in the patterns file: Pattern: LOGTIMESTAMP %{YEAR}%{MONTHNUM}%{MONTHDAY} Filter in logstash: filter { grok { patterns\_dir … --- ## [Help config plugin output mail](https://discuss.elastic.co/t/help-config-plugin-output-mail/26259)
**Author:** [@fabioebner](https://discuss.elastic.co/u/fabioebner)\ **Replies:** 9\ **Last updated:** [August 24, 2016, 6:11am UTC](https://discuss.elastic.co/t/help-config-plugin-output-mail/26259 "2016-08-24T06:11:03Z")
I'm trying to configure the logstash to send the ERROR to my email I use this: email { to =\> "fabioebner@gmail.com" codec =\> "plain" contenttype =\> "text/html; charset=UTF-8" options =\> { smpt =\> { smtpIporHos… --- ## [AWS S3 grok pattern](https://discuss.elastic.co/t/aws-s3-grok-pattern/58752)
**Author:** [@luisgzvs](https://discuss.elastic.co/u/luisgzvs)\ **Replies:** 0\ **Last updated:** [August 23, 2016, 10:20pm UTC](https://discuss.elastic.co/t/aws-s3-grok-pattern/58752 "2016-08-23T22:20:12Z")
Hi! Can someone help me creating a grook pattern for this kind of log: 79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be mybucket \[06/Feb/2014:00:00:38 +0000\] 192.0.2.3 79a59df900b949e55d96a1e698fbacedfd… --- ## [Collect logs from a remote servers without installing Filebeat on all the remote servers](https://discuss.elastic.co/t/collect-logs-from-a-remote-servers-without-installing-filebeat-on-all-the-remote-servers/58490)
**Author:** [@shravan521](https://discuss.elastic.co/u/shravan521)\ **Replies:** 3\ **Last updated:** [August 23, 2016, 8:26pm UTC](https://discuss.elastic.co/t/collect-logs-from-a-remote-servers-without-installing-filebeat-on-all-the-remote-servers/58490 "2016-08-23T20:26:36Z")
We are currently using Logstash and Filebeat to collect logs from multiple servers on same network, where Filebeat is installed on each server to ship the logs. Now we have a different requirement to collect logs from a … --- ## [Logstash fails to send events to ES when one instance of cluster stops](https://discuss.elastic.co/t/logstash-fails-to-send-events-to-es-when-one-instance-of-cluster-stops/58724)
**Author:** [@jcballara](https://discuss.elastic.co/u/jcballara)\ **Replies:** 0\ **Last updated:** [August 23, 2016, 5:35pm UTC](https://discuss.elastic.co/t/logstash-fails-to-send-events-to-es-when-one-instance-of-cluster-stops/58724 "2016-08-23T17:35:32Z")
Hi, I have this config. Logstash (2.3\*) and ES (2.1.1). Logstash ES output are configured with two ES servers in cluster: hosts =\> \["192.168.48.120","192.168.48.121"\] There are 2 ES instances in this IPs configured… --- ## [S3 assets log](https://discuss.elastic.co/t/s3-assets-log/58401)
**Author:** [@luisgzvs](https://discuss.elastic.co/u/luisgzvs)\ **Replies:** 7\ **Last updated:** [August 23, 2016, 5:35pm UTC](https://discuss.elastic.co/t/s3-assets-log/58401 "2016-08-23T17:35:14Z")
Hi guys, I need help with this topic, I have to analice assets\_logs from AWS bucket, I've dowloaded them to my local machine, but the problem is when I use this command to get @timestamp from log message, the log example… --- ## [Error installing plugin contrib](https://discuss.elastic.co/t/error-installing-plugin-contrib/58633)
**Author:** [@scosta](https://discuss.elastic.co/u/scosta)\ **Replies:** 5\ **Last updated:** [August 23, 2016, 5:33pm UTC](https://discuss.elastic.co/t/error-installing-plugin-contrib/58633 "2016-08-23T17:33:51Z")
I'm having trouble installing the plugin contrib in logstash . It returns me the following error: plugin contrib does not exist. Can someone help me? Thank you. --- ## [Metric filter based on more than one variable](https://discuss.elastic.co/t/metric-filter-based-on-more-than-one-variable/58708)
**Author:** [@bebe](https://discuss.elastic.co/u/bebe)\ **Replies:** 0\ **Last updated:** [August 23, 2016, 3:55pm UTC](https://discuss.elastic.co/t/metric-filter-based-on-more-than-one-variable/58708 "2016-08-23T15:55:53Z")
Hello Everyone, Is there a way to create a metric events based on more than one field, e.g. status code AND hostname? I have tried adding a field with dynamic value, but apparently dynamic variables resolution is br… --- ## [Logstash Grok custom pattern](https://discuss.elastic.co/t/logstash-grok-custom-pattern/58685)
**Author:** [@Abhishek](https://discuss.elastic.co/u/Abhishek)\ **Replies:** 2\ **Last updated:** [August 23, 2016, 1:55pm UTC](https://discuss.elastic.co/t/logstash-grok-custom-pattern/58685 "2016-08-23T13:55:40Z")
Hi All, I want to parse following log patterns with logstash: 2016-08-12 16:37:00,039 \[8\] ERROR OnlineService Object reference not set to an instance of an object. As of now I have created following pattern: %{TIMEST… --- ## [Filter date (easy question)](https://discuss.elastic.co/t/filter-date-easy-question/58668)
**Author:** [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Replies:** 2\ **Last updated:** [August 23, 2016, 11:36am UTC](https://discuss.elastic.co/t/filter-date-easy-question/58668 "2016-08-23T11:36:23Z")
Struggling a bit to filter the date and time correctly. The log file contains date time action protocol 2016-08-23 11:53:38 ALLOW TCP My logstash filter config is as follows: filter { csv { columns =\> \[… --- ## [Multiple heartbeat inputs with same interval doesn't work as expected](https://discuss.elastic.co/t/multiple-heartbeat-inputs-with-same-interval-doesnt-work-as-expected/58671)
**Author:** [@gerdesj](https://discuss.elastic.co/u/gerdesj)\ **Replies:** 0\ **Last updated:** [August 23, 2016, 11:33am UTC](https://discuss.elastic.co/t/multiple-heartbeat-inputs-with-same-interval-doesnt-work-as-expected/58671 "2016-08-23T11:33:03Z")
I have Logstash (receiver) -\> redis buffer -\> Logstash (indexer) -\> ES cluster I created a single heartbeat input on each Logstash instance and output to file, for example on the receiver: input { heartbeat { … --- ## [How to format, yyyymmdd to yyyy-mm-dd in a log file?](https://discuss.elastic.co/t/how-to-format-yyyymmdd-to-yyyy-mm-dd-in-a-log-file/58666)
**Author:** [@Kulasangar\_Gowrisang](https://discuss.elastic.co/u/Kulasangar_Gowrisang)\ **Replies:** 3\ **Last updated:** [August 23, 2016, 11:26am UTC](https://discuss.elastic.co/t/how-to-format-yyyymmdd-to-yyyy-mm-dd-in-a-log-file/58666 "2016-08-23T11:26:30Z")
I've started working around with the ELK tool stack since few weeks. What I need is to get the timestamp of the log event which includes date and the time and show it in a different field (ie: log\_timestamp). I've tried … --- ## [Logstash Eleasticsearch Input Plugin for streaming data](https://discuss.elastic.co/t/logstash-eleasticsearch-input-plugin-for-streaming-data/58664)
**Author:** [@sen](https://discuss.elastic.co/u/sen)\ **Replies:** 1\ **Last updated:** [August 23, 2016, 10:28am UTC](https://discuss.elastic.co/t/logstash-eleasticsearch-input-plugin-for-streaming-data/58664 "2016-08-23T10:28:35Z")
Hi everyone, I would like to know if we can use logstash-elasticsearch-input plugin for streaming data , ex: if I have data available in my database and I run ElasticSearch input plugin , it will index the data into an… --- ## [Filter doesn't work](https://discuss.elastic.co/t/filter-doesnt-work/58657)
**Author:** [@emperor](https://discuss.elastic.co/u/emperor)\ **Replies:** 3\ **Last updated:** [August 23, 2016, 9:58am UTC](https://discuss.elastic.co/t/filter-doesnt-work/58657 "2016-08-23T09:58:51Z")
i use the http://grokdebug.herokuapp.com/ to filter the log content like "Tue Aug 9 14:35:07 2016 8.8.8.8:30044 \[name\] Peer Connection Initiated with \[AF\_INET\] 8.8.8.8:30044" when i use two rules ,it show results bu… --- ## [Can logstash merge several similar lines into one line?](https://discuss.elastic.co/t/can-logstash-merge-several-similar-lines-into-one-line/56884)
**Author:** [@sen](https://discuss.elastic.co/u/sen)\ **Replies:** 3\ **Last updated:** [August 23, 2016, 9:26am UTC](https://discuss.elastic.co/t/can-logstash-merge-several-similar-lines-into-one-line/56884 "2016-08-23T09:26:40Z")
Hi Folks, Here is what I have in my json input from my kafka server: {"deviceName":"device1", "counter":125} {"deviceName":"device1", "counter":125} {"deviceName":"device2", "counter":88} {"deviceName":"device1", "coun… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1110) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1112)