# Logstash

**URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1221

[Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md)

**Page:** 1222

---

## [Syslog input, received character set?](https://discuss.elastic.co/t/syslog-input-received-character-set/34426)

<div class="topic-metadata">

**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\
**Replies:** 3\
**Last updated:** [November 12, 2015, 7:57pm UTC](https://discuss.elastic.co/t/syslog-input-received-character-set/34426 "2015-11-12T19:57:17Z")

</div>

I'm on linux using the logstash syslog input and ruby debug output but am getting this error: Received an event that has a different character encoding than you configured. {:text=\>"\\\\u0000\\\\u0005 ...", :expected\_chars…

---

## [So..what do I use with 2.0 now that "type" is gone?](https://discuss.elastic.co/t/so-what-do-i-use-with-2-0-now-that-type-is-gone/34450)

<div class="topic-metadata">

**Author:** [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\
**Replies:** 4\
**Last updated:** [November 12, 2015, 7:17pm UTC](https://discuss.elastic.co/t/so-what-do-i-use-with-2-0-now-that-type-is-gone/34450 "2015-11-12T19:17:49Z")

</div>

I use type for both input and filter: input { udp { type =\> "Snort" port =\> 5514 } } and: if \[type\] == "Snort" { grok { match =\>…

---

## [Logstash 2.0 and Zabbix plugin](https://discuss.elastic.co/t/logstash-2-0-and-zabbix-plugin/34392)

<div class="topic-metadata">

**Author:** [@MrAV](https://discuss.elastic.co/u/MrAV)\
**Replies:** 1\
**Last updated:** [November 12, 2015, 5:04pm UTC](https://discuss.elastic.co/t/logstash-2-0-and-zabbix-plugin/34392 "2015-11-12T17:04:38Z")

</div>

Hi, I'm trying to run Logstash 2.0.0 with Zabbix output plugin but, unfortunately, receiving an error: ←\[33mField referenced by \[@metadata\]\[zabbix\_key\_quikserv\] is missing {:level=\>:w arn}←\[0m ←\[33mField…

---

## [Is there a planned EOL for Logstash/Elasticsearch 1.x tree?](https://discuss.elastic.co/t/is-there-a-planned-eol-for-logstash-elasticsearch-1-x-tree/34311)

<div class="topic-metadata">

**Author:** [@Matteo\_Dessalvi](https://discuss.elastic.co/u/Matteo_Dessalvi)\
**Replies:** 1\
**Last updated:** [November 12, 2015, 4:54pm UTC](https://discuss.elastic.co/t/is-there-a-planned-eol-for-logstash-elasticsearch-1-x-tree/34311 "2015-11-12T16:54:50Z")

</div>

Hi all. As stated in the subject, I would like to know if Elastic is planning to support Logstash and ES 1.x trees in the future (at least for bugfixes) or if the development will be dropped at some point in favor of t…

---

## [De\_dot adding fields](https://discuss.elastic.co/t/de-dot-adding-fields/34425)

<div class="topic-metadata">

**Author:** [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)\
**Replies:** 2\
**Last updated:** [November 12, 2015, 4:09pm UTC](https://discuss.elastic.co/t/de-dot-adding-fields/34425 "2015-11-12T16:09:50Z")

</div>

When I use the de\_dot filter I am using a conditional statement that states: filter { if \[type\] == "BRO" { if \[log\_path\] == "software" { de\_dot { fields =\> \[ "version.major", "v…

---

## [Impossible to grok on file field, causes Logstash to fail](https://discuss.elastic.co/t/impossible-to-grok-on-file-field-causes-logstash-to-fail/33740)

<div class="topic-metadata">

**Author:** [@incogniro](https://discuss.elastic.co/u/incogniro)\
**Replies:** 0\
**Last updated:** [November 4, 2015, 12:41pm UTC](https://discuss.elastic.co/t/impossible-to-grok-on-file-field-causes-logstash-to-fail/33740 "2015-11-04T12:41:40Z")

</div>

Using the most basic configuration below (I've removed the more complex match patterns): input { lumberjack { port =\> 6782 ssl\_certificate =\> '/etc/pki/tls/certs/logstash-forwarder.crt' ssl\_ke…

---

## [Using logstash filters to convert stdin message to be the json written to elasticsearch](https://discuss.elastic.co/t/using-logstash-filters-to-convert-stdin-message-to-be-the-json-written-to-elasticsearch/33821)

<div class="topic-metadata">

**Author:** [@iamthealex](https://discuss.elastic.co/u/iamthealex)\
**Replies:** 2\
**Last updated:** [November 12, 2015, 3:04pm UTC](https://discuss.elastic.co/t/using-logstash-filters-to-convert-stdin-message-to-be-the-json-written-to-elasticsearch/33821 "2015-11-12T15:04:50Z")

</div>

Newbie alert. Today is my first day reading through elasticsearch, logstash and kibana. Goal: post a chunked stream of json strings to an http server that converts each json chunk to be the message inserted into elast…

---

## [Writing a Logstash GEM gives an error: undefined local variable or method](https://discuss.elastic.co/t/writing-a-logstash-gem-gives-an-error-undefined-local-variable-or-method/34422)

<div class="topic-metadata">

**Author:** [@Isra](https://discuss.elastic.co/u/Isra)\
**Replies:** 0\
**Last updated:** [November 12, 2015, 2:22pm UTC](https://discuss.elastic.co/t/writing-a-logstash-gem-gives-an-error-undefined-local-variable-or-method/34422 "2015-11-12T14:22:46Z")

</div>

Trying to build a gem to use it for Logstash as a filter. I'm using classes and methods from a .jar file. The following is my Jruby code so far: require 'java' require 'Processing.jar' $CLASSPATH \<\< 'lib' class Log…

---

## [Name each URISTEM with unique name](https://discuss.elastic.co/t/name-each-uristem-with-unique-name/34339)

<div class="topic-metadata">

**Author:** [@bnixon](https://discuss.elastic.co/u/bnixon)\
**Replies:** 2\
**Last updated:** [November 12, 2015, 2:19pm UTC](https://discuss.elastic.co/t/name-each-uristem-with-unique-name/34339 "2015-11-12T14:19:46Z")

</div>

Hey all, I’m new to Logstash (and ELK) I’ve been playing with it for the past couple weeks here at work. I have a project where a business unit wants to track activity on our company portal page. What I’m looking to do…

---

## [Pathetic Logstash performance using UDP input and Redis output](https://discuss.elastic.co/t/pathetic-logstash-performance-using-udp-input-and-redis-output/34409)

<div class="topic-metadata">

**Author:** [@venkaty](https://discuss.elastic.co/u/venkaty)\
**Replies:** 0\
**Last updated:** [November 12, 2015, 12:30pm UTC](https://discuss.elastic.co/t/pathetic-logstash-performance-using-udp-input-and-redis-output/34409 "2015-11-12T12:30:01Z")

</div>

Following is the log-stash configuration used for capturing the events from various nodes streaming syslog events and the events are buffered in Redis queue for processing. input { udp { port =\> 514 …

---

## [Access metadata from input elasticsearch](https://discuss.elastic.co/t/access-metadata-from-input-elasticsearch/34357)

<div class="topic-metadata">

**Author:** [@vilas](https://discuss.elastic.co/u/vilas)\
**Replies:** 3\
**Last updated:** [November 12, 2015, 3:45am UTC](https://discuss.elastic.co/t/access-metadata-from-input-elasticsearch/34357 "2015-11-12T03:45:42Z")

</div>

Hi, I am using elasticsearch input plugin for copying index from one cluster to another. I would like to access document metadata like \_type. I want to modify the \_type and store in target elasticsearch cluster with ne…

---

## [Logstash Syslog,UDP listener died](https://discuss.elastic.co/t/logstash-syslog-udp-listener-died/34183)

<div class="topic-metadata">

**Author:** [@kmroz](https://discuss.elastic.co/u/kmroz)\
**Replies:** 5\
**Last updated:** [November 11, 2015, 10:09pm UTC](https://discuss.elastic.co/t/logstash-syslog-udp-listener-died/34183 "2015-11-11T22:09:14Z")

</div>

elasticsearch=1.6 logstash=1.5 Im using docker with supervisord to start up the ELK stack. I tried using ort 514 and port 5009 to listen on the syslog messages coming in but when i run logstash in debug mode i get the …

---

## [How to define the patterns for filters in logstash](https://discuss.elastic.co/t/how-to-define-the-patterns-for-filters-in-logstash/34150)

<div class="topic-metadata">

**Author:** [@shravankodipaka](https://discuss.elastic.co/u/shravankodipaka)\
**Replies:** 9\
**Last updated:** [November 11, 2015, 9:42pm UTC](https://discuss.elastic.co/t/how-to-define-the-patterns-for-filters-in-logstash/34150 "2015-11-11T21:42:48Z")

</div>

Hi all, I am new to Logstash, I've just set up a new ELK-stack for analyzing of Tomcat and EC2 instance logs in Kibana. For this how to define the patterns and how to configure custom patterns in logstash.conf file...…

---

## [Consume a large of data in a file with Logstash](https://discuss.elastic.co/t/consume-a-large-of-data-in-a-file-with-logstash/34322)

<div class="topic-metadata">

**Author:** [@Jeferson\_Martins](https://discuss.elastic.co/u/Jeferson_Martins)\
**Replies:** 6\
**Last updated:** [November 11, 2015, 4:37pm UTC](https://discuss.elastic.co/t/consume-a-large-of-data-in-a-file-with-logstash/34322 "2015-11-11T16:37:40Z")

</div>

Hi, I have a logstash that have consume nginx logs and another applications in multiples input file in logstash configuration, but when we increase the acess in the website, we have a lot of access and the logstash del…

---

## [Checking for existence of nested field](https://discuss.elastic.co/t/checking-for-existence-of-nested-field/34330)

<div class="topic-metadata">

**Author:** [@danieljamesscott](https://discuss.elastic.co/u/danieljamesscott)\
**Replies:** 2\
**Last updated:** [November 11, 2015, 1:10pm UTC](https://discuss.elastic.co/t/checking-for-existence-of-nested-field/34330 "2015-11-11T13:10:54Z")

</div>

Hi, Some of my grok filters parse my message fields into 2 parts: "message" =\> \[ \[0\] "2015-11-11 12:44:38.225 +0000 INFO \[pool-1-thread-1\] com.example.JavaClass: logmessage", \[1\] "logmessage" \], …

---

## [Field name duplicate with file conf and json doc](https://discuss.elastic.co/t/field-name-duplicate-with-file-conf-and-json-doc/34295)

<div class="topic-metadata">

**Author:** [@no\_jihun](https://discuss.elastic.co/u/no_jihun)\
**Replies:** 6\
**Last updated:** [November 11, 2015, 1:08pm UTC](https://discuss.elastic.co/t/field-name-duplicate-with-file-conf-and-json-doc/34295 "2015-11-11T13:08:23Z")

</div>

Hello. I'm new to logstash and using logstash 1.5.2, elasticsearch 1.7.x And have logstash configuration like... 1 input { 2 file { 3 path =\> "/home1/.logstash/json\_input/deco.json.\*" 4 …

---

## [Logstash Conf file (geo location) - coordinates points always to the center of the Kibana tile map](https://discuss.elastic.co/t/logstash-conf-file-geo-location-coordinates-points-always-to-the-center-of-the-kibana-tile-map/34275)

<div class="topic-metadata">

**Author:** [@luis.carvalho](https://discuss.elastic.co/u/luis.carvalho)\
**Replies:** 2\
**Last updated:** [November 11, 2015, 11:58am UTC](https://discuss.elastic.co/t/logstash-conf-file-geo-location-coordinates-points-always-to-the-center-of-the-kibana-tile-map/34275 "2015-11-11T11:58:46Z")

</div>

Hi, I´m having some trouble configuring the logstash conf file with geo location. I have in my mapping: "cpe\_location" : { "properties" : { "location" : { "type" : "geo\_point" } } } And I try in the filter part: …

---

## [Including local jar files](https://discuss.elastic.co/t/including-local-jar-files/34319)

<div class="topic-metadata">

**Author:** [@Isra](https://discuss.elastic.co/u/Isra)\
**Replies:** 0\
**Last updated:** [November 11, 2015, 11:33am UTC](https://discuss.elastic.co/t/including-local-jar-files/34319 "2015-11-11T11:33:54Z")

</div>

I'm trying to build and use a gem that is dependent on jar files. I do not wanna publish the jar files, I need them locally only. in my .gemspec file, I have the following # Jar dependencies s.requirements \<\< "ja…

---

## [Problem with kv filter and empty value](https://discuss.elastic.co/t/problem-with-kv-filter-and-empty-value/34306)

<div class="topic-metadata">

**Author:** [@rzrucher](https://discuss.elastic.co/u/rzrucher)\
**Replies:** 0\
**Last updated:** [November 11, 2015, 8:46am UTC](https://discuss.elastic.co/t/problem-with-kv-filter-and-empty-value/34306 "2015-11-11T08:46:00Z")

</div>

Hi, I'm trying to parse some syslog-ng event using grok and the kv filter. I've this kind of log: AMOUNT=-4,700.00\\tEXCH.RATE=\\tCCY=ETB\\t I use the KV filter like this : kv { field\_split =\> "\\t" source =\> …

---

## [S3 input plugin without AWS credentials](https://discuss.elastic.co/t/s3-input-plugin-without-aws-credentials/34152)

<div class="topic-metadata">

**Author:** [@an1994](https://discuss.elastic.co/u/an1994)\
**Replies:** 1\
**Last updated:** [November 10, 2015, 9:53pm UTC](https://discuss.elastic.co/t/s3-input-plugin-without-aws-credentials/34152 "2015-11-10T21:53:57Z")

</div>

Hi, I'm working on something that needs data to be transfered from s3 to Elasticsearch.So, I chose Logstash for transfering data. In the config file of logstash, I dont want to provide AWS credentials(access\_key\_id,secre…

---

## [Odd problem withe "failed to parse" messages in logstash.log](https://discuss.elastic.co/t/odd-problem-withe-failed-to-parse-messages-in-logstash-log/34266)

<div class="topic-metadata">

**Author:** [@Cylindric](https://discuss.elastic.co/u/Cylindric)\
**Replies:** 0\
**Last updated:** [November 10, 2015, 4:35pm UTC](https://discuss.elastic.co/t/odd-problem-withe-failed-to-parse-messages-in-logstash-log/34266 "2015-11-10T16:35:24Z")

</div>

I am using nxlog to pump Windows EventLogs to Logstash. It's filling my logs with warnings, but I can't work out why it's even complaining. It looks like it's a field I'm not even using. Part of the problem is that the…

---

## [How to setup logstash-forwarder to read recursively on a path?](https://discuss.elastic.co/t/how-to-setup-logstash-forwarder-to-read-recursively-on-a-path/29814)

<div class="topic-metadata">

**Author:** [@chaudharikiranp](https://discuss.elastic.co/u/chaudharikiranp)\
**Replies:** 6\
**Last updated:** [November 10, 2015, 4:25pm UTC](https://discuss.elastic.co/t/how-to-setup-logstash-forwarder-to-read-recursively-on-a-path/29814 "2015-11-10T16:25:27Z")

</div>

Please helo me understand how to setup logstash-forwarder to read recursively on a path. For example if the file I want logstash forwarder is build.xml and it can be anywhere in the directory structure "/foo/bar/jobs/…

---

## [Not getting any ouput](https://discuss.elastic.co/t/not-getting-any-ouput/34172)

<div class="topic-metadata">

**Author:** [@Jackal9301](https://discuss.elastic.co/u/Jackal9301)\
**Replies:** 11\
**Last updated:** [November 10, 2015, 2:33pm UTC](https://discuss.elastic.co/t/not-getting-any-ouput/34172 "2015-11-10T14:33:12Z")

</div>

I'm trying to see the output of an if statement with grok filters. Logstash says it's starting however there is no file ever created in my output. Shouldn't I be seeing something even if there arent any matches? Test i…

---

## [Logstash 2.0.0 SSLError: Socket closed](https://discuss.elastic.co/t/logstash-2-0-0-sslerror-socket-closed/33748)

<div class="topic-metadata">

**Author:** [@pbmsys](https://discuss.elastic.co/u/pbmsys)\
**Replies:** 3\
**Last updated:** [November 10, 2015, 12:42pm UTC](https://discuss.elastic.co/t/logstash-2-0-0-sslerror-socket-closed/33748 "2015-11-10T12:42:24Z")

</div>

I am receiving an SSL socket error when using the new Logstash 2.0.0. My configuration: NXLog conf: \<Input network\> Module im\_tcp Host 0.0.0.0 Port 1514 Exec …

---

## [INFLIGHT\_EVENTS\_REPORT being logged every 5 seconds](https://discuss.elastic.co/t/inflight-events-report-being-logged-every-5-seconds/33759)

<div class="topic-metadata">

**Author:** [@weevil](https://discuss.elastic.co/u/weevil)\
**Replies:** 3\
**Last updated:** [November 9, 2015, 11:42pm UTC](https://discuss.elastic.co/t/inflight-events-report-being-logged-every-5-seconds/33759 "2015-11-09T23:42:38Z")

</div>

I am seeing the following in my log every 5 seconds: {:timestamp=\>"2015-11-04T09:18:47.164000-0600", :message=\>\["INFLIGHT\_EVENTS\_REPORT", "2015-11-04T09:18:47-06:00", {"input\_to\_filter"=\>0, "filter\_to\_output"=\>0, "outp…

---

## [Please read: Upgrading Logstash and Elasticsearch to 2.0](https://discuss.elastic.co/t/please-read-upgrading-logstash-and-elasticsearch-to-2-0/33791)

<div class="topic-metadata">

**Author:** [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Replies:** 1\
**Last updated:** [November 9, 2015, 10:53pm UTC](https://discuss.elastic.co/t/please-read-upgrading-logstash-and-elasticsearch-to-2-0/33791 "2015-11-09T22:53:24Z")

</div>

Dots in Fields: For those of you who have not yet upgraded to Logstash 2.0 and Elasticsearch 2.0, please be advised that Elasticsearch 2.0 does not allow field names to contain the . character (further details here). Th…

---

## [Statsd not work in my logstash](https://discuss.elastic.co/t/statsd-not-work-in-my-logstash/34197)

<div class="topic-metadata">

**Author:** [@Chengcheng\_Pei](https://discuss.elastic.co/u/Chengcheng_Pei)\
**Replies:** 1\
**Last updated:** [November 9, 2015, 9:18pm UTC](https://discuss.elastic.co/t/statsd-not-work-in-my-logstash/34197 "2015-11-09T21:18:55Z")

</div>

The config file: # input are the kafka messages input { kafka { topic\_id =\> 'test2' } } # Try to match sensor info filter { json { source =\> "message"} } # StatsD and stdout output output { …

---

## [Use of \[@metadata\] in logstash 2.0](https://discuss.elastic.co/t/use-of-metadata-in-logstash-2-0/34191)

<div class="topic-metadata">

**Author:** [@Philip\_Pum](https://discuss.elastic.co/u/Philip_Pum)\
**Replies:** 2\
**Last updated:** [November 9, 2015, 8:09pm UTC](https://discuss.elastic.co/t/use-of-metadata-in-logstash-2-0/34191 "2015-11-09T20:09:57Z")

</div>

I read about the use of the \[@metadata\] field in the 1.5 version (https://www.elastic.co/blog/logstash-metadata) and wanted to try it out on logstash 2.0. Unfortunately, there seems to be no way of forcing to output meta…

---

## [Mutate and statsd not work in my logstash](https://discuss.elastic.co/t/mutate-and-statsd-not-work-in-my-logstash/34185)

<div class="topic-metadata">

**Author:** [@Chengcheng\_Pei](https://discuss.elastic.co/u/Chengcheng_Pei)\
**Replies:** 7\
**Last updated:** [November 9, 2015, 7:59pm UTC](https://discuss.elastic.co/t/mutate-and-statsd-not-work-in-my-logstash/34185 "2015-11-09T19:59:52Z")

</div>

New to logstash. Assume messages in kafka look like: {'test message':\[{'key':'key-1234'}\]} {"test message":\[{"key":"key-1234"}\]} ... Try to increase the statsd key:test-1234. There are two problems: (1) when i…

---

## [Final word on the need for brokers with Logstash 2.0?](https://discuss.elastic.co/t/final-word-on-the-need-for-brokers-with-logstash-2-0/34188)

<div class="topic-metadata">

**Author:** [@ehogan](https://discuss.elastic.co/u/ehogan)\
**Replies:** 1\
**Last updated:** [November 9, 2015, 7:35pm UTC](https://discuss.elastic.co/t/final-word-on-the-need-for-brokers-with-logstash-2-0/34188 "2015-11-09T19:35:47Z")

</div>

I have gone through what documents I can find, and a presentation (which had no audio, only slides) and I find a number of references to the new built-in queuing mechanism in Logstash 2.0. But I cannot find anything def…

[Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1220)

[Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1222)
