# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1216 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1217 --- ## [Logstash, nested json and AWS ELB logs](https://discuss.elastic.co/t/logstash-nested-json-and-aws-elb-logs/34957)
**Author:** [@aodj](https://discuss.elastic.co/u/aodj)\ **Replies:** 2\ **Last updated:** [November 27, 2015, 11:39am UTC](https://discuss.elastic.co/t/logstash-nested-json-and-aws-elb-logs/34957 "2015-11-27T11:39:46Z")
I've been using Logstash to parse my AWS ELB logs for a little while, but need to change my parsing since it's causing the ElasticSearch mapping to grow. What I have currently is this: if \[type\] == "elb" { grok { … --- ## [Logstash and RabbitMQ](https://discuss.elastic.co/t/logstash-and-rabbitmq/35724)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 1\ **Last updated:** [November 27, 2015, 9:18am UTC](https://discuss.elastic.co/t/logstash-and-rabbitmq/35724 "2015-11-27T09:18:22Z")
I read some post about using RabbitMQ with logstash. So what best deplyoyment Model for RabbitMQ and Logstash ? I have many Server (150GB log/day - \> 100 Server), so i need load balance for logstash and for RabbitMQ ??? --- ## [Logstash 2.0.0 Running Out of Memory](https://discuss.elastic.co/t/logstash-2-0-0-running-out-of-memory/34846)
**Author:** [@hiranya911](https://discuss.elastic.co/u/hiranya911)\ **Replies:** 1\ **Last updated:** [November 27, 2015, 8:12am UTC](https://discuss.elastic.co/t/logstash-2-0-0-running-out-of-memory/34846 "2015-11-27T08:12:08Z")
I'm using Logstash 2.0.0 to parse and index some Nginx access logs received through remote FileBeat. My configuration is very simple (shown below). But every now and then Logstash runs out of memory and crashes. It seems… --- ## ["rename" filter didn't rename the field in events](https://discuss.elastic.co/t/rename-filter-didnt-rename-the-field-in-events/35711)
**Author:** [@KavyaS](https://discuss.elastic.co/u/KavyaS)\ **Replies:** 1\ **Last updated:** [November 27, 2015, 7:27am UTC](https://discuss.elastic.co/t/rename-filter-didnt-rename-the-field-in-events/35711 "2015-11-27T07:27:37Z")
I used the filter plugin "rename" to rename a field in my events. Logstash didn't show any error on restart. But there doesn't seem to be any changes to the field name(checked in sense plugin). I added "rename" after ind… --- ## [How to make filter when using json format file](https://discuss.elastic.co/t/how-to-make-filter-when-using-json-format-file/35680)
**Author:** [@Young\_Heon\_Kim](https://discuss.elastic.co/u/Young_Heon_Kim)\ **Replies:** 3\ **Last updated:** [November 27, 2015, 5:18am UTC](https://discuss.elastic.co/t/how-to-make-filter-when-using-json-format-file/35680 "2015-11-27T05:18:59Z")
Hi everyone. I would like to parse json format log file which logged on Percona audit plugin. Actually format of log file is following Nov 25 23:25:18 serverHostName percona-audit: {"audit\_record":{"name":"Query", … --- ## [Logstash Forwarder Error in Windows](https://discuss.elastic.co/t/logstash-forwarder-error-in-windows/25140)
**Author:** [@rameeelastic](https://discuss.elastic.co/u/rameeelastic)\ **Replies:** 4\ **Last updated:** [November 26, 2015, 8:33pm UTC](https://discuss.elastic.co/t/logstash-forwarder-error-in-windows/25140 "2015-11-26T20:33:46Z")
Hi All I am running ELK stack on windows OS , below is the versions ElasticSearch -- 1.6.0 LogStash -- 1.5.2 Kibana -- 4.1.1 I am trying to push logs from local windows laptop to ELK windows server using logstash fo… --- ## [Parent/child with logstash 2.0 or 2.1](https://discuss.elastic.co/t/parent-child-with-logstash-2-0-or-2-1/35609)
**Author:** [@fchantrel](https://discuss.elastic.co/u/fchantrel)\ **Replies:** 2\ **Last updated:** [November 26, 2015, 8:04pm UTC](https://discuss.elastic.co/t/parent-child-with-logstash-2-0-or-2-1/35609 "2015-11-26T20:04:16Z")
Hi, I would like to migrate my elasticsearch cluster to 2.x but I am blocked with a problem on indexing child documents with Logstash. In 1.6 I had something working but parent/child relation has changed since 2.0 and I … --- ## [Shipping log files](https://discuss.elastic.co/t/shipping-log-files/35224)
**Author:** [@dhaval1](https://discuss.elastic.co/u/dhaval1)\ **Replies:** 18\ **Last updated:** [November 26, 2015, 6:07pm UTC](https://discuss.elastic.co/t/shipping-log-files/35224 "2015-11-26T18:07:41Z")
This is my first post. If it is not the right place, kindly redirect me to the correct place. I want to first ship the log files from the remote server to the central sever "as it is" with out any formatting using log… --- ## [Getting windows event log to elk server](https://discuss.elastic.co/t/getting-windows-event-log-to-elk-server/35661)
**Author:** [@ShraddhaBandhakavi](https://discuss.elastic.co/u/ShraddhaBandhakavi)\ **Replies:** 1\ **Last updated:** [November 26, 2015, 12:28pm UTC](https://discuss.elastic.co/t/getting-windows-event-log-to-elk-server/35661 "2015-11-26T12:28:53Z")
Hi All, I am new to ELK. I have setup of ELK on Linux server. I want to send windows event logs to elk server. i am using NXlog to collect windows logs But i was unable to collect events. please provide me basic s… --- ## [CRASHING: source sequence is illegal/malformed utf-8](https://discuss.elastic.co/t/crashing-source-sequence-is-illegal-malformed-utf-8/28920)
**Author:** [@alexolivan](https://discuss.elastic.co/u/alexolivan)\ **Replies:** 1\ **Last updated:** [November 26, 2015, 11:47am UTC](https://discuss.elastic.co/t/crashing-source-sequence-is-illegal-malformed-utf-8/28920 "2015-11-26T11:47:32Z")
Hi forum! I'm experimenting incresing number of logstash crashing with the following error log: JSON::GeneratorError: source sequence is illegal/malformed utf-8 to\_json at json/ext/GeneratorMethods.java:71 … --- ## [Logstash-input-stomp installation stuck](https://discuss.elastic.co/t/logstash-input-stomp-installation-stuck/35652)
**Author:** [@jusw85](https://discuss.elastic.co/u/jusw85)\ **Replies:** 0\ **Last updated:** [November 26, 2015, 9:56am UTC](https://discuss.elastic.co/t/logstash-input-stomp-installation-stuck/35652 "2015-11-26T09:56:59Z")
I was trying to install the latest GitHub version of logstash-input-stomp (which fixes some critical bugs) but the plugin manager seems to be stuck during installation. $ wget https://download.elastic.co/logstash/logst… --- ## [How can ship logs to Logstash using Windows Events?](https://discuss.elastic.co/t/how-can-ship-logs-to-logstash-using-windows-events/27644)
**Author:** [@thyfere](https://discuss.elastic.co/u/thyfere)\ **Replies:** 20\ **Last updated:** [November 26, 2015, 7:23am UTC](https://discuss.elastic.co/t/how-can-ship-logs-to-logstash-using-windows-events/27644 "2015-11-26T07:23:57Z")
Hi, I am very new to ELK stack. I have setup a stack using two Windows Server 2012 R2 VM machines, one is for Logstash and other for Kibana and Elastic search. Initially, I was thinking to use nxlog for logs forwarding… --- ## [Logstash Forwarder 0.4.0 for windows](https://discuss.elastic.co/t/logstash-forwarder-0-4-0-for-windows/35631)
**Author:** [@usahitya](https://discuss.elastic.co/u/usahitya)\ **Replies:** 1\ **Last updated:** [November 26, 2015, 7:07am UTC](https://discuss.elastic.co/t/logstash-forwarder-0-4-0-for-windows/35631 "2015-11-26T07:07:14Z")
Hi all, Is Logstash Forwarder 0.4.0 supports Windows2008R2 server? is there any document regarding how to install it in windows? and how to work with it? --- ## [Add field from an array](https://discuss.elastic.co/t/add-field-from-an-array/35453)
**Author:** [@incogniro](https://discuss.elastic.co/u/incogniro)\ **Replies:** 4\ **Last updated:** [November 26, 2015, 6:55am UTC](https://discuss.elastic.co/t/add-field-from-an-array/35453 "2015-11-26T06:55:12Z")
The syslog patterns parse into the message field which causes the message field to become an array. I would like to add a new field with mutate which uses as it's value the content of message\[1\] although I'm getting is… --- ## [Logstash forwarder is sending a message to multiple servers](https://discuss.elastic.co/t/logstash-forwarder-is-sending-a-message-to-multiple-servers/35574)
**Author:** [@tsvivek](https://discuss.elastic.co/u/tsvivek)\ **Replies:** 2\ **Last updated:** [November 26, 2015, 5:59am UTC](https://discuss.elastic.co/t/logstash-forwarder-is-sending-a-message-to-multiple-servers/35574 "2015-11-26T05:59:01Z")
We have multiple ELK environments (dev, test etc) and have observed recently that the forwarder is sending messages to multiple environments. In all environments, the forwarder config looks like this: { # The netwo… --- ## [How to use ruby filter](https://discuss.elastic.co/t/how-to-use-ruby-filter/35555)
**Author:** [@happyeveryday](https://discuss.elastic.co/u/happyeveryday)\ **Replies:** 1\ **Last updated:** [November 25, 2015, 7:12pm UTC](https://discuss.elastic.co/t/how-to-use-ruby-filter/35555 "2015-11-25T19:12:05Z")
My problem is specific as below. the event message is: {"a":\[{"aa":"v1"},{"bb":"v2"}\]} how to add a field ,namely, "a\_size" and its value is array a's length. so the output is like below: {"a":\[{"aa":"v1"},{"bb… --- ## [Read error looking for ack: i/o timeout in logstash-forwarder](https://discuss.elastic.co/t/read-error-looking-for-ack-i-o-timeout-in-logstash-forwarder/35594)
**Author:** [@bluethundr](https://discuss.elastic.co/u/bluethundr)\ **Replies:** 0\ **Last updated:** [November 25, 2015, 5:50pm UTC](https://discuss.elastic.co/t/read-error-looking-for-ack-i-o-timeout-in-logstash-forwarder/35594 "2015-11-25T17:50:32Z")
Guys, I keep seeing these messages repeated over and over again my logstash-forwarder logs: 2015/11/25 12:42:04.001162 Setting trusted CA from file: /etc/pki/CA/certs/ca.crt 2015/11/25 12:42:04.002295 Connecting to l… --- ## [Out put type as "float" instead of "String"](https://discuss.elastic.co/t/out-put-type-as-float-instead-of-string/35501)
**Author:** [@ruchii](https://discuss.elastic.co/u/ruchii)\ **Replies:** 2\ **Last updated:** [November 25, 2015, 5:55pm UTC](https://discuss.elastic.co/t/out-put-type-as-float-instead-of-string/35501 "2015-11-25T17:55:19Z")
I am looking for output with "User" and "Nice" parameters as float. I am getting as "String" because of "%" I guess. Any suggestions appreciated. $ ./logstash -e 'input { stdin { } } filter { grok { match =\> \[ "messa… --- ## [Multiline filter sometimes doesn't work](https://discuss.elastic.co/t/multiline-filter-sometimes-doesnt-work/27568)
**Author:** [@alessandro\_aglietti](https://discuss.elastic.co/u/alessandro_aglietti)\ **Replies:** 3\ **Last updated:** [November 25, 2015, 4:46pm UTC](https://discuss.elastic.co/t/multiline-filter-sometimes-doesnt-work/27568 "2015-11-25T16:46:06Z")
Hi there, I've setup the following configuration to parse a WordPress debug.log file and then send to Google Cloud Logging. input { file { path =\> \["/path/to/wp-content/debug.log"\] type =\> "debug" … --- ## [Logstash 2.1 : Multiple Instances per Host](https://discuss.elastic.co/t/logstash-2-1-multiple-instances-per-host/35548)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 4\ **Last updated:** [November 25, 2015, 2:03pm UTC](https://discuss.elastic.co/t/logstash-2-1-multiple-instances-per-host/35548 "2015-11-25T14:03:52Z")
Hi, I am trying to set up multiple logstash instances running on one host. I'm doing this by modifying the $name variable in the init script and modifying logfile output names and path to config etc. This works fine exce… --- ## [Need splitting incoming data](https://discuss.elastic.co/t/need-splitting-incoming-data/35515)
**Author:** [@Ahmed\_Faik](https://discuss.elastic.co/u/Ahmed_Faik)\ **Replies:** 3\ **Last updated:** [November 25, 2015, 12:24pm UTC](https://discuss.elastic.co/t/need-splitting-incoming-data/35515 "2015-11-25T12:24:17Z")
Hi all, i try using split filter property to split incoming data from tcp port, this data has the line format of seven values plus time stamp separated by comma "," and terminated with "\\n" as following : \[ v1,v2,v3,v4… --- ## [Exception: java.lang.ThreadDeath thrown from the UncaughtExceptionHandler in thread - logstash.err](https://discuss.elastic.co/t/exception-java-lang-threaddeath-thrown-from-the-uncaughtexceptionhandler-in-thread-logstash-err/33618)
**Author:** [@ottignon](https://discuss.elastic.co/u/ottignon)\ **Replies:** 2\ **Last updated:** [November 25, 2015, 10:29am UTC](https://discuss.elastic.co/t/exception-java-lang-threaddeath-thrown-from-the-uncaughtexceptionhandler-in-thread-logstash-err/33618 "2015-11-25T10:29:46Z")
Hi, My logstash.err file continues to have vague details about an exception added to it that unfortunately my favourite search engine has not been able to enlighten me one: cat /var/log/logstash/logstash.err '\[DEPREC… --- ## [NoMethodError: undefined method \`start\_with?' for nil:NilClass](https://discuss.elastic.co/t/nomethoderror-undefined-method-start-with-for-nil-nilclass/35139)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 1\ **Last updated:** [November 25, 2015, 10:28am UTC](https://discuss.elastic.co/t/nomethoderror-undefined-method-start-with-for-nil-nilclass/35139 "2015-11-25T10:28:57Z")
Hi, I am trying to send logs to zabbix server using logstash zabbix output. Sample log file contents 2015-11-11 10:27:40.8798 FATAL Some Service Providers are missing data required by FAS: TestServiceProvider-7 2015-1… --- ## [Browse RPM repositories](https://discuss.elastic.co/t/browse-rpm-repositories/35534)
**Author:** [@danieljamesscott](https://discuss.elastic.co/u/danieljamesscott)\ **Replies:** 0\ **Last updated:** [November 25, 2015, 9:56am UTC](https://discuss.elastic.co/t/browse-rpm-repositories/35534 "2015-11-25T09:56:43Z")
Is it possible to make the RPM/DEB repositories browsable? Currently, my browser just displays: http://packages.elastic.co/logstash/2.1/centos \ \NoSuchKey\ \The specified key does not exist.\<… --- ## [Determine field type](https://discuss.elastic.co/t/determine-field-type/34146)
**Author:** [@radu.stefanache](https://discuss.elastic.co/u/radu.stefanache)\ **Replies:** 3\ **Last updated:** [November 25, 2015, 9:50am UTC](https://discuss.elastic.co/t/determine-field-type/34146 "2015-11-25T09:50:55Z")
Hello everyone, I need a bit of help with this one . I get an object in two forms : 1st form { "parent": { "child": { "id": 123 } } } 2nd form { "parent": { "child": \[{ "id": 123 }\] } } I know that in the 2nd ca… --- ## [How log stash is recognizing file rotation and resetting byte offset in since db files of that file](https://discuss.elastic.co/t/how-log-stash-is-recognizing-file-rotation-and-resetting-byte-offset-in-since-db-files-of-that-file/35514)
**Author:** [@Sai\_Birada](https://discuss.elastic.co/u/Sai_Birada)\ **Replies:** 2\ **Last updated:** [November 25, 2015, 9:02am UTC](https://discuss.elastic.co/t/how-log-stash-is-recognizing-file-rotation-and-resetting-byte-offset-in-since-db-files-of-that-file/35514 "2015-11-25T09:02:13Z")
currently i have a data.log file from which log stash is reading data continuously, after some one lakh lines i am moving this data to data1.log and adding some 100 lines of data into new data.log file. How does log stas… --- ## [Grok filter confusion - Am I missing something?](https://discuss.elastic.co/t/grok-filter-confusion-am-i-missing-something/35461)
**Author:** [@plonka2000](https://discuss.elastic.co/u/plonka2000)\ **Replies:** 8\ **Last updated:** [November 25, 2015, 9:00am UTC](https://discuss.elastic.co/t/grok-filter-confusion-am-i-missing-something/35461 "2015-11-25T09:00:36Z")
Hi, I've been wresting with getting a grok filter to separate the "message" field to some new fields, so I can use them in Kibana. I understand that this should be a fairly straightforward process, when using the correc… --- ## [.har files into ELK stack](https://discuss.elastic.co/t/har-files-into-elk-stack/32296)
**Author:** [@philippconcur](https://discuss.elastic.co/u/philippconcur)\ **Replies:** 1\ **Last updated:** [November 25, 2015, 8:19am UTC](https://discuss.elastic.co/t/har-files-into-elk-stack/32296 "2015-11-25T08:19:22Z")
Has anyone had any success trending/visualizing .HAR (httparchive) files in Kibana? Har files seem to be in json format, hoping there is a shortcut there but just guessing. I am pretty new to to the ELK stack. Curious … --- ## [Cannot update logstash-input-beats plugin](https://discuss.elastic.co/t/cannot-update-logstash-input-beats-plugin/35436)
**Author:** [@ogauchard](https://discuss.elastic.co/u/ogauchard)\ **Replies:** 2\ **Last updated:** [November 25, 2015, 7:59am UTC](https://discuss.elastic.co/t/cannot-update-logstash-input-beats-plugin/35436 "2015-11-25T07:59:56Z")
Hello, I try to update logstash-input-beats plugin but it does not work : D:\\logstash\\bin\>plugin update logstash-input-beats io/console not supported; tty will not be manipulated You are updating logstash-input-beats… --- ## [Logstash failing to connect with Elasticsearch and file input plugin crashes](https://discuss.elastic.co/t/logstash-failing-to-connect-with-elasticsearch-and-file-input-plugin-crashes/35487)
**Author:** [@Vanguard\_space](https://discuss.elastic.co/u/Vanguard_space)\ **Replies:** 4\ **Last updated:** [November 25, 2015, 6:37am UTC](https://discuss.elastic.co/t/logstash-failing-to-connect-with-elasticsearch-and-file-input-plugin-crashes/35487 "2015-11-25T06:37:02Z")
I am trying to read a .txt file into Logstash and then grok parse it into ES. I am encountering this error: {:timestamp=\>"2015-11-24T20:46:44.957000+0000", :message=\>"A plugin had an unrecoverable error. Will restart… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1215) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1217)