# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1181 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1182 --- ## [Logstash and Kafka, nothing happening and nothing in the logs](https://discuss.elastic.co/t/logstash-and-kafka-nothing-happening-and-nothing-in-the-logs/43809)
**Author:** [@nvigier](https://discuss.elastic.co/u/nvigier)\ **Replies:** 5\ **Last updated:** [March 9, 2016, 4:57pm UTC](https://discuss.elastic.co/t/logstash-and-kafka-nothing-happening-and-nothing-in-the-logs/43809 "2016-03-09T16:57:16Z")
Hi everyone, I have a setup that looks like this: (The data nodes also have the master role) With Rsyslog used as a lightweight forwarder between the rest of my network and my ELK setup, storing the logs in Kafka so… --- ## [SSL client failed to connect](https://discuss.elastic.co/t/ssl-client-failed-to-connect/43917)
**Author:** [@stevebeattie](https://discuss.elastic.co/u/stevebeattie)\ **Replies:** 0\ **Last updated:** [March 9, 2016, 3:46pm UTC](https://discuss.elastic.co/t/ssl-client-failed-to-connect/43917 "2016-03-09T15:46:51Z")
Hi, I have a server running ES & LS versions as follows:- # rpm -qa | grep -E "logstash|elast logstash-2.1.1-1.noarch elasticsearch-2.1.1-1.noarch And a number of clients running the following beats versions:- # r… --- ## [Disk and file system usage collection using Logstash](https://discuss.elastic.co/t/disk-and-file-system-usage-collection-using-logstash/43731)
**Author:** [@gabe](https://discuss.elastic.co/u/gabe)\ **Replies:** 17\ **Last updated:** [March 9, 2016, 8:34am UTC](https://discuss.elastic.co/t/disk-and-file-system-usage-collection-using-logstash/43731 "2016-03-09T08:34:21Z")
Hi All, Looking for some help to implement disk and file system (files and directories) usage collection by Logstash and storage in Elasticsearch. Basically, data processing workflows that we want to evaluate server … --- ## [Elasticsearch output plugin: frequent Connection Reset errors](https://discuss.elastic.co/t/elasticsearch-output-plugin-frequent-connection-reset-errors/43755)
**Author:** [@mguiwin](https://discuss.elastic.co/u/mguiwin)\ **Replies:** 0\ **Last updated:** [March 8, 2016, 10:30am UTC](https://discuss.elastic.co/t/elasticsearch-output-plugin-frequent-connection-reset-errors/43755 "2016-03-08T10:30:18Z")
Hello, I often get this in Logstash logs (every 5-10 minutes): {:timestamp=\>"2016-03-08T10:53:34.880000+0100", :message=\>"Attempted to send a bulk request to Elasticsearch configured at '\[\\"http://xxx.xxx.xxx.xxx:9200/… --- ## [GeoIP : DAT file belongs to logstash but it says permission denied](https://discuss.elastic.co/t/geoip-dat-file-belongs-to-logstash-but-it-says-permission-denied/43881)
**Author:** [@teewhey](https://discuss.elastic.co/u/teewhey)\ **Replies:** 0\ **Last updated:** [March 9, 2016, 9:29am UTC](https://discuss.elastic.co/t/geoip-dat-file-belongs-to-logstash-but-it-says-permission-denied/43881 "2016-03-09T09:29:14Z")
Got this on the logstash log. /etc/logstash/GeoLiteCity.dat belongs to the "logstash" user. Any idea why logstash complains about Permission Denied ? {:timestamp=\>"2016-03-09T09:26:43.710000+0000", :message=\>"Unknown… --- ## [Logstash complains permission denied when the file is owned by logstash user](https://discuss.elastic.co/t/logstash-complains-permission-denied-when-the-file-is-owned-by-logstash-user/43870)
**Author:** [@teewhey](https://discuss.elastic.co/u/teewhey)\ **Replies:** 2\ **Last updated:** [March 9, 2016, 9:22am UTC](https://discuss.elastic.co/t/logstash-complains-permission-denied-when-the-file-is-owned-by-logstash-user/43870 "2016-03-09T09:22:57Z")
I keep on getting this in the logstash log where Logstash complains saying unable to open the file (permission denied), but the file actually is set to belong the logstash user. Any idea why? root@PROD-logger-10-2-20… --- ## [Rsyslog collection using Logstash](https://discuss.elastic.co/t/rsyslog-collection-using-logstash/43728)
**Author:** [@gabe](https://discuss.elastic.co/u/gabe)\ **Replies:** 8\ **Last updated:** [March 9, 2016, 6:52am UTC](https://discuss.elastic.co/t/rsyslog-collection-using-logstash/43728 "2016-03-09T06:52:02Z")
Hi All, I was hoping for some help setting up syslog collection using the syslog input plugin for Logstash on Ubuntu 14.04 LTS. We have configured Topbeat, Logstash, Elasticsearch and Kibana and that seems to be wo… --- ## [How to drop delayed events beyond a certain threshold?](https://discuss.elastic.co/t/how-to-drop-delayed-events-beyond-a-certain-threshold/43851)
**Author:** [@mallah.rajesh](https://discuss.elastic.co/u/mallah.rajesh)\ **Replies:** 1\ **Last updated:** [March 9, 2016, 6:47am UTC](https://discuss.elastic.co/t/how-to-drop-delayed-events-beyond-a-certain-threshold/43851 "2016-03-09T06:47:55Z")
Hi, I need correct syntax for below pseudo code filter { if \[timestamp\] \< now - 120 { drop {} } } i want to drop messages delayed for more than 120secs in certain usage. I did search for 2 hrs before p… --- ## [Metrics Filter - Trying to count and graph multiple event sources](https://discuss.elastic.co/t/metrics-filter-trying-to-count-and-graph-multiple-event-sources/43623)
**Author:** [@berglh](https://discuss.elastic.co/u/berglh)\ **Replies:** 1\ **Last updated:** [March 8, 2016, 11:18pm UTC](https://discuss.elastic.co/t/metrics-filter-trying-to-count-and-graph-multiple-event-sources/43623 "2016-03-08T23:18:31Z")
What on earth I'm trying to do I'm receiving events from lots of remote hosts, I want to produce metrics per event source. I'm pretty sure my method is flawed and looking for some help to get my on the right path for us… --- ## [Error while starting Logstash 2.1.3](https://discuss.elastic.co/t/error-while-starting-logstash-2-1-3/43839)
**Author:** [@novicelogstash](https://discuss.elastic.co/u/novicelogstash)\ **Replies:** 0\ **Last updated:** [March 9, 2016, 12:34am UTC](https://discuss.elastic.co/t/error-while-starting-logstash-2-1-3/43839 "2016-03-09T00:34:57Z")
hi, I am a newbie trying to load up the ELK stack. I am trying to start logstash version 2.1.3, but I see the following error - bin/logstash -e 'input { stdin { } } output { stdout {} }' NameError: undefined method '… --- ## [Multiple outputs?](https://discuss.elastic.co/t/multiple-outputs/43746)
**Author:** [@Mathias\_Adler\_2](https://discuss.elastic.co/u/Mathias_Adler_2)\ **Replies:** 7\ **Last updated:** [March 8, 2016, 10:40pm UTC](https://discuss.elastic.co/t/multiple-outputs/43746 "2016-03-08T22:40:11Z")
Hi Experts, I Want to send events to multiple outputs but don't understand the options I have? From the docs it say: "Outputs are the final phase of the Logstash pipeline. An event can pass through multiple outputs, but … --- ## [How to increase max message size?](https://discuss.elastic.co/t/how-to-increase-max-message-size/43165)
**Author:** [@Junyan](https://discuss.elastic.co/u/Junyan)\ **Replies:** 5\ **Last updated:** [March 8, 2016, 9:38pm UTC](https://discuss.elastic.co/t/how-to-increase-max-message-size/43165 "2016-03-08T21:38:02Z")
I am using below logstash.conf. I am using all default settings. The max size of http requests that can be processed correctly is around 130 kb. If the request is larger than 130kb, LogStash does not output to stdout, an… --- ## [Logstash 2.2 OutOfMemoryError](https://discuss.elastic.co/t/logstash-2-2-outofmemoryerror/43710)
**Author:** [@Jim\_Jepson](https://discuss.elastic.co/u/Jim_Jepson)\ **Replies:** 4\ **Last updated:** [March 8, 2016, 9:25pm UTC](https://discuss.elastic.co/t/logstash-2-2-outofmemoryerror/43710 "2016-03-08T21:25:17Z")
We are trying to use Winlogbeats to archive off our Windows logs. To start this, we setup a Linux server running the latest ELK. On one of our three Windows 2012 R2 DC's we setup winlogbeats to ship the logs to the Ubu… --- ## [Getting Index Info from curl](https://discuss.elastic.co/t/getting-index-info-from-curl/43822)
**Author:** [@bluethundr](https://discuss.elastic.co/u/bluethundr)\ **Replies:** 1\ **Last updated:** [March 8, 2016, 9:36pm UTC](https://discuss.elastic.co/t/getting-index-info-from-curl/43822 "2016-03-08T21:36:57Z")
Guys, When I do a verbose curl on my indexes this is what I'll see: health status index pri rep docs.count docs.deleted store.size pri.store.size green open logstash-2016.02.22 5 1 3362748 … --- ## [Logstash 2.2.0 throughput slower than 2.1.1](https://discuss.elastic.co/t/logstash-2-2-0-throughput-slower-than-2-1-1/41650)
**Author:** [@tinle](https://discuss.elastic.co/u/tinle)\ **Replies:** 18\ **Last updated:** [March 8, 2016, 9:14pm UTC](https://discuss.elastic.co/t/logstash-2-2-0-throughput-slower-than-2-1-1/41650 "2016-03-08T21:14:05Z")
I wonder if anyone notices similar problem. I tested updating from 2.1.1 to 2.2.0 and immediate get these warnings in the logs {:timestamp=\>"2016-02-12T01:14:13.718000+0000", :message=\>"retrying failed action with re… --- ## [Logstash does not create index and doesn't give out any errors. Help](https://discuss.elastic.co/t/logstash-does-not-create-index-and-doesnt-give-out-any-errors-help/43630)
**Author:** [@MRamakri](https://discuss.elastic.co/u/MRamakri)\ **Replies:** 10\ **Last updated:** [March 8, 2016, 8:41pm UTC](https://discuss.elastic.co/t/logstash-does-not-create-index-and-doesnt-give-out-any-errors-help/43630 "2016-03-08T20:41:17Z")
Hi everyone, I'm new to Elasticsearch. I've been playing with Logstash, and I was able to create a couple of indexes using logstash. Now suddenly, it's not creating any indices. Whenever I run a conf file, it says star… --- ## [Logstash stops automatically with an error](https://discuss.elastic.co/t/logstash-stops-automatically-with-an-error/43774)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 6\ **Last updated:** [March 8, 2016, 3:32pm UTC](https://discuss.elastic.co/t/logstash-stops-automatically-with-an-error/43774 "2016-03-08T15:32:00Z")
Hi Experts, My logstash1.5.4 stops working with the flowing error . Please help me to resolve and understand this issue . {:timestamp=\>"2016-03-08T17:01:33.421000+0530", :message=\>"Failed to flush outgoing items", :o… --- ## [Redis input host array](https://discuss.elastic.co/t/redis-input-host-array/33248)
**Author:** [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)\ **Replies:** 3\ **Last updated:** [March 8, 2016, 3:28pm UTC](https://discuss.elastic.co/t/redis-input-host-array/33248 "2016-03-08T15:28:52Z")
Is there a reason the redis output can use an array but the redis input cannot. I would like to use my domain name (a DNS entry for many IP's with the same domain name) in the redis input so that I can spin up new no… --- ## [Logstash 2.2.2 not working on windows 7](https://discuss.elastic.co/t/logstash-2-2-2-not-working-on-windows-7/43716)
**Author:** [@sanaya](https://discuss.elastic.co/u/sanaya)\ **Replies:** 2\ **Last updated:** [March 8, 2016, 2:06pm UTC](https://discuss.elastic.co/t/logstash-2-2-2-not-working-on-windows-7/43716 "2016-03-08T14:06:23Z")
I've been trying to upgrade to the latest versions of ELK on a windows box to no avail. Testing my configurations one piece at a time starting with logstash. Current version of logstash that works with my configurations … --- ## [Local and offline plugin install with LS 2.2](https://discuss.elastic.co/t/local-and-offline-plugin-install-with-ls-2-2/43538)
**Author:** [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Replies:** 1\ **Last updated:** [March 8, 2016, 12:38pm UTC](https://discuss.elastic.co/t/local-and-offline-plugin-install-with-ls-2-2/43538 "2016-03-08T12:38:47Z")
I have a bespoke Logstash plugin which was written for LS 1.4.3 and works perfectly in that release. I'm trying to understand the steps involved in migrating it to work in LS 2.2 The machine(s) which need this plugi… --- ## [Logstash reads the same data multiple times](https://discuss.elastic.co/t/logstash-reads-the-same-data-multiple-times/43771)
**Author:** [@Suren92](https://discuss.elastic.co/u/Suren92)\ **Replies:** 0\ **Last updated:** [March 8, 2016, 12:47pm UTC](https://discuss.elastic.co/t/logstash-reads-the-same-data-multiple-times/43771 "2016-03-08T12:47:08Z")
Here is my logstash configuration that I am using on Linux system. input { file{ codec =\> json path =\> "/home/admin/ELK\_Demo/demo.txt" stat\_interval =\> 5 start\_position =\> "end… --- ## [Second fraction into timesamp \[Apache errorlog timestamp\]](https://discuss.elastic.co/t/second-fraction-into-timesamp-apache-errorlog-timestamp/43761)
**Author:** [@tikejhya](https://discuss.elastic.co/u/tikejhya)\ **Replies:** 1\ **Last updated:** [March 8, 2016, 11:41am UTC](https://discuss.elastic.co/t/second-fraction-into-timesamp-apache-errorlog-timestamp/43761 "2016-03-08T11:41:43Z")
Error: value=\>"Tue Mar 08 10:51:38.971058 2016" is malformed at \\".971058 2016\\"", :config\_parsers=\>"EEE MMM dd HH:mm:ss yyyy" I am trying to pass Aapache error log into logstash with timestamp being log time rather … --- ## [Aggregation by sum of field](https://discuss.elastic.co/t/aggregation-by-sum-of-field/43621)
**Author:** [@chenbe2204](https://discuss.elastic.co/u/chenbe2204)\ **Replies:** 2\ **Last updated:** [March 8, 2016, 7:36am UTC](https://discuss.elastic.co/t/aggregation-by-sum-of-field/43621 "2016-03-08T07:36:00Z")
Hi , If I want to aggregate by sum of specific field (for example build). Can you give me an example to upload all the fields from the json file and also the ability to aggregate by sum of build ? This is my conf fil… --- ## [Logstast startup issue](https://discuss.elastic.co/t/logstast-startup-issue/41771)
**Author:** [@ravitandur](https://discuss.elastic.co/u/ravitandur)\ **Replies:** 3\ **Last updated:** [February 16, 2016, 3:20am UTC](https://discuss.elastic.co/t/logstast-startup-issue/41771 "2016-02-16T03:20:25Z")
We are getting the below exception while starting the logstash agent. Any one can help me please? LoadError: Could not load FFI Provider: (NotImplementedError) FFI not available: null See http://jira.codehaus.org/browse… --- ## [How to create an index for Windows Log Event?](https://discuss.elastic.co/t/how-to-create-an-index-for-windows-log-event/43696)
**Author:** [@eso](https://discuss.elastic.co/u/eso)\ **Replies:** 4\ **Last updated:** [March 8, 2016, 6:45am UTC](https://discuss.elastic.co/t/how-to-create-an-index-for-windows-log-event/43696 "2016-03-08T06:45:28Z")
I have a Winlogbeat set up, and I want to set an explicit index for it as follows: output { elasticsearch { if \[type\] == "wineventlog" { hosts =\> \["localhost:9200"\] index =\> "winlogbeat" … --- ## [Querying elasticsearch for doc id](https://discuss.elastic.co/t/querying-elasticsearch-for-doc-id/43669)
**Author:** [@LiorK](https://discuss.elastic.co/u/LiorK)\ **Replies:** 3\ **Last updated:** [March 8, 2016, 6:31am UTC](https://discuss.elastic.co/t/querying-elasticsearch-for-doc-id/43669 "2016-03-08T06:31:40Z")
Hi all, I have logstash configuration and I'm trying to get the "\_id" of a document that answer to some terms (using elasticsearch filter) for example: i did a grok filter and now i have values for testA and testB an… --- ## [What is the best throughput for logstash?](https://discuss.elastic.co/t/what-is-the-best-throughput-for-logstash/43711)
**Author:** [@Kimzone](https://discuss.elastic.co/u/Kimzone)\ **Replies:** 1\ **Last updated:** [March 8, 2016, 12:12am UTC](https://discuss.elastic.co/t/what-is-the-best-throughput-for-logstash/43711 "2016-03-08T00:12:32Z")
Hi, I want to use ELK for analyzing big data. I'm talking about 24 files every day, each with ~10GB of data. There are ~ 104 columns in each CSV and about 60M lines. I'm interested to upload those files to elasticsearc… --- ## [Grok filter parsing tibco](https://discuss.elastic.co/t/grok-filter-parsing-tibco/43688)
**Author:** [@Sureshkumar](https://discuss.elastic.co/u/Sureshkumar)\ **Replies:** 7\ **Last updated:** [March 7, 2016, 9:29pm UTC](https://discuss.elastic.co/t/grok-filter-parsing-tibco/43688 "2016-03-07T21:29:58Z")
Currently i am parsing the tibco log using grok filter below is logs 2016 Mar 07 04:01:04:145 GMT -8 BW.TESServices-APPTST-TESTServices-APPTST-P13 User \[BW-User\] - Job-129210119 \[UtilityProcesses/CreateAuditTrail.proces… --- ## [Is "worker" configuration for output plugin deprecated?](https://discuss.elastic.co/t/is-worker-configuration-for-output-plugin-deprecated/43275)
**Author:** [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Replies:** 5\ **Last updated:** [March 7, 2016, 8:53pm UTC](https://discuss.elastic.co/t/is-worker-configuration-for-output-plugin-deprecated/43275 "2016-03-07T20:53:14Z")
From almost all output plugins, we can see "workers" parameter: https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-workers Is it still working with new l… --- ## [Grok filter patterns for syslog](https://discuss.elastic.co/t/grok-filter-patterns-for-syslog/43628)
**Author:** [@Shreejit\_Nair](https://discuss.elastic.co/u/Shreejit_Nair)\ **Replies:** 2\ **Last updated:** [March 7, 2016, 7:12pm UTC](https://discuss.elastic.co/t/grok-filter-patterns-for-syslog/43628 "2016-03-07T19:12:11Z")
Hi All, I am new to logstash and still learning something new about this cool tool everyday. I found that grok pattens work only with the patter we define Like %{IP:client} %{WORD:method} %{URIPATHPARAM:request} %… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1180) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1182)