# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1197 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1198 --- ## [How does Logstash process Configs? Order that is processes input?](https://discuss.elastic.co/t/how-does-logstash-process-configs-order-that-is-processes-input/40413)
**Author:** [@jnpetty](https://discuss.elastic.co/u/jnpetty)\ **Replies:** 4\ **Last updated:** [January 28, 2016, 8:05pm UTC](https://discuss.elastic.co/t/how-does-logstash-process-configs-order-that-is-processes-input/40413 "2016-01-28T20:05:36Z")
I have a Syslog tcp input thats my catch all when I cant setup a defined input. Anyway, the input file sets the type as "syslog" tcp { port =\> 5140 type =\> "syslog" } I then have another config file, called 01… --- ## [Getting started with Ruby and Logstash](https://discuss.elastic.co/t/getting-started-with-ruby-and-logstash/40386)
**Author:** [@jjdepaul](https://discuss.elastic.co/u/jjdepaul)\ **Replies:** 1\ **Last updated:** [January 28, 2016, 6:11pm UTC](https://discuss.elastic.co/t/getting-started-with-ruby-and-logstash/40386 "2016-01-28T18:11:28Z")
I've been able to get a lot done with your help here and with documentation on Logstash. Now I want to start learning how to program in Logstash with Ruby. How does one get started with understanding of the Logstash … --- ## [Logstash: Disabling logstash's file modification waiting behavior, while uploading data from a file](https://discuss.elastic.co/t/logstash-disabling-logstashs-file-modification-waiting-behavior-while-uploading-data-from-a-file/40322)
**Author:** [@krushnat\_khawale](https://discuss.elastic.co/u/krushnat_khawale)\ **Replies:** 3\ **Last updated:** [January 28, 2016, 5:57pm UTC](https://discuss.elastic.co/t/logstash-disabling-logstashs-file-modification-waiting-behavior-while-uploading-data-from-a-file/40322 "2016-01-28T17:57:46Z")
I'm working with logstash, for uploading data into the elaststicsearch from a CSV file. I've succeeded to upload it to the elasticsearch, but the only issue I am having right now is when the CSV file is uploaded, logstas… --- ## [Logstash not starting on Ubuntu](https://discuss.elastic.co/t/logstash-not-starting-on-ubuntu/39002)
**Author:** [@Devnull](https://discuss.elastic.co/u/Devnull)\ **Replies:** 4\ **Last updated:** [January 28, 2016, 5:17pm UTC](https://discuss.elastic.co/t/logstash-not-starting-on-ubuntu/39002 "2016-01-28T17:17:51Z")
Hi, Have installed Logstash from deb http://packages.elastic.co/logstash/2.1/debian stable main. I have also created a very simple config file : input { file { path =\> "/tmp/test" start\_position … --- ## [Repeated values with mutate filter in nested json output](https://discuss.elastic.co/t/repeated-values-with-mutate-filter-in-nested-json-output/40298)
**Author:** [@fraescaya10](https://discuss.elastic.co/u/fraescaya10)\ **Replies:** 2\ **Last updated:** [January 28, 2016, 3:38pm UTC](https://discuss.elastic.co/t/repeated-values-with-mutate-filter-in-nested-json-output/40298 "2016-01-28T15:38:29Z")
I have the following input log: .... proto=6 source=10.13.30.47 .... I need the following output in json: "data": { "key1": { "src": "10.13.30.47", "proto": 6 }, "key2": { "src": … --- ## [Count metric not counting real events](https://discuss.elastic.co/t/count-metric-not-counting-real-events/40379)
**Author:** [@marcmga](https://discuss.elastic.co/u/marcmga)\ **Replies:** 0\ **Last updated:** [January 28, 2016, 3:23pm UTC](https://discuss.elastic.co/t/count-metric-not-counting-real-events/40379 "2016-01-28T15:23:18Z")
Hello, I'm using metric filter to count all events received from each server i have (separatelly). The value i get is not real, count show a number which is higher than the real events i received. metrics { m… --- ## [Multiple patterns in multiline codec](https://discuss.elastic.co/t/multiple-patterns-in-multiline-codec/40336)
**Author:** [@MatthiasLenhardt](https://discuss.elastic.co/u/MatthiasLenhardt)\ **Replies:** 0\ **Last updated:** [January 28, 2016, 8:28am UTC](https://discuss.elastic.co/t/multiple-patterns-in-multiline-codec/40336 "2016-01-28T08:28:13Z")
Hello All, I'm sending Syslog messages to our elasticsearch cluster via logstash and have currently configured one multiline codec in my logstash.conf input section to handle a default behaviour of syslog for many equa… --- ## [Logstash listening on port 5000 and is able to connect with Filebeat but not parsing logs](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329)
**Author:** [@abinay](https://discuss.elastic.co/u/abinay)\ **Replies:** 17\ **Last updated:** [January 28, 2016, 8:15am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329 "2016-01-28T08:15:15Z")
Hii I have this architecture with me - Filebeat ships logs to logstash(listening on port 5000) . Logstash parses logs (i can see on my console too) and pushes them to elasticsearch. Now when I am running my Filebeat , it… --- ## [\_grokparsefailure while reading logs from file by logstash](https://discuss.elastic.co/t/-grokparsefailure-while-reading-logs-from-file-by-logstash/40285)
**Author:** [@maciej.kola](https://discuss.elastic.co/u/maciej.kola)\ **Replies:** 1\ **Last updated:** [January 28, 2016, 7:24am UTC](https://discuss.elastic.co/t/-grokparsefailure-while-reading-logs-from-file-by-logstash/40285 "2016-01-28T07:24:44Z")
Hi Mates, Can anyone please tell me, why I got "\_grokparsefailure" while reading logs from file? My logs look like this: 2016-01-27T21:17:31+01:00 10.1.2.132 %ASA-1-302013: Built outbound TCP connection 890918059 for… --- ## [Split message by field to different outputs](https://discuss.elastic.co/t/split-message-by-field-to-different-outputs/40296)
**Author:** [@Alexander\_Popov](https://discuss.elastic.co/u/Alexander_Popov)\ **Replies:** 1\ **Last updated:** [January 28, 2016, 7:19am UTC](https://discuss.elastic.co/t/split-message-by-field-to-different-outputs/40296 "2016-01-28T07:19:01Z")
I have udp input with json codec my records looks like : { "id":"value", "field1:"val", "fieldN:"val", details: { big nested structure } } How can I split this message into { "id":"value", "field1:"val", "… --- ## [Logstash 2.1: Dynamically Altering Field Names](https://discuss.elastic.co/t/logstash-2-1-dynamically-altering-field-names/36845)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 5\ **Last updated:** [January 28, 2016, 4:46am UTC](https://discuss.elastic.co/t/logstash-2-1-dynamically-altering-field-names/36845 "2016-01-28T04:46:52Z")
Hi, I've seen an issue in recent days with sending data to elasticsearch 2.1 where inbound messages contained an "uid" field which clashed with the type of the in-built meta-field "uid". This led to indexing failing, sha… --- ## [Javax.mail.internet.AddressException while using logstash kafka input/output plugin](https://discuss.elastic.co/t/javax-mail-internet-addressexception-while-using-logstash-kafka-input-output-plugin/40183)
**Author:** [@biswajit86](https://discuss.elastic.co/u/biswajit86)\ **Replies:** 6\ **Last updated:** [January 28, 2016, 2:04am UTC](https://discuss.elastic.co/t/javax-mail-internet-addressexception-while-using-logstash-kafka-input-output-plugin/40183 "2016-01-28T02:04:36Z")
Hi, While using logstash kafka input/output plugins , i am getting an exception in my logs before logstash starts up. This isi a brief snippet of the error Caused by: java.lang.ClassNotFoundException: javax.mail.inter… --- ## [What logs should I ship, and how should I filter them?](https://discuss.elastic.co/t/what-logs-should-i-ship-and-how-should-i-filter-them/40178)
**Author:** [@jerrac](https://discuss.elastic.co/u/jerrac)\ **Replies:** 2\ **Last updated:** [January 28, 2016, 12:29am UTC](https://discuss.elastic.co/t/what-logs-should-i-ship-and-how-should-i-filter-them/40178 "2016-01-28T00:29:07Z")
As part of upgrading to 2.x, I'm reevaluating what logs I ship, and how I filter them. I'll be working on figuring this all out on my own, of course. But if someone has already figured this out, and can quickly share it,… --- ## [How many redis input plugins Logstash can Handle?](https://discuss.elastic.co/t/how-many-redis-input-plugins-logstash-can-handle/40170)
**Author:** [@GustavoHoyer1](https://discuss.elastic.co/u/GustavoHoyer1)\ **Replies:** 4\ **Last updated:** [January 27, 2016, 11:24pm UTC](https://discuss.elastic.co/t/how-many-redis-input-plugins-logstash-can-handle/40170 "2016-01-27T23:24:27Z")
Hi guys, Is there a limit of how many redis input plugins can be declared on a single instance of Logstash? We've decided some time ago to split the redis keys per client/system so that every system had its own queue… --- ## [Sincedb, file input plugin used in multiple Logstash process](https://discuss.elastic.co/t/sincedb-file-input-plugin-used-in-multiple-logstash-process/40137)
**Author:** [@sharon.c](https://discuss.elastic.co/u/sharon.c)\ **Replies:** 0\ **Last updated:** [January 26, 2016, 3:58pm UTC](https://discuss.elastic.co/t/sincedb-file-input-plugin-used-in-multiple-logstash-process/40137 "2016-01-26T15:58:53Z")
I am using logstash 1.5.4. and logstash file input plugin to poll data from files. If I use multiple instance of logstash, portfile1.conf file is like this: input { file { path =\> "/…/access\_log2" sincedb\_path =\> "… --- ## [Mutate - Lowercase](https://discuss.elastic.co/t/mutate-lowercase/40271)
**Author:** [@bblank](https://discuss.elastic.co/u/bblank)\ **Replies:** 1\ **Last updated:** [January 27, 2016, 6:23pm UTC](https://discuss.elastic.co/t/mutate-lowercase/40271 "2016-01-27T18:23:23Z")
If I try to add a new field based on another existing field, and then make the new field value lowercase I get what looks like an array in my new field and it isn't lowercase. FILTER: mutate { add\_field =\> \[ "myind… --- ## [Eliminating \_grokparsefailure](https://discuss.elastic.co/t/eliminating--grokparsefailure/40033)
**Author:** [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Replies:** 4\ **Last updated:** [January 27, 2016, 4:13pm UTC](https://discuss.elastic.co/t/eliminating--grokparsefailure/40033 "2016-01-27T16:13:34Z")
Hello, all. I've been working on setting up my ELK stack for about the past two weeks. I must admit that I've cobbled together my instance based on various posts, how-to's, etc. Because of that, my configuration is p… --- ## [Launch simultaneous multiples logtsatash's files configuration](https://discuss.elastic.co/t/launch-simultaneous-multiples-logtsatashs-files-configuration/40108)
**Author:** [@Diyal](https://discuss.elastic.co/u/Diyal)\ **Replies:** 8\ **Last updated:** [January 27, 2016, 3:00pm UTC](https://discuss.elastic.co/t/launch-simultaneous-multiples-logtsatashs-files-configuration/40108 "2016-01-27T15:00:00Z")
Hi, I need to launch simultaneous multiples logtsatash's files configuration . Is it possible with this only command ? =\> bin/logstash agent -f logstashfile1.conf logstashfile2.conf logstashfile3.conf Thanks, --- ## [LogStash Stack Decsision](https://discuss.elastic.co/t/logstash-stack-decsision/40029)
**Author:** [@NekoNova](https://discuss.elastic.co/u/NekoNova)\ **Replies:** 5\ **Last updated:** [January 27, 2016, 12:18pm UTC](https://discuss.elastic.co/t/logstash-stack-decsision/40029 "2016-01-27T12:18:30Z")
Hello, I'm currently in the middle of setting up a LogStash stack for our company. We have multiple Ruby on Rails applications running, representing various environments such as our Production, staging etc. At the mo… --- ## [Logstash and syslog: logs from specific IP](https://discuss.elastic.co/t/logstash-and-syslog-logs-from-specific-ip/39976)
**Author:** [@maciej.kola](https://discuss.elastic.co/u/maciej.kola)\ **Replies:** 6\ **Last updated:** [January 27, 2016, 10:04am UTC](https://discuss.elastic.co/t/logstash-and-syslog-logs-from-specific-ip/39976 "2016-01-27T10:04:32Z")
Hi there, Firstly, let me just say that I am new in elk stack but I already think it's great solution. I use elk with syslog, to collect and search logs from multiple client systems. What I did so far: syslog is liste… --- ## [Convert values in kv-pairs](https://discuss.elastic.co/t/convert-values-in-kv-pairs/39904)
**Author:** [@intellion](https://discuss.elastic.co/u/intellion)\ **Replies:** 2\ **Last updated:** [January 27, 2016, 9:41am UTC](https://discuss.elastic.co/t/convert-values-in-kv-pairs/39904 "2016-01-27T09:41:08Z")
Hii there, I am trying to make some of our logs available in kibana using logstash-elasticsearch. the logs are not all structured. they do have some common patterns though. So far I could successfully extract date, lo… --- ## [Gracefully shutdown logstash](https://discuss.elastic.co/t/gracefully-shutdown-logstash/40132)
**Author:** [@shaharmor](https://discuss.elastic.co/u/shaharmor)\ **Replies:** 3\ **Last updated:** [January 27, 2016, 8:53am UTC](https://discuss.elastic.co/t/gracefully-shutdown-logstash/40132 "2016-01-27T08:53:14Z")
Hi, How can i gracefully shutdown Logstash without losing any events that it might still be processing? Basically what i want is to stop the input from processing and then wait until all events that are already under… --- ## [What is meant by Logstash Forwarder tail files from the beginning?](https://discuss.elastic.co/t/what-is-meant-by-logstash-forwarder-tail-files-from-the-beginning/40197)
**Author:** [@abinay](https://discuss.elastic.co/u/abinay)\ **Replies:** 1\ **Last updated:** [January 27, 2016, 8:24am UTC](https://discuss.elastic.co/t/what-is-meant-by-logstash-forwarder-tail-files-from-the-beginning/40197 "2016-01-27T08:24:52Z")
Logstash Forwarder tails a logfile from the beginning . So it must be sending the whole file to the Logstash whenever even a single line is added to the log file . So don't you think that it will cause a heavy traffic ov… --- ## [Duplicate items in elasticsearch](https://discuss.elastic.co/t/duplicate-items-in-elasticsearch/39765)
**Author:** [@nkoleff](https://discuss.elastic.co/u/nkoleff)\ **Replies:** 2\ **Last updated:** [January 27, 2016, 7:48am UTC](https://discuss.elastic.co/t/duplicate-items-in-elasticsearch/39765 "2016-01-27T07:48:46Z")
Hello, I'm using logstash forwarder to ship logs from centralized location to logstash servers and then to elasticsearch. Recently, I've found that I have the same documents but with different ids twice in my Kibana… --- ## [Can you do Or statments in a filter?](https://discuss.elastic.co/t/can-you-do-or-statments-in-a-filter/40179)
**Author:** [@jnpetty](https://discuss.elastic.co/u/jnpetty)\ **Replies:** 1\ **Last updated:** [January 27, 2016, 7:28am UTC](https://discuss.elastic.co/t/can-you-do-or-statments-in-a-filter/40179 "2016-01-27T07:28:30Z")
So I have two firewalls and a filter setup to process with a "if \[host\]" check is met. Is there a way to something like: if \[host\] =~ /192\\.168\\.100\\.1/ OR /192\\.168\\.100\\.2/ { } --- ## [Windows nxlog filtering](https://discuss.elastic.co/t/windows-nxlog-filtering/40061)
**Author:** [@jnpetty](https://discuss.elastic.co/u/jnpetty)\ **Replies:** 12\ **Last updated:** [January 27, 2016, 7:13am UTC](https://discuss.elastic.co/t/windows-nxlog-filtering/40061 "2016-01-27T07:13:18Z")
So now that I am receiving logs from my Windows server with NXLOG I an having trouble filtering it and making it useful. Most of the configurations below is from this example. https://gist.github.com/stuart-warren/672608… --- ## [VMware logging](https://discuss.elastic.co/t/vmware-logging/40036)
**Author:** [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Replies:** 2\ **Last updated:** [January 26, 2016, 9:03pm UTC](https://discuss.elastic.co/t/vmware-logging/40036 "2016-01-26T21:03:51Z")
Hey, all. I want to get logs from my VMware hosts and VMs into logstash. I am getting logs from the hosts though, in truth, most of what I see is rather meaningless. I'm not able to figure out how to get individual V… --- ## [Jdbc query error (postgresql)](https://discuss.elastic.co/t/jdbc-query-error-postgresql/40142)
**Author:** [@carmelom](https://discuss.elastic.co/u/carmelom)\ **Replies:** 0\ **Last updated:** [January 26, 2016, 5:09pm UTC](https://discuss.elastic.co/t/jdbc-query-error-postgresql/40142 "2016-01-26T17:09:44Z")
I connected to the db through jdbc I am using this query : select \*, (end\_time - start\_time) as duration from system.connection\_log order by start\_time desc it is parsing most of the field, but for duration field I a… --- ## [File plugin path setup to multiple files](https://discuss.elastic.co/t/file-plugin-path-setup-to-multiple-files/40040)
**Author:** [@yahoo](https://discuss.elastic.co/u/yahoo)\ **Replies:** 2\ **Last updated:** [January 26, 2016, 4:46pm UTC](https://discuss.elastic.co/t/file-plugin-path-setup-to-multiple-files/40040 "2016-01-26T16:46:07Z")
I have a process that generates log files aaalog\_20160101 aaalog\_20160102 aaalog\_20160103 This config file did not work: file { path =\> \[ "D:\\a\\b\\d\\e\\f\\aaalog\_\*" \] start\_position =\> beginning … --- ## [Logstash elasticsearch output retry logic](https://discuss.elastic.co/t/logstash-elasticsearch-output-retry-logic/40133)
**Author:** [@shaharmor](https://discuss.elastic.co/u/shaharmor)\ **Replies:** 0\ **Last updated:** [January 26, 2016, 3:34pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-retry-logic/40133 "2016-01-26T15:34:22Z")
Hi, When multiple hosts are configured in the elasticsearch-ouput plugin configuration, does Logstash retry the action to the same host that it originally failed at or to a different one? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1196) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1198)