# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1208 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1209 --- ## [Start log stash as windows service without nssm](https://discuss.elastic.co/t/start-log-stash-as-windows-service-without-nssm/32430)
**Author:** [@DomsRoberts](https://discuss.elastic.co/u/DomsRoberts)\ **Replies:** 1\ **Last updated:** [December 17, 2015, 3:15pm UTC](https://discuss.elastic.co/t/start-log-stash-as-windows-service-without-nssm/32430 "2015-12-17T15:15:47Z")
I want to install logstash as a windows service on my windows server. But, for security reasons I'm not able to download and use Nssm. Are the instructions anywhere to do that? All the info I can find involves using NSS… --- ## [Lumberjack Client write error](https://discuss.elastic.co/t/lumberjack-client-write-error/37479)
**Author:** [@treehouse](https://discuss.elastic.co/u/treehouse)\ **Replies:** 0\ **Last updated:** [December 17, 2015, 2:58pm UTC](https://discuss.elastic.co/t/lumberjack-client-write-error/37479 "2015-12-17T14:58:28Z")
hey guys my stack had been working fine until just recently... i have two logstash instances, one accepting courier input which uses the lumberjack output, and one accepting lumberjack input... the first logstash insta… --- ## [Logstash Configuration Doubt](https://discuss.elastic.co/t/logstash-configuration-doubt/37461)
**Author:** [@shabinhashim](https://discuss.elastic.co/u/shabinhashim)\ **Replies:** 3\ **Last updated:** [December 17, 2015, 1:39pm UTC](https://discuss.elastic.co/t/logstash-configuration-doubt/37461 "2015-12-17T13:39:08Z")
HI, I need a setup to monitor logs from 100 different machines. I checked beats framework and it seems i cannot install a package in the machine. Also does beats support Multi-line? Also as per this page https://www.el… --- ## [Create an array from a nested XML field](https://discuss.elastic.co/t/create-an-array-from-a-nested-xml-field/37377)
**Author:** [@Shayleen](https://discuss.elastic.co/u/Shayleen)\ **Replies:** 3\ **Last updated:** [December 17, 2015, 11:18am UTC](https://discuss.elastic.co/t/create-an-array-from-a-nested-xml-field/37377 "2015-12-17T11:18:50Z")
Hi, I'm quite new in ELK and I've encountered a problem almost imposible to break for me. I have an XML like this: \< root \> \< field\_A\> \< subfield\_A \>8100291240 \< subfield\_B \>3 \< subfield\_C \>6000436355 \< subfield\_D \> \<… --- ## [Input tails off over time?](https://discuss.elastic.co/t/input-tails-off-over-time/37455)
**Author:** [@paul.mason](https://discuss.elastic.co/u/paul.mason)\ **Replies:** 0\ **Last updated:** [December 17, 2015, 9:53am UTC](https://discuss.elastic.co/t/input-tails-off-over-time/37455 "2015-12-17T09:53:03Z")
Hi Hoping you can help me understand what might be happening with our new Logstash installation. There are a number of devices sending logs, when we start the service everything is processed by Logstash and outputted… --- ## [Log4j input with empty members](https://discuss.elastic.co/t/log4j-input-with-empty-members/37221)
**Author:** [@David\_Pocivalnik](https://discuss.elastic.co/u/David_Pocivalnik)\ **Replies:** 4\ **Last updated:** [December 17, 2015, 9:07am UTC](https://discuss.elastic.co/t/log4j-input-with-empty-members/37221 "2015-12-17T09:07:16Z")
I'm trying to use Logstash to index my log files created by log4j. I'm using the following versions: Logstash: 2.1.1 ES: 2.1.0 log4j: 1.2.17 I followed the recommendation found at http://stackoverflow.com/a/26227002 … --- ## [Logstash mysql using jdbc on windows: library error](https://discuss.elastic.co/t/logstash-mysql-using-jdbc-on-windows-library-error/37313)
**Author:** [@krushnat\_khawale](https://discuss.elastic.co/u/krushnat_khawale)\ **Replies:** 9\ **Last updated:** [December 17, 2015, 8:45am UTC](https://discuss.elastic.co/t/logstash-mysql-using-jdbc-on-windows-library-error/37313 "2015-12-17T08:45:46Z")
I want to insert data from mysql to elasticsearch using logstash. when I am running command, logstash -f myconf.conf I am getting following error: D:\\Users\\admin\\setups\\logstash-2.1.1\\bin\>logstash -f myconf.conf io/… --- ## [Why can't i install http\_poller?](https://discuss.elastic.co/t/why-cant-i-install-http-poller/37350)
**Author:** [@swelltt0576](https://discuss.elastic.co/u/swelltt0576)\ **Replies:** 2\ **Last updated:** [December 17, 2015, 4:00am UTC](https://discuss.elastic.co/t/why-cant-i-install-http-poller/37350 "2015-12-17T04:00:05Z")
when i install http\_poller, the error occured. \[mlink@vm107 logstash-2.1.1\]$ bin/plugin install logstash-input-http\_poller Validating logstash-input-http\_poller Unable to download data from https://rubygems.org - Con… --- ## [When i use logstash-gelf to provide log messages, i find some messages lost](https://discuss.elastic.co/t/when-i-use-logstash-gelf-to-provide-log-messages-i-find-some-messages-lost/37211)
**Author:** [@swelltt0576](https://discuss.elastic.co/u/swelltt0576)\ **Replies:** 2\ **Last updated:** [December 17, 2015, 12:51am UTC](https://discuss.elastic.co/t/when-i-use-logstash-gelf-to-provide-log-messages-i-find-some-messages-lost/37211 "2015-12-17T00:51:59Z")
i sent 100,000 message, but there are only about 80,000 messages in the logstash. what happend? this is my log4j2 configuration: \ **Author:** [@gaurav1424](https://discuss.elastic.co/u/gaurav1424)\ **Replies:** 4\ **Last updated:** [December 16, 2015, 10:37pm UTC](https://discuss.elastic.co/t/parsing-syslog-messages-with-different-network-equipment-providers/37401 "2015-12-16T22:37:57Z") Hello All, I am currently working on problem which involves setting up pipeline with Logstash for syslog events. Here are the requirements 1: I need to parse syslog events , without identifying / marking them as syslo… --- ## [ELK for global dashboard](https://discuss.elastic.co/t/elk-for-global-dashboard/37384)
**Author:** [@fluidjeel](https://discuss.elastic.co/u/fluidjeel)\ **Replies:** 1\ **Last updated:** [December 16, 2015, 8:25pm UTC](https://discuss.elastic.co/t/elk-for-global-dashboard/37384 "2015-12-16T20:25:05Z")
I am looking to implement a global dashboard for all the Devops tools (JIRA, Jenkins, ASGARD, Sonarqube, Nexus, Sensu, A bunch of In house tools with API exposed) i am looking at various features of the ELK stack. Most o… --- ## [Elasticsearch Output retry strategy](https://discuss.elastic.co/t/elasticsearch-output-retry-strategy/37403)
**Author:** [@talvey](https://discuss.elastic.co/u/talvey)\ **Replies:** 0\ **Last updated:** [December 16, 2015, 8:02pm UTC](https://discuss.elastic.co/t/elasticsearch-output-retry-strategy/37403 "2015-12-16T20:02:22Z")
I'm looking for suggestions on how best to prevent dropping Logstash messages in the Elasticsearch output plugin. We run a fairly large ES (1.7) cluster fed exclusively by LS(1.5.4) -- on average we process 30k message… --- ## [Logstash Ruby exceptions](https://discuss.elastic.co/t/logstash-ruby-exceptions/37391)
**Author:** [@Omer\_Uludag](https://discuss.elastic.co/u/Omer_Uludag)\ **Replies:** 4\ **Last updated:** [December 16, 2015, 7:25pm UTC](https://discuss.elastic.co/t/logstash-ruby-exceptions/37391 "2015-12-16T19:25:56Z")
Hello together, I have the following Logstash pipeline: input { file { path =\> "/test/home.w10\_vinfo-sg-war.log.\*" start\_position =\> "beginning" exclude =\> "\*.gz" exclude =\> "\*.snappy" } } filter{ xml { … --- ## [Meaning of \*\* in filter input context](https://discuss.elastic.co/t/meaning-of-in-filter-input-context/37348)
**Author:** [@nerdstrike](https://discuss.elastic.co/u/nerdstrike)\ **Replies:** 3\ **Last updated:** [December 16, 2015, 2:16pm UTC](https://discuss.elastic.co/t/meaning-of-in-filter-input-context/37348 "2015-12-16T14:16:33Z")
While attempting to sort out my recalcitrant ELK setup, I am finding scattered examples of logstash filter patterns like: file { path =\> "/var/log/\*\*/\*" } Does this have any specific meaning? The problem at han… --- ## [Add\_field parameter in filter plugins](https://discuss.elastic.co/t/add-field-parameter-in-filter-plugins/37333)
**Author:** [@srinivas](https://discuss.elastic.co/u/srinivas)\ **Replies:** 3\ **Last updated:** [December 16, 2015, 11:37am UTC](https://discuss.elastic.co/t/add-field-parameter-in-filter-plugins/37333 "2015-12-16T11:37:08Z")
Hi, Can anyone explain what does add\_field do ? filter { mutate { add\_field =\> { "COLUMN2" =\> "%{COLUMN1}" } } } What is the difference between add\_field present in mutate, clone, kv and grok plugins --- ## [Logstash jdbc plugin - filter issue](https://discuss.elastic.co/t/logstash-jdbc-plugin-filter-issue/37342)
**Author:** [@srinivas](https://discuss.elastic.co/u/srinivas)\ **Replies:** 1\ **Last updated:** [December 16, 2015, 11:34am UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-filter-issue/37342 "2015-12-16T11:34:36Z")
I am using below filter to copy content in COLUMN1 into COLUMN2. filter { mutate { add\_field =\> { "COLUMN2" =\> "%{COLUMN1 }" } } } But if any value in COLUMN1 is null then its copying the value as %{COLUMN1 … --- ## [Logstash service keeps crashing](https://discuss.elastic.co/t/logstash-service-keeps-crashing/37330)
**Author:** [@simroo](https://discuss.elastic.co/u/simroo)\ **Replies:** 7\ **Last updated:** [December 16, 2015, 11:25am UTC](https://discuss.elastic.co/t/logstash-service-keeps-crashing/37330 "2015-12-16T11:25:44Z")
Hello I have som problems with logstash and it keeps crashing. Im not sure why, i thought it was a premisson problem but after chmod 777 on the files, the problem still prisets. Note that this is during installation on… --- ## [Multiple individual date & time fields from CSV into ES](https://discuss.elastic.co/t/multiple-individual-date-time-fields-from-csv-into-es/37305)
**Author:** [@zarec](https://discuss.elastic.co/u/zarec)\ **Replies:** 2\ **Last updated:** [December 16, 2015, 7:11am UTC](https://discuss.elastic.co/t/multiple-individual-date-time-fields-from-csv-into-es/37305 "2015-12-16T07:11:50Z")
Howdy all, I've spent the past few days reading forums, researching guides and using trial & error to get my head around the following situation, and now I realise I need help. I want to be able to display/work on date… --- ## [Rename a field that has variable in name](https://discuss.elastic.co/t/rename-a-field-that-has-variable-in-name/37284)
**Author:** [@spuder](https://discuss.elastic.co/u/spuder)\ **Replies:** 1\ **Last updated:** [December 16, 2015, 6:55am UTC](https://discuss.elastic.co/t/rename-a-field-that-has-variable-in-name/37284 "2015-12-16T06:55:54Z")
What would be the best way to remove a nested field that has a variable in its name? For example, the logs I'm parsing as json { "foo": { "bar(987234)": "derp" } } Represented as nested field \[foo\]\[bar(23… --- ## [Weird mutate gsub behaviour double quotes](https://discuss.elastic.co/t/weird-mutate-gsub-behaviour-double-quotes/37307)
**Author:** [@Raggyman](https://discuss.elastic.co/u/Raggyman)\ **Replies:** 0\ **Last updated:** [December 16, 2015, 6:25am UTC](https://discuss.elastic.co/t/weird-mutate-gsub-behaviour-double-quotes/37307 "2015-12-16T06:25:18Z")
Have found myself in the position where I have malformed json, that won't be changed for a while. Basically I have "realip": ""111.111.111.111" where I have an incidental extra double quote at the front. This is … --- ## [Logstash inserting 2 things into elasticsearch per log](https://discuss.elastic.co/t/logstash-inserting-2-things-into-elasticsearch-per-log/37295)
**Author:** [@mtopolski](https://discuss.elastic.co/u/mtopolski)\ **Replies:** 1\ **Last updated:** [December 16, 2015, 4:11am UTC](https://discuss.elastic.co/t/logstash-inserting-2-things-into-elasticsearch-per-log/37295 "2015-12-16T04:11:10Z")
I'm using Monolog with PHP, using beaver to forward the logs to a redis queue, and redis feeds the log in queue to logstash. I've monitored every step up to logstash and everything looks normal. When logstash gets its lo… --- ## [Logstash 2.1.0 stop creating raw fields](https://discuss.elastic.co/t/logstash-2-1-0-stop-creating-raw-fields/36598)
**Author:** [@danilo\_gimenez\_ramos](https://discuss.elastic.co/u/danilo_gimenez_ramos)\ **Replies:** 7\ **Last updated:** [December 15, 2015, 10:43pm UTC](https://discuss.elastic.co/t/logstash-2-1-0-stop-creating-raw-fields/36598 "2015-12-15T22:43:16Z")
Hello, I've upgrade all my client servers with Logstash 2.1.0 and then I wipe out all data in /var/lib/elasticsearch/ on Elasticsearch server in order to start form scratch with indices. After that I can't see the raw … --- ## [Merging dynamic updates triggered a conflict](https://discuss.elastic.co/t/merging-dynamic-updates-triggered-a-conflict/37276)
**Author:** [@spuder](https://discuss.elastic.co/u/spuder)\ **Replies:** 3\ **Last updated:** [December 15, 2015, 10:28pm UTC](https://discuss.elastic.co/t/merging-dynamic-updates-triggered-a-conflict/37276 "2015-12-15T22:28:40Z")
I'm using elasticsearch 2.1 and logstash 2.1 to parse logs. One log line in particular always throws an error in logstash. :response=\>{ "create"=\>{ "\_index"=\>"logstash-2015.12.15", "\_type"=\>"logs", "\_id"=\>"A… --- ## [Logstash.conf encoding issue?](https://discuss.elastic.co/t/logstash-conf-encoding-issue/37283)
**Author:** [@Anil\_G](https://discuss.elastic.co/u/Anil_G)\ **Replies:** 0\ **Last updated:** [December 15, 2015, 9:59pm UTC](https://discuss.elastic.co/t/logstash-conf-encoding-issue/37283 "2015-12-15T21:59:46Z")
I am using logstash 1.5.5. I am using "sed" command to replace ELASTICSEARCH\_HOST variable in my logstash.conf file. But when I start logstash with such a conf file, I see that logstash cannot talk to Elasticsearch. Belo… --- ## [If conditions in Ruby code in Logstash](https://discuss.elastic.co/t/if-conditions-in-ruby-code-in-logstash/37274)
**Author:** [@Omer\_Uludag](https://discuss.elastic.co/u/Omer_Uludag)\ **Replies:** 2\ **Last updated:** [December 15, 2015, 7:29pm UTC](https://discuss.elastic.co/t/if-conditions-in-ruby-code-in-logstash/37274 "2015-12-15T19:29:07Z")
Hello together, I am new in the Elasticsearch Ecosystem. For that reason, I have unfortunately some problems with the syntax of Ruby in Logstash. I have the following snippet code: if \[Clat\] != '' { ruby { code… --- ## [Rolling Logs in Logstash?](https://discuss.elastic.co/t/rolling-logs-in-logstash/37268)
**Author:** [@parthu](https://discuss.elastic.co/u/parthu)\ **Replies:** 1\ **Last updated:** [December 15, 2015, 6:26pm UTC](https://discuss.elastic.co/t/rolling-logs-in-logstash/37268 "2015-12-15T18:26:21Z")
Every time there's an event in Logstash, I ensure that it's logged in a log file. But the problem is that .. the log file keeps growing every day. I need to segregrate logs by the day so I could delete old logs as per my… --- ## [Fields and Space](https://discuss.elastic.co/t/fields-and-space/36756)
**Author:** [@Jackal9301](https://discuss.elastic.co/u/Jackal9301)\ **Replies:** 8\ **Last updated:** [December 15, 2015, 3:41pm UTC](https://discuss.elastic.co/t/fields-and-space/36756 "2015-12-15T15:41:09Z")
I understand how to create fields in Logstash however once they go into Elasticsearch and into Kibana I get .raw versions and the space difference between my indexes before the new ELK 2+ stack were significantly smalle… --- ## [Transfering logs by Logstash to Logstash - help needed](https://discuss.elastic.co/t/transfering-logs-by-logstash-to-logstash-help-needed/37218)
**Author:** [@daniel2](https://discuss.elastic.co/u/daniel2)\ **Replies:** 4\ **Last updated:** [December 15, 2015, 2:50pm UTC](https://discuss.elastic.co/t/transfering-logs-by-logstash-to-logstash-help-needed/37218 "2015-12-15T14:50:35Z")
Hey guys! I have a problem with configuring Logstash to send logs to Logstash on a different server. I was able to send logs from Agent1 server by Filebeat to Logstash on Agent2 server, but i don't know how to send logs … --- ## [Is it possible to connect logstash with elasticsearch master node?](https://discuss.elastic.co/t/is-it-possible-to-connect-logstash-with-elasticsearch-master-node/37115)
**Author:** [@jffifa](https://discuss.elastic.co/u/jffifa)\ **Replies:** 1\ **Last updated:** [December 15, 2015, 11:23am UTC](https://discuss.elastic.co/t/is-it-possible-to-connect-logstash-with-elasticsearch-master-node/37115 "2015-12-15T11:23:43Z")
When I read hints about elasticsearch output plugin I noticed that It is important to exclude dedicated master nodes from the hosts list to prevent LS from sending bulk requests to the master nodes. So this parameter s… --- ## [Split csv column to several fields](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927)
**Author:** [@eprst](https://discuss.elastic.co/u/eprst)\ **Replies:** 15\ **Last updated:** [December 15, 2015, 10:25am UTC](https://discuss.elastic.co/t/split-csv-column-to-several-fields/36927 "2015-12-15T10:25:01Z")
i use logstash to import from csv to elastic. the problem is that one of the fields contains the string formatted as "int|string|string" and i need to parse this and put into nested field. what are possible solutions for… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1207) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1209)