# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1225 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1226 --- ## [\_grokparsefailure and missing events when using Logstash Forwarder](https://discuss.elastic.co/t/-grokparsefailure-and-missing-events-when-using-logstash-forwarder/33523)
**Author:** [@incogniro](https://discuss.elastic.co/u/incogniro)\ **Replies:** 2\ **Last updated:** [November 2, 2015, 2:54pm UTC](https://discuss.elastic.co/t/-grokparsefailure-and-missing-events-when-using-logstash-forwarder/33523 "2015-11-02T14:54:40Z")
I have a strange issue in which all events sent to Logstash via Logstash Forwarder using the Lumberjack input plugin are not indexed at all (completely missing, no indexes created in ElasticSearch) and a subset of the ev… --- ## [Std out without output configured](https://discuss.elastic.co/t/std-out-without-output-configured/33498)
**Author:** [@Stewart\_Gray](https://discuss.elastic.co/u/Stewart_Gray)\ **Replies:** 2\ **Last updated:** [November 2, 2015, 2:45pm UTC](https://discuss.elastic.co/t/std-out-without-output-configured/33498 "2015-11-02T14:45:01Z")
After upgrading to logstash 2 on friday all of the messages seem to be getting logged to my logstash.log file (which has filled up the drive). My output is only to elasticsearch and not to STDOUT so I'm not sure why this… --- ## [Logstash split xml into array](https://discuss.elastic.co/t/logstash-split-xml-into-array/33516)
**Author:** [@buinauskas\_evaldas](https://discuss.elastic.co/u/buinauskas_evaldas)\ **Replies:** 0\ **Last updated:** [November 2, 2015, 12:44pm UTC](https://discuss.elastic.co/t/logstash-split-xml-into-array/33516 "2015-11-02T12:44:07Z")
Is it possible to convert xml into array of objects using logstash? That'd be my sample document: { "Title" : "My blog title", "Body" : "My first post ever", "Metadata" : "\\\1\\ **Author:** [@reject](https://discuss.elastic.co/u/reject)\ **Replies:** 2\ **Last updated:** [November 2, 2015, 9:51am UTC](https://discuss.elastic.co/t/error-the-setting-host-in-plugin-elasticsearch-is-obsolete-and-is-no-longer-available-please-use-the-hosts-setting-instead-you-can-specify-multiple-entries-separated-by-comma-in-host-port-format/33489 "2015-11-02T09:51:52Z") /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash\_test.conf test.conf: input { stdin{} } output { elasticsearch { host =\> localhost } stdout { codec =\> rubydebug } } why this error?plz.thank… --- ## [Logstash going OOM while dumping data from ES to CSV](https://discuss.elastic.co/t/logstash-going-oom-while-dumping-data-from-es-to-csv/33491)
**Author:** [@ankmathur14](https://discuss.elastic.co/u/ankmathur14)\ **Replies:** 2\ **Last updated:** [November 2, 2015, 9:49am UTC](https://discuss.elastic.co/t/logstash-going-oom-while-dumping-data-from-es-to-csv/33491 "2015-11-02T09:49:03Z")
Hello all, I have 60 timestamp wise indexes in my cluster (total containing 200 million documents (150 GB) ) I am trying to export some data (around 15 million) from these indexes to a csv file using logstash. Logstash… --- ## [Logs are getting parsed multiple times due to ?\[31mA plugin. can some one help me on this issue](https://discuss.elastic.co/t/logs-are-getting-parsed-multiple-times-due-to-31ma-plugin-can-some-one-help-me-on-this-issue/33293)
**Author:** [@sree1](https://discuss.elastic.co/u/sree1)\ **Replies:** 4\ **Last updated:** [November 2, 2015, 9:14am UTC](https://discuss.elastic.co/t/logs-are-getting-parsed-multiple-times-due-to-31ma-plugin-can-some-one-help-me-on-this-issue/33293 "2015-11-02T09:14:53Z")
I am newbie to elk . currently,I am working with logstash-2.0.0-beta2 . whenever, I will run my logstash conf file .I am getting following errors. 1)io/console not supported; tty will not be manipulated 2)Error: No such … --- ## [Unable to parse log containing UNICODE characters and ANSI colour codes using grok](https://discuss.elastic.co/t/unable-to-parse-log-containing-unicode-characters-and-ansi-colour-codes-using-grok/33487)
**Author:** [@safiyat](https://discuss.elastic.co/u/safiyat)\ **Replies:** 0\ **Last updated:** [November 2, 2015, 7:58am UTC](https://discuss.elastic.co/t/unable-to-parse-log-containing-unicode-characters-and-ansi-colour-codes-using-grok/33487 "2015-11-02T07:58:31Z")
I am trying to parse the following line: 2015-09-17 17:44:49.663 ^\[\[00;32mDEBUG oslo\_concurrency.lockutils \[^\[\[00;36m-^\[\[00;32m\] ^\[\[01;35m^\[\[00;32mAcquired semaphore "singleton\_lock"^\[\[00m ^\[\[00;33mfrom (pid=30534) loc… --- ## [Rabbitmq logstash connection error](https://discuss.elastic.co/t/rabbitmq-logstash-connection-error/33479)
**Author:** [@krish0608](https://discuss.elastic.co/u/krish0608)\ **Replies:** 4\ **Last updated:** [November 2, 2015, 6:55am UTC](https://discuss.elastic.co/t/rabbitmq-logstash-connection-error/33479 "2015-11-02T06:55:39Z")
This is my logstash.conf file. I want my output in rabbitmq server... but it showing me RABBITMQ connection Error.please help me out. input { file { path =\> "/var/log/" type =\> "syslog" } rabbitmq { hos… --- ## [Is it possible to setup multiple logstash indexer service for same index?](https://discuss.elastic.co/t/is-it-possible-to-setup-multiple-logstash-indexer-service-for-same-index/33355)
**Author:** [@macymin](https://discuss.elastic.co/u/macymin)\ **Replies:** 3\ **Last updated:** [November 2, 2015, 6:51am UTC](https://discuss.elastic.co/t/is-it-possible-to-setup-multiple-logstash-indexer-service-for-same-index/33355 "2015-11-02T06:51:28Z")
we have 2 index which has huge data feeding. the forwarder sends data to redis then indexer. by checking the LLEN of the KEYS in redis-cli, the number keeps increasing which means the indexer is not picking up the fil… --- ## [Logstash Jdbc-input Field mapping issue](https://discuss.elastic.co/t/logstash-jdbc-input-field-mapping-issue/33358)
**Author:** [@Hoffin](https://discuss.elastic.co/u/Hoffin)\ **Replies:** 2\ **Last updated:** [November 2, 2015, 6:06am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-field-mapping-issue/33358 "2015-11-02T06:06:02Z")
In Lostash JDBC input all fields defaulted to String mapping. Auto field mapping is not available. How achieve the field map exactly match to to mysql table mapping. --- ## [Http polar authentication](https://discuss.elastic.co/t/http-polar-authentication/33366)
**Author:** [@ushadatt](https://discuss.elastic.co/u/ushadatt)\ **Replies:** 2\ **Last updated:** [November 2, 2015, 5:44am UTC](https://discuss.elastic.co/t/http-polar-authentication/33366 "2015-11-02T05:44:59Z")
I am using http polar to parse a json API with authentication required at the beginning. Its showing an error that the URL is incorrect. And when I am checking my config file using --configtest, its showing the conf file… --- ## [Conditional statement one of two patterns](https://discuss.elastic.co/t/conditional-statement-one-of-two-patterns/33412)
**Author:** [@ruchii](https://discuss.elastic.co/u/ruchii)\ **Replies:** 3\ **Last updated:** [November 1, 2015, 8:33pm UTC](https://discuss.elastic.co/t/conditional-statement-one-of-two-patterns/33412 "2015-11-01T20:33:01Z")
I am interested in two types of lines in log file as shown below. What options will be most effective? 20151020 00:08:00 0 0 0 0 0 0 0 100 0 0 0 1615 855 1 1056 0 0.95 1.09 1.06 1 0 264688028 27462076 237225952 0 10… --- ## [When is the data "acked" to LSF/FileBeat during output progressing](https://discuss.elastic.co/t/when-is-the-data-acked-to-lsf-filebeat-during-output-progressing/33464)
**Author:** [@normp](https://discuss.elastic.co/u/normp)\ **Replies:** 0\ **Last updated:** [November 1, 2015, 1:09pm UTC](https://discuss.elastic.co/t/when-is-the-data-acked-to-lsf-filebeat-during-output-progressing/33464 "2015-11-01T13:09:20Z")
Quick question on the reliable delivery of a log from LSF/FileBeat to a logstash output. At what point is it ACKed. For example, if you are writing to the file output, does it get acked as soon as it it written or only w… --- ## [Specifying with Elasticsearch template and file to use](https://discuss.elastic.co/t/specifying-with-elasticsearch-template-and-file-to-use/33362)
**Author:** [@kelv1n](https://discuss.elastic.co/u/kelv1n)\ **Replies:** 2\ **Last updated:** [October 31, 2015, 11:57pm UTC](https://discuss.elastic.co/t/specifying-with-elasticsearch-template-and-file-to-use/33362 "2015-10-31T23:57:08Z")
Hi Guys This is probably a newbie config mistake. When specifying a custom Elasticsearch template for an output, it appears the index and template\_name need to be the same.. so if I had a generic template that I wan… --- ## [Getting 401 response while sending log to ELK using http](https://discuss.elastic.co/t/getting-401-response-while-sending-log-to-elk-using-http/33370)
**Author:** [@aniamin](https://discuss.elastic.co/u/aniamin)\ **Replies:** 1\ **Last updated:** [October 31, 2015, 11:13pm UTC](https://discuss.elastic.co/t/getting-401-response-while-sending-log-to-elk-using-http/33370 "2015-10-31T23:13:50Z")
I tried it from ELk server using command: curl -H "content-type: application/json" -XPUT 'http://elk.example.net:5555' -d '{ "post\_date" : "2015-10-29T15:20:34","message" : "Nowshad\_amin" }' -v Response: \* Rebuilt URL to… --- ## [\_grokparsefailure when parsing IIS logs if the %{URIPATH:page} has square brackets \[\] in it](https://discuss.elastic.co/t/-grokparsefailure-when-parsing-iis-logs-if-the-uripath-page-has-square-brackets-in-it/33416)
**Author:** [@mc1903](https://discuss.elastic.co/u/mc1903)\ **Replies:** 2\ **Last updated:** [October 31, 2015, 10:30pm UTC](https://discuss.elastic.co/t/-grokparsefailure-when-parsing-iis-logs-if-the-uripath-page-has-square-brackets-in-it/33416 "2015-10-31T22:30:15Z")
Hi, 3rd day after I installed ELK - so I am still very new at all this. I have noticed that some of my IIS log files have \[\] (square brackets?) in the %{URIPATH:page} and this is causing a \_grokparsefailure. My GRO… --- ## [Http plugin of logstash](https://discuss.elastic.co/t/http-plugin-of-logstash/33257)
**Author:** [@aniamin](https://discuss.elastic.co/u/aniamin)\ **Replies:** 12\ **Last updated:** [October 31, 2015, 6:15pm UTC](https://discuss.elastic.co/t/http-plugin-of-logstash/33257 "2015-10-31T18:15:53Z")
Hi, I am using ELK stack for Log Centralisation. I have written an ansible role (logstash-forwarder) which helps my application to send log to ELK using certificate. My ELK is currently supporting lumberjack. One of my a… --- ## [Logstash output in rabbitmq queue](https://discuss.elastic.co/t/logstash-output-in-rabbitmq-queue/33365)
**Author:** [@krish0608](https://discuss.elastic.co/u/krish0608)\ **Replies:** 2\ **Last updated:** [October 31, 2015, 5:46am UTC](https://discuss.elastic.co/t/logstash-output-in-rabbitmq-queue/33365 "2015-10-31T05:46:21Z")
please guys help me out...i want my output in a rabbitmq Queue.while running this code am getting this error. bin/logstash -f /home/krishna/Downloads/logstash-2.0.0-rc1/bin/logstash.conf The error reported is: Co… --- ## [Learning logstash groksample.log file mia?](https://discuss.elastic.co/t/learning-logstash-groksample-log-file-mia/33280)
**Author:** [@doghousesoftware](https://discuss.elastic.co/u/doghousesoftware)\ **Replies:** 3\ **Last updated:** [October 31, 2015, 1:03am UTC](https://discuss.elastic.co/t/learning-logstash-groksample-log-file-mia/33280 "2015-10-31T01:03:41Z")
looking for the precious "groksample.log" file. pointer? --- ## [Logstash Plugin 1.5.3 offline installation](https://discuss.elastic.co/t/logstash-plugin-1-5-3-offline-installation/32899)
**Author:** [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Replies:** 4\ **Last updated:** [October 31, 2015, 12:30am UTC](https://discuss.elastic.co/t/logstash-plugin-1-5-3-offline-installation/32899 "2015-10-31T00:30:17Z")
As far i am aware that pre logstash 2.0 , none of the version supports offline installation of logstash plugins . Is it possible if i install the logstash plugin using plugin manager in a machine having internet connecti… --- ## [Logstast apache configuration](https://discuss.elastic.co/t/logstast-apache-configuration/33343)
**Author:** [@krish0608](https://discuss.elastic.co/u/krish0608)\ **Replies:** 11\ **Last updated:** [October 30, 2015, 11:26pm UTC](https://discuss.elastic.co/t/logstast-apache-configuration/33343 "2015-10-30T23:26:28Z")
Guysss please help me out...i have created one configuration file with the code input { file { path =\> "/var/log/messages" type =\> "syslog" } file { path =\> "/var/log/apache/access.log" type =\> … --- ## [Configuring Input file on Windows](https://discuss.elastic.co/t/configuring-input-file-on-windows/33297)
**Author:** [@Didier](https://discuss.elastic.co/u/Didier)\ **Replies:** 5\ **Last updated:** [October 30, 2015, 8:16pm UTC](https://discuss.elastic.co/t/configuring-input-file-on-windows/33297 "2015-10-30T20:16:07Z")
Hello, I am trying to use an input file to feed a log file into Logstash on a Windows 2008 machine. However, it doesn't seem to be working properly. My configuration file looks like this: input { file { path =\> … --- ## [Send LSF to Redis to Logstash](https://discuss.elastic.co/t/send-lsf-to-redis-to-logstash/33294)
**Author:** [@Kaufusihm](https://discuss.elastic.co/u/Kaufusihm)\ **Replies:** 11\ **Last updated:** [October 30, 2015, 7:56pm UTC](https://discuss.elastic.co/t/send-lsf-to-redis-to-logstash/33294 "2015-10-30T19:56:33Z")
I am able to send windows server 2012 logs to logstash with lumberjack with the following configs below, but I want to now send to redis from windows server 2012 and then to logstash. lumberjack config: { "networ… --- ## [Subsequently get only first line of multi-line grok](https://discuss.elastic.co/t/subsequently-get-only-first-line-of-multi-line-grok/33381)
**Author:** [@RobT](https://discuss.elastic.co/u/RobT)\ **Replies:** 1\ **Last updated:** [October 30, 2015, 7:11pm UTC](https://discuss.elastic.co/t/subsequently-get-only-first-line-of-multi-line-grok/33381 "2015-10-30T19:11:29Z")
So, I'm retrieving a multi-line input just fine and storing in message. However, for throttling purposes I'd like to also just consider the first line so I'm trying to break the multi-line message up. I'm trying this: g… --- ## [Advice needed on configuration approach](https://discuss.elastic.co/t/advice-needed-on-configuration-approach/33319)
**Author:** [@i8beef](https://discuss.elastic.co/u/i8beef)\ **Replies:** 3\ **Last updated:** [October 30, 2015, 5:53pm UTC](https://discuss.elastic.co/t/advice-needed-on-configuration-approach/33319 "2015-10-30T17:53:33Z")
Greetings all. I'm starting down a path of implementing ELK + NXLog for various reasons. Our current use case involves wanting to monitor: Certain Windows event log events (Really ASP.NET related ones, but I haven't f… --- ## [Logstash Input JDBC Error - 2.0 - Sql Server](https://discuss.elastic.co/t/logstash-input-jdbc-error-2-0-sql-server/33178)
**Author:** [@canoycano](https://discuss.elastic.co/u/canoycano)\ **Replies:** 7\ **Last updated:** [October 30, 2015, 5:07pm UTC](https://discuss.elastic.co/t/logstash-input-jdbc-error-2-0-sql-server/33178 "2015-10-30T17:07:40Z")
Hello all I've got a pretty major issue with the logstash 2.0 jdbc input plugin, It seems to not connect to my jdbc input for sqljdbc4.jar, when I run logstash I get the following error message "The error reported is: … --- ## [Logstash & Redis](https://discuss.elastic.co/t/logstash-redis/33332)
**Author:** [@genebean](https://discuss.elastic.co/u/genebean)\ **Replies:** 3\ **Last updated:** [October 30, 2015, 5:04pm UTC](https://discuss.elastic.co/t/logstash-redis/33332 "2015-10-30T17:04:09Z")
Can anyone suggest a way to read entries from Redis faster? I seem to be inserting them faster than I can read them as I have 8 Logstash indexers running and either they can't read fast enough or there is a bottleneck th… --- ## [Logstash not ignoring syslog event date](https://discuss.elastic.co/t/logstash-not-ignoring-syslog-event-date/33329)
**Author:** [@kkrumlian](https://discuss.elastic.co/u/kkrumlian)\ **Replies:** 2\ **Last updated:** [October 30, 2015, 1:32pm UTC](https://discuss.elastic.co/t/logstash-not-ignoring-syslog-event-date/33329 "2015-10-30T13:32:19Z")
I have the following setup ELK server on Ubuntu 14.04 - timezone: EDT syslogd server on Ubuntu 10.04 - timezone: UTC (logstash-forwarder installed) app server 1 on Ubuntu 14.04 - timezone: EST (sending lo… --- ## [How cluster logstash?](https://discuss.elastic.co/t/how-cluster-logstash/33069)
**Author:** [@calle\_eman](https://discuss.elastic.co/u/calle_eman)\ **Replies:** 8\ **Last updated:** [October 30, 2015, 12:21pm UTC](https://discuss.elastic.co/t/how-cluster-logstash/33069 "2015-10-30T12:21:55Z")
I want to setup the logstash configuration described in the docs, but I don't understand how to set it up. Here's the sentence I'm struggling with: you can set up a load balancer between your data source machines and t… --- ## [Importing old IIS logfiles. Timestamp is today instead of date/time in logfile](https://discuss.elastic.co/t/importing-old-iis-logfiles-timestamp-is-today-instead-of-date-time-in-logfile/33311)
**Author:** [@mc1903](https://discuss.elastic.co/u/mc1903)\ **Replies:** 2\ **Last updated:** [October 30, 2015, 11:07am UTC](https://discuss.elastic.co/t/importing-old-iis-logfiles-timestamp-is-today-instead-of-date-time-in-logfile/33311 "2015-10-30T11:07:30Z")
I must be a complete idiot - I have been trying to get this working for 2 days. I have load of old IIS log files that I wish to import into elasticsearch-1.7.3 using logstash\_1.5.4-1 and view with kibana-4.1.2. I am us… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1224) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1226)