# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1155 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1156 --- ## [IIS Logs](https://discuss.elastic.co/t/iis-logs/49711)
**Author:** [@Michael1](https://discuss.elastic.co/u/Michael1)\ **Replies:** 2\ **Last updated:** [May 11, 2016, 11:55pm UTC](https://discuss.elastic.co/t/iis-logs/49711 "2016-05-11T23:55:16Z")
Is it possible to have too large of IIS logs for Elasticsearch? For example, there are at least ten servers sending IIS logs that are about 3GB each per file. I have noticed today that IIS logs have not been in Kiban… --- ## [A plugin had an unrecoverable error. Will restart this plugin](https://discuss.elastic.co/t/a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/49695)
**Author:** [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 11:38pm UTC](https://discuss.elastic.co/t/a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/49695 "2016-05-11T23:38:24Z")
Logstash crashed with the following error. A plugin had an unrecoverable error. Will restart this plugin.\\n Plugin: \[\\"/apps/\_\_302023\_\_\_\\", \\"/apps/completed//302023\_\_\_\*\\"\], start\_position=\>\\"beginning\\", type=\>\\"tRANS\\… --- ## [Any Idea Parsing it?](https://discuss.elastic.co/t/any-idea-parsing-it/49685)
**Author:** [@Rubytor](https://discuss.elastic.co/u/Rubytor)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 11:31pm UTC](https://discuss.elastic.co/t/any-idea-parsing-it/49685 "2016-05-11T23:31:21Z")
Hi, I've been thinking and looking for the best way to parse this kind of lines: 2016-05-04 05:17:03,486 INFO \[customPoolExecutor-249587\] , messageFormat=STANDARD, messageBody=WEVXEVMA01US00010136 00000000 … --- ## [Logstash stops send information to elasticsearch](https://discuss.elastic.co/t/logstash-stops-send-information-to-elasticsearch/49677)
**Author:** [@daguero](https://discuss.elastic.co/u/daguero)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 11:30pm UTC](https://discuss.elastic.co/t/logstash-stops-send-information-to-elasticsearch/49677 "2016-05-11T23:30:41Z")
I am using elasticsearch 2.2.0, logstash 2.2.2, Kibana 4.4.1, topbeat 1.1.1 and filebeat 1.1.1 containers deployed Docker in a cluster of Mesos. When I deploy services everything seems to work fine, but after a few hou… --- ## [Jdbc Input error cause : no sqljdbc\_auth in java.library.path](https://discuss.elastic.co/t/jdbc-input-error-cause-no-sqljdbc-auth-in-java-library-path/49698)
**Author:** [@Alejandro\_Valdes\_Arc](https://discuss.elastic.co/u/Alejandro_Valdes_Arc)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 9:51pm UTC](https://discuss.elastic.co/t/jdbc-input-error-cause-no-sqljdbc-auth-in-java-library-path/49698 "2016-05-11T21:51:35Z")
Hello everyone Im trying to connect to a Sql Server database with integrated authentication, i tried with 3 differents drivers sqljdbc4.jar, sqljdbc.jar, sqljdbc41.jar and differents conections strings jdbc\_connecti… --- ## [Upgrading, which version?](https://discuss.elastic.co/t/upgrading-which-version/49819)
**Author:** [@powellchristoph](https://discuss.elastic.co/u/powellchristoph)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 9:33pm UTC](https://discuss.elastic.co/t/upgrading-which-version/49819 "2016-05-11T21:33:12Z")
Greetings! I am looking to upgrade from version 1.5.6 to a more recent version, but I have noticed there has been quite a bit of development and several releases. Which is considered to be the most stable version? Whic… --- ## [Email configuration in logstash](https://discuss.elastic.co/t/email-configuration-in-logstash/1760)
**Author:** [@gaurav2090](https://discuss.elastic.co/u/gaurav2090)\ **Replies:** 12\ **Last updated:** [May 11, 2016, 9:27pm UTC](https://discuss.elastic.co/t/email-configuration-in-logstash/1760 "2016-05-11T21:27:59Z")
Hi, Below is my configuration file filter { if \[type\] == "sftp\_authlog" { grok { pattern =\> "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host\_target} sshd\[%{BASE10NUM}\]: Failed password for (invalid user |)%{USERNAME:… --- ## [How to debug log entries which just contain "%{host} %{message}" solved!](https://discuss.elastic.co/t/how-to-debug-log-entries-which-just-contain-host-message-solved/49597)
**Author:** [@davidweir](https://discuss.elastic.co/u/davidweir)\ **Replies:** 5\ **Last updated:** [May 11, 2016, 9:27pm UTC](https://discuss.elastic.co/t/how-to-debug-log-entries-which-just-contain-host-message-solved/49597 "2016-05-11T21:27:04Z")
I configured logstash to updload to s3. When I check the files on s3, they are full of entries like this: 2016-05-09T23:27:31.811Z %{host} %{message} 2016-05-09T23:27:31.811Z %{host} %{message} 2016-05-09T23:27:31.809Z %… --- ## [Syslog Multiple log entries in a single log](https://discuss.elastic.co/t/syslog-multiple-log-entries-in-a-single-log/49823)
**Author:** [@davidhowell-tx](https://discuss.elastic.co/u/davidhowell-tx)\ **Replies:** 0\ **Last updated:** [May 11, 2016, 8:01pm UTC](https://discuss.elastic.co/t/syslog-multiple-log-entries-in-a-single-log/49823 "2016-05-11T20:01:50Z")
I am using the Logstash Syslog input plugin to receive events from a single source. I've noticed that sometimes the message actually includes multiple log entries. I've seen up to 5 events in the message field of a sin… --- ## [Logstash + Split filepath with backslash \\](https://discuss.elastic.co/t/logstash-split-filepath-with-backslash/49806)
**Author:** [@sarthak2016](https://discuss.elastic.co/u/sarthak2016)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 5:55pm UTC](https://discuss.elastic.co/t/logstash-split-filepath-with-backslash/49806 "2016-05-11T17:55:37Z")
Hello All, I am facing a issue to split the filepath with backslash, Below is my code for split filepath with \\ Logstash Code input { file { path =\> "C:\\logstash-2.3.1\\CSV\\SQL.csv" type=\>"sql" } } … --- ## [Back references in mutate gsub?](https://discuss.elastic.co/t/back-references-in-mutate-gsub/49707)
**Author:** [@jeffkirk1](https://discuss.elastic.co/u/jeffkirk1)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 4:59pm UTC](https://discuss.elastic.co/t/back-references-in-mutate-gsub/49707 "2016-05-11T16:59:04Z")
I'm in the middle of implementing a PostgreSQL log parser that will normalize the queries so I can do stats on them. As part of this, I need to be able to search for components of fields and replace them with back refere… --- ## [Handle multiple log levels within an event using grok](https://discuss.elastic.co/t/handle-multiple-log-levels-within-an-event-using-grok/49804)
**Author:** [@vikas\_J](https://discuss.elastic.co/u/vikas_J)\ **Replies:** 0\ **Last updated:** [May 11, 2016, 4:25pm UTC](https://discuss.elastic.co/t/handle-multiple-log-levels-within-an-event-using-grok/49804 "2016-05-11T16:25:48Z")
I have a apache tomcat log of the following format: Mar 15, 2014 8:18:33 AM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 576 ms ERROR: org.apache.thrift.server.TThreadPoolServer - Thrift … --- ## [Log4J input question about Map](https://discuss.elastic.co/t/log4j-input-question-about-map/49802)
**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Replies:** 0\ **Last updated:** [May 11, 2016, 4:13pm UTC](https://discuss.elastic.co/t/log4j-input-question-about-map/49802 "2016-05-11T16:13:45Z")
I see at https://www.elastic.co/guide/en/logstash/current/plugins-inputs-log4j.html, "if the original log4j LoggingEvent contains MDC hash entries, they will be merged in the event as fields.". I want to take advantage o… --- ## [Logstash unhandled exception](https://discuss.elastic.co/t/logstash-unhandled-exception/49798)
**Author:** [@Zfs](https://discuss.elastic.co/u/Zfs)\ **Replies:** 0\ **Last updated:** [May 11, 2016, 3:54pm UTC](https://discuss.elastic.co/t/logstash-unhandled-exception/49798 "2016-05-11T15:54:09Z")
I've started to see these exceptions thrown after updating to 2.3.2. Source is beats 1.2.2 Logstash input { beats { port =\> "5001" add\_field =\> { "beats" =\> "topic"} } } I have 3 instances of logstash run… --- ## [Grok Pattern not parsing](https://discuss.elastic.co/t/grok-pattern-not-parsing/49750)
**Author:** [@dlopez](https://discuss.elastic.co/u/dlopez)\ **Replies:** 8\ **Last updated:** [May 11, 2016, 2:43pm UTC](https://discuss.elastic.co/t/grok-pattern-not-parsing/49750 "2016-05-11T14:43:17Z")
Hi, I'm trying to parse a log file with content like this: 2016-05-09 12:00:00,006 INFO \[com.level2.quartz.BaseLevel2EngineJob\] (MVCScheduler\_Worker-1) Executing job It removes expired activation codes 2016-05-09 12:0… --- ## [Logstash '\_grokparsefailure' issue](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756)
**Author:** [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)\ **Replies:** 13\ **Last updated:** [May 11, 2016, 1:22pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756 "2016-05-11T13:22:56Z")
I am using custom grok pattern to parse my log file, Not matter what i do it always give me the \_grokparsefailure exception. However, it work perfectly on https://grokdebug.herokuapp.com/ My custom pattern file is loca… --- ## [SOLVED: How to implement Geoip with new index](https://discuss.elastic.co/t/solved-how-to-implement-geoip-with-new-index/49664)
**Author:** [@Clement\_Ros](https://discuss.elastic.co/u/Clement_Ros)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 12:54pm UTC](https://discuss.elastic.co/t/solved-how-to-implement-geoip-with-new-index/49664 "2016-05-11T12:54:15Z")
Hi, I use logstash 2.2 and elasticsearch 2.2 and Kibana 4.4 My problem is that the Geoip fields do not appear in geoip Kibana. I use a specific index named scarlette-\*. here is my logstash filter: geoip { … --- ## [Logs to Elastic cloud](https://discuss.elastic.co/t/logs-to-elastic-cloud/49634)
**Author:** [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Replies:** 3\ **Last updated:** [May 11, 2016, 9:47am UTC](https://discuss.elastic.co/t/logs-to-elastic-cloud/49634 "2016-05-11T09:47:22Z")
Hi, I've just started with the elastic cloud, i would like to send logs from devices ( routers) where i can only set the ipadress of the log receiver to the cloud. Which is the best solution ? Do i have to install a… --- ## [Grok Pattern for Handling and merging different log formats in the same file](https://discuss.elastic.co/t/grok-pattern-for-handling-and-merging-different-log-formats-in-the-same-file/49670)
**Author:** [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)\ **Replies:** 0\ **Last updated:** [May 10, 2016, 2:38pm UTC](https://discuss.elastic.co/t/grok-pattern-for-handling-and-merging-different-log-formats-in-the-same-file/49670 "2016-05-10T14:38:57Z")
Hi, I am trying to create a GROK filter that will handle different type of logtype and merge the log for java stacktrace exception. Like i am having a below two log type Log with Message type 20160509 115807 I SD … --- ## [Help with HTTP\_Poller](https://discuss.elastic.co/t/help-with-http-poller/35871)
**Author:** [@Tal\_Bar\_Or](https://discuss.elastic.co/u/Tal_Bar_Or)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 3:48am UTC](https://discuss.elastic.co/t/help-with-http-poller/35871 "2016-05-11T03:48:24Z")
Hi All, I am trying to pull out from rest API using http\_poller data that should eventually presented in Kibana. When i am using curl i have successful results show below , in other hand when using Http\_Poller with … --- ## [Is there a way to add cookie field to logstash http output plugin?‏](https://discuss.elastic.co/t/is-there-a-way-to-add-cookie-field-to-logstash-http-output-plugin/49600)
**Author:** [@prince6635](https://discuss.elastic.co/u/prince6635)\ **Replies:** 2\ **Last updated:** [May 10, 2016, 9:54pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-add-cookie-field-to-logstash-http-output-plugin/49600 "2016-05-10T21:54:24Z")
With the following configuration, I just want to forward logs from a file to an http endpoint with cookie being set, but I always got the error "Unknown setting 'cookie' for http {:level=\>:error}", looks like http output… --- ## [Logstash and cisco asa v9 netflow](https://discuss.elastic.co/t/logstash-and-cisco-asa-v9-netflow/42027)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 16\ **Last updated:** [May 10, 2016, 8:57pm UTC](https://discuss.elastic.co/t/logstash-and-cisco-asa-v9-netflow/42027 "2016-05-10T20:57:24Z")
has anyone got this working, i can see some work has been done on the netflow.yaml file in regards to this I've tried adding the changed to my netflow.yaml but i keep getting this error. no matching template for… --- ## [Logstash creating day based indexes](https://discuss.elastic.co/t/logstash-creating-day-based-indexes/49633)
**Author:** [@rajkumar3v](https://discuss.elastic.co/u/rajkumar3v)\ **Replies:** 4\ **Last updated:** [May 10, 2016, 2:58pm UTC](https://discuss.elastic.co/t/logstash-creating-day-based-indexes/49633 "2016-05-10T14:58:16Z")
Hello guys, I am new to Logstash, i have configured logstash to process syslog messages and it is creating indexes every day but i need single index for syslog messages. (I have understand logstash will cre… --- ## [Multiple Elasticsearch types output for a same log](https://discuss.elastic.co/t/multiple-elasticsearch-types-output-for-a-same-log/49547)
**Author:** [@LetMeR00t](https://discuss.elastic.co/u/LetMeR00t)\ **Replies:** 7\ **Last updated:** [May 10, 2016, 1:09pm UTC](https://discuss.elastic.co/t/multiple-elasticsearch-types-output-for-a-same-log/49547 "2016-05-10T13:09:12Z")
Hello everyone, I'm working on a project using ELK and I would like to know if it's possible for Logstash to store information of a same document but in different Elasticsearch types ? Because my explanation is not v… --- ## [Beats input blocked](https://discuss.elastic.co/t/beats-input-blocked/47664)
**Author:** [@ally](https://discuss.elastic.co/u/ally)\ **Replies:** 7\ **Last updated:** [May 10, 2016, 12:51pm UTC](https://discuss.elastic.co/t/beats-input-blocked/47664 "2016-05-10T12:51:41Z")
Hi, I'm experiencing problems with beats input in Logstash getting the following error: {:timestamp=\>"2016-04-15T10:18:32.126000+0200", :message=\>"CircuitBreaker::Open", :name=\>"Beats input", :level=\>:warn} {:timesta… --- ## [I copy this picture from "https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html", and I can't understand this architecture how to avoid message queue reveiving duplicate data](https://discuss.elastic.co/t/i-copy-this-picture-from-https-www-elastic-co-guide-en-logstash-current-deploying-and-scaling-html-and-i-cant-understand-this-architecture-how-to-avoid-message-queue-reveiving-duplicate-data/49362)
**Author:** [@parsons-smith](https://discuss.elastic.co/u/parsons-smith)\ **Replies:** 6\ **Last updated:** [May 10, 2016, 12:43pm UTC](https://discuss.elastic.co/t/i-copy-this-picture-from-https-www-elastic-co-guide-en-logstash-current-deploying-and-scaling-html-and-i-cant-understand-this-architecture-how-to-avoid-message-queue-reveiving-duplicate-data/49362 "2016-05-10T12:43:44Z")
--- ## [Send logstash data to cluster](https://discuss.elastic.co/t/send-logstash-data-to-cluster/49644)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 1\ **Last updated:** [May 10, 2016, 12:21pm UTC](https://discuss.elastic.co/t/send-logstash-data-to-cluster/49644 "2016-05-10T12:21:02Z")
what does the correct output section look like? output { elasticsearch { host =\> "172.17.20.207","172.17.20.208" } } will above still work as a cluster or am i kust sending the same data to two hosts independen… --- ## [Configuring Logstash to parse Apache log files and send](https://discuss.elastic.co/t/configuring-logstash-to-parse-apache-log-files-and-send/49591)
**Author:** [@Jemli\_Fathi](https://discuss.elastic.co/u/Jemli_Fathi)\ **Replies:** 4\ **Last updated:** [May 10, 2016, 11:06am UTC](https://discuss.elastic.co/t/configuring-logstash-to-parse-apache-log-files-and-send/49591 "2016-05-10T11:06:30Z")
I'm trying to parse xampp log files and to send them to ES, i found a configuration example: input{ file{ path =\> '/opt/lampp/logs/\*.log' start\_position =\> beginning } } filter { if \[path\] =~ "access" { m… --- ## [How to send NULL DATE value to elasticsearch using logstash](https://discuss.elastic.co/t/how-to-send-null-date-value-to-elasticsearch-using-logstash/49560)
**Author:** [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\ **Replies:** 7\ **Last updated:** [May 10, 2016, 10:04am UTC](https://discuss.elastic.co/t/how-to-send-null-date-value-to-elasticsearch-using-logstash/49560 "2016-05-10T10:04:49Z")
Hi, Its a blocker to me. I am not able to send NULL DATE value to the elasticsearch using logstash. null value storing as string in elasticsearch. if actual comes with proper date like 2012-01-01 11:11:11 it is also str… --- ## [Question on logstash output to redis](https://discuss.elastic.co/t/question-on-logstash-output-to-redis/49374)
**Author:** [@macymin](https://discuss.elastic.co/u/macymin)\ **Replies:** 7\ **Last updated:** [May 10, 2016, 9:58am UTC](https://discuss.elastic.co/t/question-on-logstash-output-to-redis/49374 "2016-05-10T09:58:25Z")
in logstash 2.3, it says in redis it support host as array like below: Does anyone know when i configure multiple server, the message are duplicated to those servers or load balanced to those servers??? host Value type… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1154) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1156)