# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1265 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1266 --- ## [How to unzip compressed logs and ship through forwarder?](https://discuss.elastic.co/t/how-to-unzip-compressed-logs-and-ship-through-forwarder/685)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 2\ **Last updated:** [May 15, 2015, 5:58am UTC](https://discuss.elastic.co/t/how-to-unzip-compressed-logs-and-ship-through-forwarder/685 "2015-05-15T05:58:32Z")
Hello friends, I am using forwarder to ship log to logstash 1.4.2 where those are filtered. I have one system which compressed log files in a zip file. there are more than one zip files created at static location once in… --- ## [Pipeline blocking from output plugins](https://discuss.elastic.co/t/pipeline-blocking-from-output-plugins/713)
**Author:** [@cschotke](https://discuss.elastic.co/u/cschotke)\ **Replies:** 0\ **Last updated:** [May 14, 2015, 7:41pm UTC](https://discuss.elastic.co/t/pipeline-blocking-from-output-plugins/713 "2015-05-14T19:41:14Z")
We keep running into an issue where all logstash processing gets halted if one of our output plugins runs into an issue. For example, if our PagerDuty or Graphite outputs are not able to function correctly due to a conne… --- ## [Logstash 1.5.0 GA released](https://discuss.elastic.co/t/logstash-1-5-0-ga-released/712)
**Author:** [@suyograo](https://discuss.elastic.co/u/suyograo)\ **Replies:** 0\ **Last updated:** [May 14, 2015, 7:36pm UTC](https://discuss.elastic.co/t/logstash-1-5-0-ga-released/712 "2015-05-14T19:36:53Z")
We are pleased to announce the GA release of Logstash 1.5.0 with plugin changes, performance improvements, Apache Kafka integration and more! A big thank you to all our users for the many contributions, feedback, GitHub … --- ## [Windows event collection using Logstah Forwarder](https://discuss.elastic.co/t/windows-event-collection-using-logstah-forwarder/683)
**Author:** [@venkaty](https://discuss.elastic.co/u/venkaty)\ **Replies:** 1\ **Last updated:** [May 14, 2015, 2:44pm UTC](https://discuss.elastic.co/t/windows-event-collection-using-logstah-forwarder/683 "2015-05-14T14:44:20Z")
I would like to collect multiple Windows host events using WMI. Can I deploy Logstash-forwarder in each of the Windows machine to collect events (either using Windows Eventlog/WMI input plugins) and forward those collect… --- ## [How to run logstash-forwarder on unix](https://discuss.elastic.co/t/how-to-run-logstash-forwarder-on-unix/686)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 1\ **Last updated:** [May 14, 2015, 2:25pm UTC](https://discuss.elastic.co/t/how-to-run-logstash-forwarder-on-unix/686 "2015-05-14T14:25:23Z")
Hello Firends, I am new to logstash/ELK. I have successfully configured logstash-forwarder, logstash, Elasticsearch and Kibana on windows 7 and indexed logs. When I copied all setup to unix, then all went OK except forwa… --- ## [JMX CompositeDataSupport and Array](https://discuss.elastic.co/t/jmx-compositedatasupport-and-array/629)
**Author:** [@michaelT](https://discuss.elastic.co/u/michaelT)\ **Replies:** 1\ **Last updated:** [May 14, 2015, 12:33pm UTC](https://discuss.elastic.co/t/jmx-compositedatasupport-and-array/629 "2015-05-14T12:33:14Z")
Hi, Using the logstash jmx plugin, I'm retrieving a CompositeDataSupport Object which is then pushed to Elasticsearch. The problem is that the value pushed is not an array converted as a String but more something which… --- ## [Logstash-forwarder setup... where is my data?](https://discuss.elastic.co/t/logstash-forwarder-setup-where-is-my-data/639)
**Author:** [@alexolivan](https://discuss.elastic.co/u/alexolivan)\ **Replies:** 3\ **Last updated:** [May 14, 2015, 12:18am UTC](https://discuss.elastic.co/t/logstash-forwarder-setup-where-is-my-data/639 "2015-05-14T00:18:29Z")
I again forum... I have setup a server to send log data to a central logstash server... after some pain with TLS IP SAN certificates it seems it connects and does something according to forwarder logs: root@nodo01:/e… --- ## [Failed to create mapping\["date"\], following steps in getting started doc](https://discuss.elastic.co/t/failed-to-create-mapping-date-following-steps-in-getting-started-doc/655)
**Author:** [@Stan\_Linderstein](https://discuss.elastic.co/u/Stan_Linderstein)\ **Replies:** 0\ **Last updated:** [May 14, 2015, 12:15am UTC](https://discuss.elastic.co/t/failed-to-create-mapping-date-following-steps-in-getting-started-doc/655 "2015-05-14T00:15:17Z")
\[ I posted this question (and others) on Google Groups with no response so I'm trying this newbie question here. \] I've played a bit with Elasticsearch and now I'm wanting to get going with logstash. I found this docum… --- ## [Logstash email out all fields](https://discuss.elastic.co/t/logstash-email-out-all-fields/621)
**Author:** [@dalore](https://discuss.elastic.co/u/dalore)\ **Replies:** 1\ **Last updated:** [May 13, 2015, 6:24pm UTC](https://discuss.elastic.co/t/logstash-email-out-all-fields/621 "2015-05-13T18:24:54Z")
With the email output plugin I can't work out how to get it to dump the whole event. Seems like I have to specify the fields that I want. Trouble is that it's schemaless, unless I go and specify every field that might… --- ## [Sending mltiple fields per file... possible?](https://discuss.elastic.co/t/sending-mltiple-fields-per-file-possible/586)
**Author:** [@alexolivan](https://discuss.elastic.co/u/alexolivan)\ **Replies:** 5\ **Last updated:** [May 13, 2015, 10:12am UTC](https://discuss.elastic.co/t/sending-mltiple-fields-per-file-possible/586 "2015-05-13T10:12:25Z")
Hi forum.... seems this is the first post here!!! OK... By reading blogs and howtos, I'm confident we can make a cluster of servers to send their logs, and setup sending a "type" field, with a value, in order to distin… --- ## [How to scale daily indexing](https://discuss.elastic.co/t/how-to-scale-daily-indexing/236)
**Author:** [@gmu3](https://discuss.elastic.co/u/gmu3)\ **Replies:** 2\ **Last updated:** [May 12, 2015, 9:03pm UTC](https://discuss.elastic.co/t/how-to-scale-daily-indexing/236 "2015-05-12T21:03:45Z")
I know the default in Logstash is to create a new index daily. I've also seen in different places people recommend not running more than a couple shards per node, because each shard is a Lucene instance under the surfac… --- ## [Rsyslog and ELK](https://discuss.elastic.co/t/rsyslog-and-elk/548)
**Author:** [@Sdclmb](https://discuss.elastic.co/u/Sdclmb)\ **Replies:** 5\ **Last updated:** [May 12, 2015, 7:39pm UTC](https://discuss.elastic.co/t/rsyslog-and-elk/548 "2015-05-12T19:39:22Z")
hello, I was aked to install a Log server with rsyslog or syslog-ng(don't know which one is better) and to analysis this data with ELK. My question is : is it possible to tell stack ELK to get the log on the first serv… --- ## [Send Windows Log to Logstash Server](https://discuss.elastic.co/t/send-windows-log-to-logstash-server/560)
**Author:** [@SJ04](https://discuss.elastic.co/u/SJ04)\ **Replies:** 3\ **Last updated:** [May 12, 2015, 6:01pm UTC](https://discuss.elastic.co/t/send-windows-log-to-logstash-server/560 "2015-05-12T18:01:29Z")
Hi there! I know, this topic have been discussed many time before on this Platform, But i still have some issue with Receving Log from the Windows Platform to Logstash Server. Logstash Server: Ubuntu. i have tried … --- ## [Nagios event/log to Logstash](https://discuss.elastic.co/t/nagios-event-log-to-logstash/563)
**Author:** [@Penny\_Lee](https://discuss.elastic.co/u/Penny_Lee)\ **Replies:** 2\ **Last updated:** [May 12, 2015, 5:56pm UTC](https://discuss.elastic.co/t/nagios-event-log-to-logstash/563 "2015-05-12T17:56:33Z")
Hi, Is it possible to send the Nagios event/log (not local log) to logstash for index and store to elasticsearch? I found some plugin but I am not sure are they work. I found the followings: http://logstash.net/docs/1… --- ## [Question about fingerprint and de-duplicating](https://discuss.elastic.co/t/question-about-fingerprint-and-de-duplicating/403)
**Author:** [@logstash\_user](https://discuss.elastic.co/u/logstash_user)\ **Replies:** 2\ **Last updated:** [May 12, 2015, 12:13pm UTC](https://discuss.elastic.co/t/question-about-fingerprint-and-de-duplicating/403 "2015-05-12T12:13:22Z")
I have in my setup properly setup de-duplication based on fingerprinting messages. I had a question to the community as in is there a way to count the number of duplicates and store it as a field which increments every … --- ## [Duplicate field entries - multiple times even!](https://discuss.elastic.co/t/duplicate-field-entries-multiple-times-even/543)
**Author:** [@JeremyinNC](https://discuss.elastic.co/u/JeremyinNC)\ **Replies:** 0\ **Last updated:** [May 12, 2015, 10:38am UTC](https://discuss.elastic.co/t/duplicate-field-entries-multiple-times-even/543 "2015-05-12T10:38:39Z")
So far I have ~300 hosts forwarding their log4j and syslogs to Logstash, along with a handful of other log types. The log4j filters are kind of ugly and I think may be related to my problem. It started happening when I a… --- ## [Following files in rotating directories](https://discuss.elastic.co/t/following-files-in-rotating-directories/507)
**Author:** [@phunqe](https://discuss.elastic.co/u/phunqe)\ **Replies:** 2\ **Last updated:** [May 12, 2015, 7:23am UTC](https://discuss.elastic.co/t/following-files-in-rotating-directories/507 "2015-05-12T07:23:47Z")
Hello, I need to follow rotating logs, but the rotation is done on directory level. For example I need to follow file c:\\logs\\2015-05-11\\mylog.txt which next day will be c:\\logs\\2015-05-12\\mylog.txt and so on. Is i… --- ## [Newbie and new log format: shoutcast streaming server](https://discuss.elastic.co/t/newbie-and-new-log-format-shoutcast-streaming-server/500)
**Author:** [@alexolivan](https://discuss.elastic.co/u/alexolivan)\ **Replies:** 2\ **Last updated:** [May 11, 2015, 3:29pm UTC](https://discuss.elastic.co/t/newbie-and-new-log-format-shoutcast-streaming-server/500 "2015-05-11T15:29:20Z")
Hi forum... I'm quite new to all this logstash / elasticsearch (kibana) world so please take it easy... I have successfully created a centralized ELK setup to monitor our pfsense firewalls... and thereby, I have been… --- ## [Grokparse failure - custom tomcat access.log - newbee failure](https://discuss.elastic.co/t/grokparse-failure-custom-tomcat-access-log-newbee-failure/387)
**Author:** [@Martin\_Smevik](https://discuss.elastic.co/u/Martin_Smevik)\ **Replies:** 3\ **Last updated:** [May 11, 2015, 11:40am UTC](https://discuss.elastic.co/t/grokparse-failure-custom-tomcat-access-log-newbee-failure/387 "2015-05-11T11:40:31Z")
Hi, I have the following log line from tomcat access log: \[21/Apr/2015:00:00:02 +0200\] - 66.249.78.123 www.mydomain.no A7CEE20E7E0A5F1C5727E0036B868422.agap67 - GET /myuri/foobar/search-result.action?richList=false&pro… --- ## [Logstash 1.5.0 RC3 - Patterns](https://discuss.elastic.co/t/logstash-1-5-0-rc3-patterns/487)
**Author:** [@Szymon\_Raclawicki](https://discuss.elastic.co/u/Szymon_Raclawicki)\ **Replies:** 3\ **Last updated:** [May 11, 2015, 10:39am UTC](https://discuss.elastic.co/t/logstash-1-5-0-rc3-patterns/487 "2015-05-11T10:39:50Z")
Hi Guys, I think I got a bit lost since the upgrade to 1.5.0. Basically, right now I am not sure where are the patterns? Do I need to install logstash-patterns-core? Is there any documentation showing how I can install… --- ## [ELK stats on nginx](https://discuss.elastic.co/t/elk-stats-on-nginx/466)
**Author:** [@fedeisas](https://discuss.elastic.co/u/fedeisas)\ **Replies:** 1\ **Last updated:** [May 11, 2015, 1:04am UTC](https://discuss.elastic.co/t/elk-stats-on-nginx/466 "2015-05-11T01:04:10Z")
Hello everyone! I'm using an ELK stack to analyze my nginx acces logs. My grok pattern is: %{IPORHOST:clientip} %{NGUSER:ident} %{NGUSER:auth} \\\[%{HTTPDATE:timestamp}\\\] "%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NU… --- ## [Logstash Stops Processing](https://discuss.elastic.co/t/logstash-stops-processing/401)
**Author:** [@kotter](https://discuss.elastic.co/u/kotter)\ **Replies:** 2\ **Last updated:** [May 10, 2015, 12:15am UTC](https://discuss.elastic.co/t/logstash-stops-processing/401 "2015-05-10T00:15:06Z")
Hello all: I'm working with a logstash docker image (https://registry.hub.docker.com/u/helder/logstash/) linked to elasticsearch and kibana, all via docker. My problem is that logstash stops processing randomly. I o… --- ## [Logstash 1.5.0 RC4 released](https://discuss.elastic.co/t/logstash-1-5-0-rc4-released/409)
**Author:** [@suyograo](https://discuss.elastic.co/u/suyograo)\ **Replies:** 3\ **Last updated:** [May 9, 2015, 10:45pm UTC](https://discuss.elastic.co/t/logstash-1-5-0-rc4-released/409 "2015-05-09T22:45:23Z")
Hello all, a short note to announce we have released 1.5.0 RC4! Thanks for all the bug reports and patches for RC3! We are close to wrapping up 1.5.0 \[smile\] Here's the blog: https://www.elastic.co/blog/logstash-1-5-0-… --- ## [Unable to load the base.rb file in my plugin](https://discuss.elastic.co/t/unable-to-load-the-base-rb-file-in-my-plugin/219)
**Author:** [@kebbkh](https://discuss.elastic.co/u/kebbkh)\ **Replies:** 7\ **Last updated:** [May 8, 2015, 7:01pm UTC](https://discuss.elastic.co/t/unable-to-load-the-base-rb-file-in-my-plugin/219 "2015-05-08T19:01:05Z")
Hello, Whenever I attempt to run logstash using the --configtest flag this is the message I keep getting: The error reported is: Couldn't find any codec plugin named 'transit'. Are you sure this is correct? Trying t… --- ## [Using \_index to create new indexes not working?](https://discuss.elastic.co/t/using--index-to-create-new-indexes-not-working/396)
**Author:** [@efontana](https://discuss.elastic.co/u/efontana)\ **Replies:** 0\ **Last updated:** [May 8, 2015, 12:41pm UTC](https://discuss.elastic.co/t/using--index-to-create-new-indexes-not-working/396 "2015-05-08T12:41:43Z")
We are using Redis as our buffer, the Json going into redis contains a field like this: {"\_index", "mylogs-2015.05.07"} The elasticsearch outputter looks like this: output { elasticsearch { cluster =\> 'test… --- ## [Warning: "Redis connection problem" even after removing Redis from config](https://discuss.elastic.co/t/warning-redis-connection-problem-even-after-removing-redis-from-config/338)
**Author:** [@Vikram\_Nair](https://discuss.elastic.co/u/Vikram_Nair)\ **Replies:** 2\ **Last updated:** [May 8, 2015, 4:53am UTC](https://discuss.elastic.co/t/warning-redis-connection-problem-even-after-removing-redis-from-config/338 "2015-05-08T04:53:58Z")
Hi all, /etc/logstash/conf.d/logstash.conf is a below input { udp { port =\> 5000 type =\> syslog } } filter { kv { } } output { elasticsearch { cluster =\> LogStore } } I still keep getti… --- ## [Writing a codec plugin for logstash 1.4.2](https://discuss.elastic.co/t/writing-a-codec-plugin-for-logstash-1-4-2/350)
**Author:** [@kebbkh](https://discuss.elastic.co/u/kebbkh)\ **Replies:** 0\ **Last updated:** [May 7, 2015, 5:35pm UTC](https://discuss.elastic.co/t/writing-a-codec-plugin-for-logstash-1-4-2/350 "2015-05-07T17:35:26Z")
Which documentation should we be following for writing a codec plugin for logstash 1.4.2? Writing your own plugin or How to write a logstash codec plugin Still having some issues with configuring the codec plugin… --- ## [Grokparse failure - but grokdebugger works](https://discuss.elastic.co/t/grokparse-failure-but-grokdebugger-works/280)
**Author:** [@jayem](https://discuss.elastic.co/u/jayem)\ **Replies:** 3\ **Last updated:** [May 7, 2015, 10:43am UTC](https://discuss.elastic.co/t/grokparse-failure-but-grokdebugger-works/280 "2015-05-07T10:43:48Z")
hi all I have the following log line: {"rule":{"level":12,"comment":"apache error tagged by modsecurity","sidid":130401},"location":"(cli-git) 52.16.98.219-\>/var/log/apache2/error.log","full\_log":"\[Tue May 05 09:09:08… --- ## [Pattern Assistance ##](https://discuss.elastic.co/t/pattern-assistance/276)
**Author:** [@Hans](https://discuss.elastic.co/u/Hans)\ **Replies:** 3\ **Last updated:** [May 7, 2015, 6:54am UTC](https://discuss.elastic.co/t/pattern-assistance/276 "2015-05-07T06:54:50Z")
I would like to ask assistance with the following pattern to arrange them into different fields. The main challenge is the two # before each different field and then in the fields there are spaces as well. I tried to use … --- ## [Where is the config parsing logic?](https://discuss.elastic.co/t/where-is-the-config-parsing-logic/316)
**Author:** [@chun\_chen](https://discuss.elastic.co/u/chun_chen)\ **Replies:** 0\ **Last updated:** [May 6, 2015, 9:30pm UTC](https://discuss.elastic.co/t/where-is-the-config-parsing-logic/316 "2015-05-06T21:30:28Z")
I found the conditional does not work for S3 Output, i' like to debug [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1264) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1266)