# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1172 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1173 --- ## [Puppet module considered a dead project?](https://discuss.elastic.co/t/puppet-module-considered-a-dead-project/42866)
**Author:** [@jmkgreen](https://discuss.elastic.co/u/jmkgreen)\ **Replies:** 2\ **Last updated:** [April 1, 2016, 11:33am UTC](https://discuss.elastic.co/t/puppet-module-considered-a-dead-project/42866 "2016-04-01T11:33:26Z")
I ask as https://forge.puppetlabs.com/elasticsearch/logstash is approaching two years old and causes puppet master to crash on apt::source incompatibilities. Are we supposed to be removing this in favour of another mod… --- ## [TCP and json versus json\_lines when sending multiple messages](https://discuss.elastic.co/t/tcp-and-json-versus-json-lines-when-sending-multiple-messages/46048)
**Author:** [@Benson\_Margulies](https://discuss.elastic.co/u/Benson_Margulies)\ **Replies:** 0\ **Last updated:** [April 1, 2016, 11:11am UTC](https://discuss.elastic.co/t/tcp-and-json-versus-json-lines-when-sending-multiple-messages/46048 "2016-04-01T11:11:45Z")
I have a java program which needs to deliver some specialized traffic to logstash, so using log4j or logback is not appropriate. So, I set up logstash with a tcp socket and the json codec, and coded the program to con… --- ## [Reading Static logs](https://discuss.elastic.co/t/reading-static-logs/45693)
**Author:** [@debarun](https://discuss.elastic.co/u/debarun)\ **Replies:** 2\ **Last updated:** [April 1, 2016, 10:11am UTC](https://discuss.elastic.co/t/reading-static-logs/45693 "2016-04-01T10:11:49Z")
Hi, Is there any guide on reading static logs. I have the logs that will be shipped to another server and read by logstash(for now i cannot use forwarder). Regards, Debarun --- ## [LogStash: IdentityMapCodec has reached 100% capacity](https://discuss.elastic.co/t/logstash-identitymapcodec-has-reached-100-capacity/44473)
**Author:** [@lohith](https://discuss.elastic.co/u/lohith)\ **Replies:** 1\ **Last updated:** [April 1, 2016, 10:09am UTC](https://discuss.elastic.co/t/logstash-identitymapcodec-has-reached-100-capacity/44473 "2016-04-01T10:09:51Z")
In the Logstash logs, I see lots of errors such as like IdentityMapCodec has reached 100% capacity" and stops prcessing mesages . message=\>"IdentityMapCodec has reached 100% capacity", :current\_size=\>20000, :upper\_lim… --- ## [File input in Windows](https://discuss.elastic.co/t/file-input-in-windows/31969)
**Author:** [@Amir\_Gabay](https://discuss.elastic.co/u/Amir_Gabay)\ **Replies:** 3\ **Last updated:** [April 1, 2016, 4:32am UTC](https://discuss.elastic.co/t/file-input-in-windows/31969 "2016-04-01T04:32:15Z")
Hi, I'm using logstash file input in windows which locks the folder the file resides in. I've configured 'stat\_interval =\> 60' so it wouldn't sample the file every second but the folder is still being locked. Is there an… --- ## [Logstash locks file on Windows Server](https://discuss.elastic.co/t/logstash-locks-file-on-windows-server/31443)
**Author:** [@Amir\_Gabay](https://discuss.elastic.co/u/Amir_Gabay)\ **Replies:** 2\ **Last updated:** [April 1, 2016, 4:18am UTC](https://discuss.elastic.co/t/logstash-locks-file-on-windows-server/31443 "2016-04-01T04:18:16Z")
Hi, I've logstash 1.5.4 running on Win server 2008 R2. Its locking my application log, allowing it to rotate but it can't be deleted. My conf looks like this: input { file{ path =\> \["C:\\Program Files\\Apache Softwar… --- ## [Debian repo not working?](https://discuss.elastic.co/t/debian-repo-not-working/45222)
**Author:** [@stbc](https://discuss.elastic.co/u/stbc)\ **Replies:** 3\ **Last updated:** [March 31, 2016, 9:10pm UTC](https://discuss.elastic.co/t/debian-repo-not-working/45222 "2016-03-31T21:10:16Z")
Hi everybody. Just tried to set up the repos for apt, but running into problems. My sources.list deb http://packages.elastic.co/logstash/2.2/debian stable main When issuing an apt-update, apt will start and the it t… --- ## [Can I merge duplicate record and show it as a one record or document](https://discuss.elastic.co/t/can-i-merge-duplicate-record-and-show-it-as-a-one-record-or-document/45995)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 1\ **Last updated:** [March 31, 2016, 9:09pm UTC](https://discuss.elastic.co/t/can-i-merge-duplicate-record-and-show-it-as-a-one-record-or-document/45995 "2016-03-31T21:09:50Z")
Hi Experts, I can see following in elasticsearch index ID Name Marks 1 vik 10 2 vik 10 3 vik 10 4 gop 20 5 fall 20 I want to have something like … --- ## [Logstash - Scheduler (rufus scheduler)](https://discuss.elastic.co/t/logstash-scheduler-rufus-scheduler/45963)
**Author:** [@vikram\_yerneni](https://discuss.elastic.co/u/vikram_yerneni)\ **Replies:** 0\ **Last updated:** [March 31, 2016, 3:24pm UTC](https://discuss.elastic.co/t/logstash-scheduler-rufus-scheduler/45963 "2016-03-31T15:24:08Z")
Hello, I am trying to schedule logstash run for every 5 mins every day (full week) expect during certain time intervals in a day (between 10:00 AM - 10:30 AM). I know I can use the cron syntax for the job "schedule =\> "\*… --- ## [How to close the file stream once the message has been pushed.?](https://discuss.elastic.co/t/how-to-close-the-file-stream-once-the-message-has-been-pushed/45983)
**Author:** [@sdaruna](https://discuss.elastic.co/u/sdaruna)\ **Replies:** 3\ **Last updated:** [March 31, 2016, 8:20pm UTC](https://discuss.elastic.co/t/how-to-close-the-file-stream-once-the-message-has-been-pushed/45983 "2016-03-31T20:20:57Z")
Hi, My requirement is to get the remote file using beats and write it to hdfs using logstash. I need to write the output to hdfs using logstash. However, I cannot use webhdfs, so i used pipe output with hdfs append… --- ## [Mismatch in count of actual log and log inserted on ES](https://discuss.elastic.co/t/mismatch-in-count-of-actual-log-and-log-inserted-on-es/45968)
**Author:** [@ajays](https://discuss.elastic.co/u/ajays)\ **Replies:** 2\ **Last updated:** [March 31, 2016, 4:19pm UTC](https://discuss.elastic.co/t/mismatch-in-count-of-actual-log-and-log-inserted-on-es/45968 "2016-03-31T16:19:02Z")
Hello, I have inserted few logs in ES using logstash. But when I check the count of actual log and data inserted in ES, it is not matching. Seems like logstash is dropping few messages. If I check the count in kiban… --- ## [Extract on various lines with logstash](https://discuss.elastic.co/t/extract-on-various-lines-with-logstash/45916)
**Author:** [@oth1113](https://discuss.elastic.co/u/oth1113)\ **Replies:** 7\ **Last updated:** [March 31, 2016, 2:22pm UTC](https://discuss.elastic.co/t/extract-on-various-lines-with-logstash/45916 "2016-03-31T14:22:08Z")
so I just discovered logstash and I managed to extract data from log files. This time, I have to extract information on MULTIlines, I show you an example: 2016-03-07 14:09:11,613 INFO \[\]\[com.ole.ecom.jms.crm.JmsCrmSen… --- ## [Grok for this log](https://discuss.elastic.co/t/grok-for-this-log/45862)
**Author:** [@Farouk\_Samu](https://discuss.elastic.co/u/Farouk_Samu)\ **Replies:** 4\ **Last updated:** [March 31, 2016, 2:09pm UTC](https://discuss.elastic.co/t/grok-for-this-log/45862 "2016-03-31T14:09:23Z")
I want to create a grok for the log below. "2016-03-30 19:14:06-0500 \[HonsshServerTransport,13657,195.154.43.29\] \[PLUG IN\]\[EXAMPLE\] - {'honey\_port': '22', 'sensor\_name': 'node3', 'session': {'auths': \[… --- ## [Gather Syslogs from a router](https://discuss.elastic.co/t/gather-syslogs-from-a-router/45338)
**Author:** [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)\ **Replies:** 12\ **Last updated:** [March 31, 2016, 12:27pm UTC](https://discuss.elastic.co/t/gather-syslogs-from-a-router/45338 "2016-03-31T12:27:55Z")
Hi, How could I set up my router to send its Syslogs to Logstash? I already have my ELK server set up completely and got a Client Server that is successfully sending its Syslogs to Logstash where I can visualize it i… --- ## [IIS logging and \_grokparsefailure tag](https://discuss.elastic.co/t/iis-logging-and--grokparsefailure-tag/45648)
**Author:** [@thyfere](https://discuss.elastic.co/u/thyfere)\ **Replies:** 17\ **Last updated:** [March 31, 2016, 11:59am UTC](https://discuss.elastic.co/t/iis-logging-and--grokparsefailure-tag/45648 "2016-03-31T11:59:08Z")
Hi, I want to log our Microsoft's IIS logs but getting \_grokparsefailure in tag, though I can see the messages in Kibana but not sure why this tag is being appeared. Here is my NXLOG conf: define ROOT C:\\Program File… --- ## [Django log format| grok things | need help](https://discuss.elastic.co/t/django-log-format-grok-things-need-help/45915)
**Author:** [@qinwenchao](https://discuss.elastic.co/u/qinwenchao)\ **Replies:** 4\ **Last updated:** [March 31, 2016, 10:51am UTC](https://discuss.elastic.co/t/django-log-format-grok-things-need-help/45915 "2016-03-31T10:51:13Z")
log\_format Get: \[u'push\_id=', u'vendor=ziyou', u'platform=iPhone', u'app=7', u'height=568', u'width=320', u'version=2.0.3', u'app\_ver=2.0.3', u'phoneType=iPhone', u'systemVer=9.2.1', u'device\_id=0baaa6972b5d41a5900dc76c9… --- ## [Logstash in Windows platform](https://discuss.elastic.co/t/logstash-in-windows-platform/45918)
**Author:** [@Prerona](https://discuss.elastic.co/u/Prerona)\ **Replies:** 1\ **Last updated:** [March 31, 2016, 10:36am UTC](https://discuss.elastic.co/t/logstash-in-windows-platform/45918 "2016-03-31T10:36:03Z")
Hi, I want to feed an application log file to logstash so that whenever an error is recorded in the log it will be shown in elasticsearch dashboard. Is it possible? If yes , need full configuration steps for it. --- ## [Detect Logstash Idling State](https://discuss.elastic.co/t/detect-logstash-idling-state/45895)
**Author:** [@gringo](https://discuss.elastic.co/u/gringo)\ **Replies:** 1\ **Last updated:** [March 31, 2016, 9:09am UTC](https://discuss.elastic.co/t/detect-logstash-idling-state/45895 "2016-03-31T09:09:17Z")
Is there a way to check if logstash is in an idling state. I need to know if logstash has finished parsing the log files assigned to it. --- ## [Nested Mapping in Logstash](https://discuss.elastic.co/t/nested-mapping-in-logstash/45886)
**Author:** [@Hans\_Kruse](https://discuss.elastic.co/u/Hans_Kruse)\ **Replies:** 1\ **Last updated:** [March 31, 2016, 8:37am UTC](https://discuss.elastic.co/t/nested-mapping-in-logstash/45886 "2016-03-31T08:37:32Z")
Logstash Template Mappings do not have properties like ElasticSearch mappings do. How do I define nested template Mappings for Logstash? I have a Mapping for elasticsearch like: "blablabla":{ … --- ## [Kafka consumer issue](https://discuss.elastic.co/t/kafka-consumer-issue/44840)
**Author:** [@ravitandur](https://discuss.elastic.co/u/ravitandur)\ **Replies:** 2\ **Last updated:** [March 31, 2016, 7:56am UTC](https://discuss.elastic.co/t/kafka-consumer-issue/44840 "2016-03-31T07:56:29Z")
We are using logstash kafka input plugin to process the data from kafka. We are running 3 log stash instances with following config: input { heartbeat { interval =\> 60 type =\> "heartbeat" add\_… --- ## [Clamp::UsageError: Unrecognised option '-c'](https://discuss.elastic.co/t/clamp-usageerror-unrecognised-option-c/45871)
**Author:** [@seeni](https://discuss.elastic.co/u/seeni)\ **Replies:** 10\ **Last updated:** [March 31, 2016, 7:04am UTC](https://discuss.elastic.co/t/clamp-usageerror-unrecognised-option-c/45871 "2016-03-31T07:04:48Z")
Clamp::UsageError: Unrecognised option '-c' signal\_usage\_error at C:/.../Vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:103 find\_option at C:/.../Vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp… --- ## [Logstash-2.2.2, windows, IIS log file format](https://discuss.elastic.co/t/logstash-2-2-2-windows-iis-log-file-format/45501)
**Author:** [@jack8653](https://discuss.elastic.co/u/jack8653)\ **Replies:** 6\ **Last updated:** [March 31, 2016, 6:48am UTC](https://discuss.elastic.co/t/logstash-2-2-2-windows-iis-log-file-format/45501 "2016-03-31T06:48:17Z")
Hello, when I'm parsing iis log file in UTF-8 format I'm getting \[0\] "\_grokparsefailure" error and When I'm parsing log file using ANSI format there is nothing working Logstash just display message on console " Logstas… --- ## [Automatic Parse Compied to Folder](https://discuss.elastic.co/t/automatic-parse-compied-to-folder/45864)
**Author:** [@gringo](https://discuss.elastic.co/u/gringo)\ **Replies:** 1\ **Last updated:** [March 31, 2016, 5:41am UTC](https://discuss.elastic.co/t/automatic-parse-compied-to-folder/45864 "2016-03-31T05:41:08Z")
How do I make logstash such that it will automatically parse files that are copied into a folder. I have a script that moves files into the a folder once every 5 hours. Right now I have to open the log file do a se… --- ## [Can't run configtest Logstash](https://discuss.elastic.co/t/cant-run-configtest-logstash/45870)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 1\ **Last updated:** [March 31, 2016, 5:36am UTC](https://discuss.elastic.co/t/cant-run-configtest-logstash/45870 "2016-03-31T05:36:32Z")
I try to update logstash to 2.3. When i run command /opt/logstash/bin/logstash -f /etc/logstash/conf.d/ --configtest and got error An unexpected error occurred! {:error=\>#\ **Author:** [@JoaquinS](https://discuss.elastic.co/u/JoaquinS)\ **Replies:** 5\ **Last updated:** [March 31, 2016, 1:43am UTC](https://discuss.elastic.co/t/logstash-shutdown-completed/45809 "2016-03-31T01:43:39Z") Hello, What it means that if I'm consuming from an index of ES (that has always incoming data) Logstash says : "Logstash shutdown completed"?. My input config is this: input { elasticsearch { hosts =\> "192.16… --- ## [Logstash 2.3.0 causing problems with http es\_bulk inputs](https://discuss.elastic.co/t/logstash-2-3-0-causing-problems-with-http-es-bulk-inputs/45860)
**Author:** [@esds2000](https://discuss.elastic.co/u/esds2000)\ **Replies:** 0\ **Last updated:** [March 31, 2016, 12:36am UTC](https://discuss.elastic.co/t/logstash-2-3-0-causing-problems-with-http-es-bulk-inputs/45860 "2016-03-31T00:36:39Z")
After upgrading to Logstash 2.3.0 our http input using the es\_bulk codec stopped working. The only data passing through to redis or elastic is the meta blob. Downgrading to 2.2.3 fixed the issue. Is this a bug with the n… --- ## [Logstash 2.3 custom array fields](https://discuss.elastic.co/t/logstash-2-3-custom-array-fields/45856)
**Author:** [@wal](https://discuss.elastic.co/u/wal)\ **Replies:** 0\ **Last updated:** [March 30, 2016, 11:07pm UTC](https://discuss.elastic.co/t/logstash-2-3-custom-array-fields/45856 "2016-03-30T23:07:19Z")
Hi, When upgrading to 2.3 I have lost the ability to create a custom array field example filter { ruby { code =\>" event\['custom'\] ||= \[\] event\['custom\] \<\< 'test' unless event\['custo… --- ## [Log-stash forwarder in a RHEL5 system](https://discuss.elastic.co/t/log-stash-forwarder-in-a-rhel5-system/24797)
**Author:** [@Roshan\_r](https://discuss.elastic.co/u/Roshan_r)\ **Replies:** 3\ **Last updated:** [March 30, 2016, 9:34pm UTC](https://discuss.elastic.co/t/log-stash-forwarder-in-a-rhel5-system/24797 "2016-03-30T21:34:05Z")
I downloaded and installed Elastic Search, Logstash and Kibana on a RHEL6 system. My components are running on a RHEL5 system. In order to save the CPU utilisation, i am planning to install the log-stash forwarder on RHE… --- ## [Logstash 2.3 Geoip Filter Returning Invalid Coordinates](https://discuss.elastic.co/t/logstash-2-3-geoip-filter-returning-invalid-coordinates/45845)
**Author:** [@arizonawayfarer](https://discuss.elastic.co/u/arizonawayfarer)\ **Replies:** 0\ **Last updated:** [March 30, 2016, 7:35pm UTC](https://discuss.elastic.co/t/logstash-2-3-geoip-filter-returning-invalid-coordinates/45845 "2016-03-30T19:35:26Z")
So I was giving the 2.3 release a spin and am running into an issue with the geoip filter that was working fine in 2.2.1. The latitude for every single processed event is the same invalid value. If i stop and restart … --- ## [Log stash configuration file to push delta of data from SQL Server using JDBC Importer](https://discuss.elastic.co/t/log-stash-configuration-file-to-push-delta-of-data-from-sql-server-using-jdbc-importer/45841)
**Author:** [@sreeramg](https://discuss.elastic.co/u/sreeramg)\ **Replies:** 1\ **Last updated:** [March 30, 2016, 7:54pm UTC](https://discuss.elastic.co/t/log-stash-configuration-file-to-push-delta-of-data-from-sql-server-using-jdbc-importer/45841 "2016-03-30T19:54:31Z")
Hi, Any one has done this delta import in Logstatsh from sql server. my configuration file looks like this, but it fetches all the rows every time it is running. I need only the DELTA of data ( what changed/added/delet… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1171) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1173)