# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1211 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1212 --- ## [While updating logstash 1.4.2 we encounter - Unable to find JRuby error](https://discuss.elastic.co/t/while-updating-logstash-1-4-2-we-encounter-unable-to-find-jruby-error/36152)
**Author:** [@meet2datta](https://discuss.elastic.co/u/meet2datta)\ **Replies:** 26\ **Last updated:** [December 10, 2015, 10:15am UTC](https://discuss.elastic.co/t/while-updating-logstash-1-4-2-we-encounter-unable-to-find-jruby-error/36152 "2015-12-10T10:15:25Z")
Hello Team, We were trying to update logstash for 1.4.2 to 2.1.0 latest version. We encounter below issues -- /opt/logstash/bin/logstash: line 22: dirname: command not found /opt/logstash/bin/logstash: line 22: //b… --- ## [Parse directory name regex](https://discuss.elastic.co/t/parse-directory-name-regex/36778)
**Author:** [@Sonne](https://discuss.elastic.co/u/Sonne)\ **Replies:** 4\ **Last updated:** [December 10, 2015, 8:39am UTC](https://discuss.elastic.co/t/parse-directory-name-regex/36778 "2015-12-10T08:39:00Z")
Hello. I have the logline 2015.10.05 18:32:25.913:Process cannot access the file 'file.txt' because it is being used by another process I have written the grok: (?m)%{YEAR:year}.%{MONTHNUM:month}.%{MONTHDAY:day}%{SP… --- ## [Logstash-forwarder documentaion](https://discuss.elastic.co/t/logstash-forwarder-documentaion/36759)
**Author:** [@NiteshBarnwal](https://discuss.elastic.co/u/NiteshBarnwal)\ **Replies:** 5\ **Last updated:** [December 10, 2015, 7:07am UTC](https://discuss.elastic.co/t/logstash-forwarder-documentaion/36759 "2015-12-10T07:07:15Z")
Hi Team, Is there any document for setting up logstash-forwarder ? --- ## [Having file and elasticsearch output with filtering only the elastic search](https://discuss.elastic.co/t/having-file-and-elasticsearch-output-with-filtering-only-the-elastic-search/36807)
**Author:** [@shanemgleeson](https://discuss.elastic.co/u/shanemgleeson)\ **Replies:** 1\ **Last updated:** [December 10, 2015, 6:54am UTC](https://discuss.elastic.co/t/having-file-and-elasticsearch-output-with-filtering-only-the-elastic-search/36807 "2015-12-10T06:54:26Z")
Hi, I was hoping to use the ELK stack to take over from an old scp and regex based log centralization we have running. Right now all in house application logs are collecting at a central location using a remote ssh cal… --- ## [Logstash and IDMEF](https://discuss.elastic.co/t/logstash-and-idmef/36745)
**Author:** [@saidiahd](https://discuss.elastic.co/u/saidiahd)\ **Replies:** 2\ **Last updated:** [December 10, 2015, 3:36am UTC](https://discuss.elastic.co/t/logstash-and-idmef/36745 "2015-12-10T03:36:07Z")
Hi Folk, is logstash support Collect, Enrich & Transport Data normalized in IDMEF (Intrusion Detection Message Exchange Format) format ? IDMEF it's define data formats and exchange procedures for sharing information of … --- ## [Start logstash with jdbc](https://discuss.elastic.co/t/start-logstash-with-jdbc/36820)
**Author:** [@supermario](https://discuss.elastic.co/u/supermario)\ **Replies:** 0\ **Last updated:** [December 10, 2015, 1:30am UTC](https://discuss.elastic.co/t/start-logstash-with-jdbc/36820 "2015-12-10T01:30:37Z")
I want to use logstash-input-jdbc in my logstash application on the windows. My jdk version is 1.8 , logstash version is 2.1 and use sqljdbc4.jar(version 4.2 or version 6.0).Logstash config : input { tcp { codec =\> "j… --- ## [Logstash + Collectd](https://discuss.elastic.co/t/logstash-collectd/36757)
**Author:** [@Christophe\_Dumont](https://discuss.elastic.co/u/Christophe_Dumont)\ **Replies:** 3\ **Last updated:** [December 10, 2015, 1:13am UTC](https://discuss.elastic.co/t/logstash-collectd/36757 "2015-12-10T01:13:25Z")
Hello, I installed collectd, logstash, elasticsearch and kibana. \[:grin:\] I'd like to generate stats from collectd and send them to ec via logstash. It dosen't work. Here are my conf files : logstash.conf input { … --- ## [Jdbc in filter section](https://discuss.elastic.co/t/jdbc-in-filter-section/36280)
**Author:** [@adellarocca](https://discuss.elastic.co/u/adellarocca)\ **Replies:** 7\ **Last updated:** [December 10, 2015, 12:14am UTC](https://discuss.elastic.co/t/jdbc-in-filter-section/36280 "2015-12-10T00:14:53Z")
I need to lookup fields in a RDBMS (MySQL) during filter phase of Logstash to extract new informations. Neverthless jdbc plugin exists only for input section. I can't do this in input section because my fields are availa… --- ## [\[sincedb creation\]](https://discuss.elastic.co/t/sincedb-creation/34851)
**Author:** [@varun\_kumar1](https://discuss.elastic.co/u/varun_kumar1)\ **Replies:** 32\ **Last updated:** [December 9, 2015, 9:19pm UTC](https://discuss.elastic.co/t/sincedb-creation/34851 "2015-12-09T21:19:02Z")
I have a following config file for logstasth input { file{ path =\> "D:/Log/\*/\*" } file{ path =\> "D:/Log/\*/MIP/\*" } file{ path =\> "D:/Log/\*/NHGS/\*" } } filter{ mutate{ gsub =\> \["path","D:/Log/", ""\] } … --- ## [Creating grok patterns for Horizon View debug](https://discuss.elastic.co/t/creating-grok-patterns-for-horizon-view-debug/36785)
**Author:** [@havealoha](https://discuss.elastic.co/u/havealoha)\ **Replies:** 1\ **Last updated:** [December 9, 2015, 7:07pm UTC](https://discuss.elastic.co/t/creating-grok-patterns-for-horizon-view-debug/36785 "2015-12-09T19:07:53Z")
I am a little confused about how to regex\\grok the data from the Horizon View debug logs that we are forwarding to logstash. I dont know what regex to put in the pattern. Maybe I dont even have this close? Here are the… --- ## [Multiline not reading all lines?](https://discuss.elastic.co/t/multiline-not-reading-all-lines/36762)
**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Replies:** 2\ **Last updated:** [December 9, 2015, 7:07pm UTC](https://discuss.elastic.co/t/multiline-not-reading-all-lines/36762 "2015-12-09T19:07:25Z")
I have some logs all starting with a timestamp, javaclass, log level and then a message, some of them multiline. I'm just starting with a couple single-line events first. I have two of them in a file and I have the foll… --- ## [When is 2.1.1 coming out? Reaching a breaking point with this Windows JVM Memory Leak stuff](https://discuss.elastic.co/t/when-is-2-1-1-coming-out-reaching-a-breaking-point-with-this-windows-jvm-memory-leak-stuff/36338)
**Author:** [@h2k](https://discuss.elastic.co/u/h2k)\ **Replies:** 6\ **Last updated:** [December 9, 2015, 6:27pm UTC](https://discuss.elastic.co/t/when-is-2-1-1-coming-out-reaching-a-breaking-point-with-this-windows-jvm-memory-leak-stuff/36338 "2015-12-09T18:27:58Z")
I am running Logstash 1.5.4 on a number of Windows servers as an agent to collect eventlog and other data. I've done a decent job of tweaking the configuration such that it minimizes the use of expensive filtering and ot… --- ## [Ignore plugin error](https://discuss.elastic.co/t/ignore-plugin-error/36013)
**Author:** [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Replies:** 2\ **Last updated:** [December 9, 2015, 5:55pm UTC](https://discuss.elastic.co/t/ignore-plugin-error/36013 "2015-12-09T17:55:55Z")
I am using geoip plugin to geo code IP, but sometimes, event are coming with a wrong IP then geoip plugin throws error then nothing is output \[:confused:\] https://github.com/logstash-plugins/logstash-filter-geoip/blob… --- ## [Filebeat messages reaching logstash server, but then not ES](https://discuss.elastic.co/t/filebeat-messages-reaching-logstash-server-but-then-not-es/36721)
**Author:** [@Cylindric](https://discuss.elastic.co/u/Cylindric)\ **Replies:** 4\ **Last updated:** [December 9, 2015, 4:34pm UTC](https://discuss.elastic.co/t/filebeat-messages-reaching-logstash-server-but-then-not-es/36721 "2015-12-09T16:34:57Z")
I've got an odd situation I'm not sure how to resolve. I have an existing ELK setup, mostly using Filebeat to ship web logs. I am now trying to ship some CSV-based data into ES, so I'm using the csv{} stuff in logstash. M… --- ## [Unable to install logstash twitter plugin - File Not Found Exception](https://discuss.elastic.co/t/unable-to-install-logstash-twitter-plugin-file-not-found-exception/36682)
**Author:** [@mob](https://discuss.elastic.co/u/mob)\ **Replies:** 0\ **Last updated:** [December 8, 2015, 11:52pm UTC](https://discuss.elastic.co/t/unable-to-install-logstash-twitter-plugin-file-not-found-exception/36682 "2015-12-08T23:52:39Z")
I installed logstash 2.1.0 using the .deb package but when I try and install the twitter plugin i get the following FileNotFound exception Am I looking in the correct place? root@logger:/usr/share/elasticsearch# bin… --- ## [Connect multiple logstash to single elastic search](https://discuss.elastic.co/t/connect-multiple-logstash-to-single-elastic-search/36748)
**Author:** [@tejus](https://discuss.elastic.co/u/tejus)\ **Replies:** 4\ **Last updated:** [December 9, 2015, 2:57pm UTC](https://discuss.elastic.co/t/connect-multiple-logstash-to-single-elastic-search/36748 "2015-12-09T14:57:16Z")
i have an elastic search instance in found and i want to push data from different servers where logstash is running. these logstash instances are in different countries. will it work ?? --- ## [Find if logstash is omitting or skipping any data?](https://discuss.elastic.co/t/find-if-logstash-is-omitting-or-skipping-any-data/36750)
**Author:** [@tejus](https://discuss.elastic.co/u/tejus)\ **Replies:** 0\ **Last updated:** [December 9, 2015, 2:55pm UTC](https://discuss.elastic.co/t/find-if-logstash-is-omitting-or-skipping-any-data/36750 "2015-12-09T14:55:31Z")
i'm having a logstash instance where the data is live tweets data. but i'm also pushing historical tweets into logstash from a different source. i want to check if the bursts of data will be handled if i push 1 year data… --- ## [Unable to find JRuby](https://discuss.elastic.co/t/unable-to-find-jruby/33532)
**Author:** [@clemo](https://discuss.elastic.co/u/clemo)\ **Replies:** 1\ **Last updated:** [December 9, 2015, 2:00pm UTC](https://discuss.elastic.co/t/unable-to-find-jruby/33532 "2015-12-09T14:00:45Z")
Hi, Total newbie to logstash, have just downloaded it and unzipped it and then tried to execute the example on your webpages logstash-2.0.0\>bin\\logstash -e 'input { stdin { } } output { stdout {} }' and get the foll… --- ## [Error using twitter input LS 2.1.1](https://discuss.elastic.co/t/error-using-twitter-input-ls-2-1-1/36734)
**Author:** [@lwintergerst](https://discuss.elastic.co/u/lwintergerst)\ **Replies:** 0\ **Last updated:** [December 9, 2015, 11:24am UTC](https://discuss.elastic.co/t/error-using-twitter-input-ls-2-1-1/36734 "2015-12-09T11:24:12Z")
I just tried to use the new proxy setting with logstash 2.1.1 Unfortunatel y I get this error. The log file grows by about 5MB per second after starting logstash. Cant get the formatting to wark at the moment This is … --- ## [Mixed JSON input and Other Log files with Multiline codec](https://discuss.elastic.co/t/mixed-json-input-and-other-log-files-with-multiline-codec/36696)
**Author:** [@Raggyman](https://discuss.elastic.co/u/Raggyman)\ **Replies:** 3\ **Last updated:** [December 9, 2015, 10:50am UTC](https://discuss.elastic.co/t/mixed-json-input-and-other-log-files-with-multiline-codec/36696 "2015-12-09T10:50:52Z")
I have the following configured for input. My situation that I have a mix of different file formats that I need to process. Some files are the generic type of file, that have the same start pattern for multiline, (a d… --- ## [Can't make the "aggregate" filter to work](https://discuss.elastic.co/t/cant-make-the-aggregate-filter-to-work/36725)
**Author:** [@Benoit\_Chauvet](https://discuss.elastic.co/u/Benoit_Chauvet)\ **Replies:** 0\ **Last updated:** [December 9, 2015, 10:27am UTC](https://discuss.elastic.co/t/cant-make-the-aggregate-filter-to-work/36725 "2015-12-09T10:27:19Z")
Hi there, I'd like to use aggregate filter to gather informations from different log lines, but I'm not able to make it write anything in the logs. I tried to add the following lines in my logstash configuration : … --- ## [How to filter data from Filebeats in Logstash](https://discuss.elastic.co/t/how-to-filter-data-from-filebeats-in-logstash/36579)
**Author:** [@msmolyak](https://discuss.elastic.co/u/msmolyak)\ **Replies:** 5\ **Last updated:** [December 9, 2015, 7:24am UTC](https://discuss.elastic.co/t/how-to-filter-data-from-filebeats-in-logstash/36579 "2015-12-09T07:24:57Z")
I am designing an ELK stack with ElasticSearch, Logstash and Kibana installed on a dedicated server and Filebeats installed on all the machine generating log files. Sounds like a standard setup. My question is how to … --- ## [Logstash intergation with AWS Elasticsearch](https://discuss.elastic.co/t/logstash-intergation-with-aws-elasticsearch/36623)
**Author:** [@Siddharth](https://discuss.elastic.co/u/Siddharth)\ **Replies:** 2\ **Last updated:** [December 9, 2015, 5:40am UTC](https://discuss.elastic.co/t/logstash-intergation-with-aws-elasticsearch/36623 "2015-12-09T05:40:06Z")
Hello, I am using AWS Elasticsearch service to configure Elasticsearch Cluster and there is a separate server where I have installed Logstash 2.1.0 Here is my Logstash sample configuration file :- input { fil… --- ## [ClientProtocolException](https://discuss.elastic.co/t/clientprotocolexception/36515)
**Author:** [@treehouse](https://discuss.elastic.co/u/treehouse)\ **Replies:** 4\ **Last updated:** [December 8, 2015, 11:05pm UTC](https://discuss.elastic.co/t/clientprotocolexception/36515 "2015-12-08T23:05:15Z")
Hi all I'm writing to about 5 sets of indices from logstash. One of those writes is failing for some reason. Event data coming in fairly sporadically (a couple hundred a day). Seems like for every event that tries to … --- ## [Logstash - Getting failed install template](https://discuss.elastic.co/t/logstash-getting-failed-install-template/36324)
**Author:** [@Kaufusihm](https://discuss.elastic.co/u/Kaufusihm)\ **Replies:** 7\ **Last updated:** [December 8, 2015, 9:49pm UTC](https://discuss.elastic.co/t/logstash-getting-failed-install-template/36324 "2015-12-08T21:49:18Z")
I am not able to get Elasticsearch with shield working with Logstash. Here is my output-config for logstash: output { elasticsearch { Protocol =\> "http" user =\> "ls\_user" password =\> "… --- ## [What grok patterns are installed by default?](https://discuss.elastic.co/t/what-grok-patterns-are-installed-by-default/36663)
**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Replies:** 2\ **Last updated:** [December 8, 2015, 6:37pm UTC](https://discuss.elastic.co/t/what-grok-patterns-are-installed-by-default/36663 "2015-12-08T18:37:30Z")
I'm using Grok Constructor and it has 13 different "grok patterns" listed and all checked by default, e.g. "firewalls". So far I've been unchecking all of them except for "grok-patterns" not being sure the others were av… --- ## [Map ID to predefined name](https://discuss.elastic.co/t/map-id-to-predefined-name/36660)
**Author:** [@lwintergerst](https://discuss.elastic.co/u/lwintergerst)\ **Replies:** 2\ **Last updated:** [December 8, 2015, 6:25pm UTC](https://discuss.elastic.co/t/map-id-to-predefined-name/36660 "2015-12-08T18:25:35Z")
Hi, our application currently logs a locationId, which represents a city. At some points it is not possible to log the name, since some parts of the application only know the ID. I'd like to enrich my data witch logs… --- ## [Logstash is not start as expected](https://discuss.elastic.co/t/logstash-is-not-start-as-expected/36433)
**Author:** [@asifalis](https://discuss.elastic.co/u/asifalis)\ **Replies:** 17\ **Last updated:** [December 8, 2015, 6:02pm UTC](https://discuss.elastic.co/t/logstash-is-not-start-as-expected/36433 "2015-12-08T18:02:46Z")
Hi Expert, I installed Kibana 4.2 and Elasticsearch 2.1 and logstash 2.0, Kibana and elasticsearch working fine but logstash not working, the logstash.log \[root@centos logstash\]# tailf /var/log/logstash/logstash.log … --- ## [JSON en-/decoding issues starting with version 1.5.5](https://discuss.elastic.co/t/json-en-decoding-issues-starting-with-version-1-5-5/36543)
**Author:** [@USP\_rol](https://discuss.elastic.co/u/USP_rol)\ **Replies:** 1\ **Last updated:** [December 8, 2015, 2:29pm UTC](https://discuss.elastic.co/t/json-en-decoding-issues-starting-with-version-1-5-5/36543 "2015-12-08T14:29:11Z")
Hi all I'll try to summarize our issue we're having here a bit to share as much information as possible. We're running a logstash architecture Shipper -\> Redis-Queue -\> Indexer ( -\> graphite output) which runs fine wit… --- ## [Puppet: automatic installation of two logstash instances](https://discuss.elastic.co/t/puppet-automatic-installation-of-two-logstash-instances/36559)
**Author:** [@MartinG](https://discuss.elastic.co/u/MartinG)\ **Replies:** 0\ **Last updated:** [December 7, 2015, 5:09pm UTC](https://discuss.elastic.co/t/puppet-automatic-installation-of-two-logstash-instances/36559 "2015-12-07T17:09:17Z")
Hello! Did somebody manage to install two independet logstash instances on the same host with Puppet? I tried the elasticsearch/logstash Puppet module but didn't manage to create two instaces. Does anybody tried so… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1210) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1212)