# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1143 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1144 --- ## [What are the use cases for the file plugin v.s. logstash?](https://discuss.elastic.co/t/what-are-the-use-cases-for-the-file-plugin-v-s-logstash/52250)
**Author:** [@matanster](https://discuss.elastic.co/u/matanster)\ **Replies:** 12\ **Last updated:** [June 10, 2016, 7:14pm UTC](https://discuss.elastic.co/t/what-are-the-use-cases-for-the-file-plugin-v-s-logstash/52250 "2016-06-10T19:14:45Z")
Hi, I have looked at the file plugin. and am not quite sure what use cases is it meant for. Other than when sharing some sort of shared network storage, it seems as if this plugin can only ingest files from the local h… --- ## [Logstash not matching substring of custom field](https://discuss.elastic.co/t/logstash-not-matching-substring-of-custom-field/52389)
**Author:** [@Josh\_Reichardt](https://discuss.elastic.co/u/Josh_Reichardt)\ **Replies:** 1\ **Last updated:** [June 10, 2016, 4:21pm UTC](https://discuss.elastic.co/t/logstash-not-matching-substring-of-custom-field/52389 "2016-06-10T16:21:53Z")
I have the following Logstash setup. Logs are piped to Logstash from Logspout via the logspout-logstash adapter, which adds a few fields to log messages, namely the docker.image field. I am able to ingest the logs bu… --- ## [\[solved\]\[solved\]](https://discuss.elastic.co/t/solved-solved/52141)
**Author:** [@mattcrawfoord](https://discuss.elastic.co/u/mattcrawfoord)\ **Replies:** 2\ **Last updated:** [June 10, 2016, 3:19pm UTC](https://discuss.elastic.co/t/solved-solved/52141 "2016-06-10T15:19:58Z")
duplication question \[solved\] --- ## [Enable logging in the plugin ruby script](https://discuss.elastic.co/t/enable-logging-in-the-plugin-ruby-script/27335)
**Author:** [@rishi](https://discuss.elastic.co/u/rishi)\ **Replies:** 2\ **Last updated:** [June 10, 2016, 2:36pm UTC](https://discuss.elastic.co/t/enable-logging-in-the-plugin-ruby-script/27335 "2016-06-10T14:36:14Z")
I wanted to enable logging in my plugin ruby script and added the following line in the register method: LogStash::Logger.setup\_log4j(@logger) I have the log4j and the slf4j jars in my runtime dependencies. When I … --- ## [Problem sending data from logstash to elasticsearch](https://discuss.elastic.co/t/problem-sending-data-from-logstash-to-elasticsearch/52353)
**Author:** [@Rom](https://discuss.elastic.co/u/Rom)\ **Replies:** 3\ **Last updated:** [June 10, 2016, 2:16pm UTC](https://discuss.elastic.co/t/problem-sending-data-from-logstash-to-elasticsearch/52353 "2016-06-10T14:16:54Z")
Hi, I am trying to analyse my data in Kibana , from a CSV file, however logstash is not sending the data to elasticsearch. Please find the screenshots for details. Can some one tell me where exactly I am going wrong? … --- ## [How to deal with different log in input?](https://discuss.elastic.co/t/how-to-deal-with-different-log-in-input/52146)
**Author:** [@WangXiangUSTC](https://discuss.elastic.co/u/WangXiangUSTC)\ **Replies:** 2\ **Last updated:** [June 10, 2016, 12:12pm UTC](https://discuss.elastic.co/t/how-to-deal-with-different-log-in-input/52146 "2016-06-10T12:12:45Z")
I have a lot of logs to deal,they have different format. For example, some are one log with one line, but some are one log with several lines. First I use multiline in filter, but the logstash show: Defaulting pipeli… --- ## [Filebeat and Multiline](https://discuss.elastic.co/t/filebeat-and-multiline/38499)
**Author:** [@ori.rubinfeld](https://discuss.elastic.co/u/ori.rubinfeld)\ **Replies:** 11\ **Last updated:** [June 10, 2016, 12:12pm UTC](https://discuss.elastic.co/t/filebeat-and-multiline/38499 "2016-06-10T12:12:19Z")
Hi, We would like to implement the Filebeat as a shipper for log files. Some of the log files are multilines. Some servers are sending same types of logs. What is the best way of doing it ? Can I use conditions on … --- ## [How to get json subfield from input and use it as document\_id elastic?](https://discuss.elastic.co/t/how-to-get-json-subfield-from-input-and-use-it-as-document-id-elastic/52334)
**Author:** [@breezzz](https://discuss.elastic.co/u/breezzz)\ **Replies:** 2\ **Last updated:** [June 10, 2016, 11:33am UTC](https://discuss.elastic.co/t/how-to-get-json-subfield-from-input-and-use-it-as-document-id-elastic/52334 "2016-06-10T11:33:21Z")
Good day! I have this work config for logstash: input { kafka { zk\_connect =\> "kafka:2181" topic\_id =\> "requests" group\_id =\> "kafka\_requests" consumer\_threads =\> 1 } } output { elasticsearch { codec =\> … --- ## [Logstash failed to load files with permission denied](https://discuss.elastic.co/t/logstash-failed-to-load-files-with-permission-denied/51255)
**Author:** [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\ **Replies:** 6\ **Last updated:** [June 10, 2016, 11:26am UTC](https://discuss.elastic.co/t/logstash-failed-to-load-files-with-permission-denied/51255 "2016-06-10T11:26:07Z")
Logstash failed to load files with permission denied. In my setup Logstash running as service with command of "service logstash start" folder also having full permissions -rwxrwxrwx.. Please suggest what can be the caus… --- ## [Grok too slow? \[solved\]](https://discuss.elastic.co/t/grok-too-slow-solved/52348)
**Author:** [@MikeC](https://discuss.elastic.co/u/MikeC)\ **Replies:** 2\ **Last updated:** [June 10, 2016, 9:36am UTC](https://discuss.elastic.co/t/grok-too-slow-solved/52348 "2016-06-10T09:36:00Z")
Hi, There was suggested in this thread that my grok might cause issues with the performance of my filebeat/logstash/elasticseach setup. This is my config file // more if \[source\] checks are here // .... … --- ## [\[Solved\] Logstash does not work with beats](https://discuss.elastic.co/t/solved-logstash-does-not-work-with-beats/52422)
**Author:** [@bibl2008](https://discuss.elastic.co/u/bibl2008)\ **Replies:** 3\ **Last updated:** [June 10, 2016, 9:20am UTC](https://discuss.elastic.co/t/solved-logstash-does-not-work-with-beats/52422 "2016-06-10T09:20:22Z")
Hello, 1. I installed the Logstash logstash-2.3.2-1.noarch.rpm to CentOS. 2. Changed the config: input { beats { port =\> 5044 } } output { elasticsearch { … --- ## [Logstash pipeline is starting but not executing](https://discuss.elastic.co/t/logstash-pipeline-is-starting-but-not-executing/52288)
**Author:** [@ELK1](https://discuss.elastic.co/u/ELK1)\ **Replies:** 6\ **Last updated:** [June 10, 2016, 7:43am UTC](https://discuss.elastic.co/t/logstash-pipeline-is-starting-but-not-executing/52288 "2016-06-10T07:43:27Z")
Hi My logstash config file is input { file { path =\> "/home/Esearch/inp-lgstsh" start\_position =\> "beginning" ignore\_older =\> 0 } } filter { if \[path\] =~ "access" { mutate { replace =\> { "type" … --- ## [Issue changing value of JSON field in logstash](https://discuss.elastic.co/t/issue-changing-value-of-json-field-in-logstash/52309)
**Author:** [@Mick\_Mahoney](https://discuss.elastic.co/u/Mick_Mahoney)\ **Replies:** 2\ **Last updated:** [June 10, 2016, 7:33am UTC](https://discuss.elastic.co/t/issue-changing-value-of-json-field-in-logstash/52309 "2016-06-10T07:33:46Z")
Hi, I'm trying to change a value in logstash of JSON which has been generated by Topbeat Below are the input file name, the logstash filter, and the error that I am getting Can anyone spot why I'm getting the error… --- ## [Logstash in reading files issues](https://discuss.elastic.co/t/logstash-in-reading-files-issues/52275)
**Author:** [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Replies:** 2\ **Last updated:** [June 10, 2016, 6:01am UTC](https://discuss.elastic.co/t/logstash-in-reading-files-issues/52275 "2016-06-10T06:01:05Z")
I have written a logstash conf file to read through logs. It run fine when I specify the log path to a single file. However, if I made it to \*.log, it can't run any files. How can I resolve it? Here is the one which wor… --- ## [Merging two log sources](https://discuss.elastic.co/t/merging-two-log-sources/52091)
**Author:** [@falcon](https://discuss.elastic.co/u/falcon)\ **Replies:** 1\ **Last updated:** [June 10, 2016, 5:37am UTC](https://discuss.elastic.co/t/merging-two-log-sources/52091 "2016-06-10T05:37:45Z")
I need to merge access.log with application.log based on equal timestamp +/- delta, thread name and request URI to get one event (instead two) saved into DB. Is it possible with logstach or I need some preprocessing befo… --- ## [JDBC input failure connecting to FileMaker](https://discuss.elastic.co/t/jdbc-input-failure-connecting-to-filemaker/52082)
**Author:** [@belac9615](https://discuss.elastic.co/u/belac9615)\ **Replies:** 1\ **Last updated:** [June 10, 2016, 5:37am UTC](https://discuss.elastic.co/t/jdbc-input-failure-connecting-to-filemaker/52082 "2016-06-10T05:37:27Z")
I'm having an issue using the JDBC input in Logstash while attempting to pull all of the records out of our inventory database. When I run Logstash with the --debug option I see this error. (I "WHERE'd" it down to one pa… --- ## [How to automate data ingestion with logstash pipelines?](https://discuss.elastic.co/t/how-to-automate-data-ingestion-with-logstash-pipelines/52029)
**Author:** [@qbadx](https://discuss.elastic.co/u/qbadx)\ **Replies:** 3\ **Last updated:** [June 10, 2016, 5:35am UTC](https://discuss.elastic.co/t/how-to-automate-data-ingestion-with-logstash-pipelines/52029 "2016-06-10T05:35:15Z")
Hi there, I'm using ELK in order to ingest, store and visualize data, no fancy things.. Everything is working fine but each time I have new data to ingest I have to execute manually the command /opt/logstash/bin/log… --- ## [Logstash - filter and drop lines that record\_date is older than 30mins of current timestamp?](https://discuss.elastic.co/t/logstash-filter-and-drop-lines-that-record-date-is-older-than-30mins-of-current-timestamp/51975)
**Author:** [@mt.michael](https://discuss.elastic.co/u/mt.michael)\ **Replies:** 3\ **Last updated:** [June 10, 2016, 5:33am UTC](https://discuss.elastic.co/t/logstash-filter-and-drop-lines-that-record-date-is-older-than-30mins-of-current-timestamp/51975 "2016-06-10T05:33:57Z")
Hi. I have issue where I'd duplicate data send to the target through filebeat \> logstash \> target. I couldn't figure what was the issue on the server as it was terminated in aws autoscale so I'm thinking of having a fi… --- ## [Extracting week count from current System time using ruby](https://discuss.elastic.co/t/extracting-week-count-from-current-system-time-using-ruby/51907)
**Author:** [@gauravkb](https://discuss.elastic.co/u/gauravkb)\ **Replies:** 1\ **Last updated:** [June 10, 2016, 5:29am UTC](https://discuss.elastic.co/t/extracting-week-count-from-current-system-time-using-ruby/51907 "2016-06-10T05:29:28Z")
Hi im trying to extract the week count from the current System time using ruby filter but im getting error this is how my filter looks ruby { code =\> "event\['WeekNumber'\]= Time.now.strftime("%U").to\_i" } anyone p… --- ## [Combining value in DB through Logstash](https://discuss.elastic.co/t/combining-value-in-db-through-logstash/52400)
**Author:** [@wldbs0508](https://discuss.elastic.co/u/wldbs0508)\ **Replies:** 0\ **Last updated:** [June 10, 2016, 4:21am UTC](https://discuss.elastic.co/t/combining-value-in-db-through-logstash/52400 "2016-06-10T04:21:46Z")
Hi, I would like to connect my DB with ES. My DB has multiple columns such as date, unique ID, and JSON. The JSON column includes multiple JSON code as text or string. Here is an example of value in JSON column: \[ { "… --- ## [Input plugin - type not getting picked up?](https://discuss.elastic.co/t/input-plugin-type-not-getting-picked-up/52156)
**Author:** [@jamesl](https://discuss.elastic.co/u/jamesl)\ **Replies:** 2\ **Last updated:** [June 10, 2016, 1:38am UTC](https://discuss.elastic.co/t/input-plugin-type-not-getting-picked-up/52156 "2016-06-10T01:38:32Z")
Hi. Have installed filebeat on a Windows box to collect and forward logs to ELK. On the logstash input file i have setup a seperate beats section so i can define a different type compared to my other inputs, refer winl… --- ## [Logstash 5.0 - Kafka Input installation issue](https://discuss.elastic.co/t/logstash-5-0-kafka-input-installation-issue/52374)
**Author:** [@urover](https://discuss.elastic.co/u/urover)\ **Replies:** 0\ **Last updated:** [June 9, 2016, 8:34pm UTC](https://discuss.elastic.co/t/logstash-5-0-kafka-input-installation-issue/52374 "2016-06-09T20:34:24Z")
Using Logstash 5.0 Alpha. Trying to install the kafka input plugin returns this error: bin/logstash-plugin install --version 3.0.0.beta3 logstash-input-kafka Validating logstash-input-kafka-3.0.0.beta3 Installing log… --- ## [Dateparsefailure - Error in Date value](https://discuss.elastic.co/t/dateparsefailure-error-in-date-value/52326)
**Author:** [@anon45970649](https://discuss.elastic.co/u/anon45970649)\ **Replies:** 3\ **Last updated:** [June 9, 2016, 8:11pm UTC](https://discuss.elastic.co/t/dateparsefailure-error-in-date-value/52326 "2016-06-09T20:11:57Z")
Hello everyone, I'm trying a few days to enter the csv file data, but I have not been able to take the lead in this error in the date field. This message: ←\[33mFailed parsing date from field {:field=\>"Datahora", :valu… --- ## [Grok filter 2 different logs](https://discuss.elastic.co/t/grok-filter-2-different-logs/51818)
**Author:** [@bishaka](https://discuss.elastic.co/u/bishaka)\ **Replies:** 5\ **Last updated:** [June 9, 2016, 8:00pm UTC](https://discuss.elastic.co/t/grok-filter-2-different-logs/51818 "2016-06-09T20:00:58Z")
Hi, I am trying to use the following grok filter to extract fields from log file, The log file contains 2 different style of logs, hence I wrote 2 different queries to tackle the 2 logs. However, it's failing on one of t… --- ## [Exec command not triggering on logfile](https://discuss.elastic.co/t/exec-command-not-triggering-on-logfile/52364)
**Author:** [@cj\_noriega](https://discuss.elastic.co/u/cj_noriega)\ **Replies:** 0\ **Last updated:** [June 9, 2016, 5:55pm UTC](https://discuss.elastic.co/t/exec-command-not-triggering-on-logfile/52364 "2016-06-09T17:55:08Z")
Hello, I'd appreciate some guidance where logstash isn't triggering our exec command in spite of there being a valid string found by our filter. We have a third party service running on our Windows server which is wr… --- ## [Lumberjack java.nio.BufferOverflowException](https://discuss.elastic.co/t/lumberjack-java-nio-bufferoverflowexception/52337)
**Author:** [@YLombardi](https://discuss.elastic.co/u/YLombardi)\ **Replies:** 0\ **Last updated:** [June 9, 2016, 2:13pm UTC](https://discuss.elastic.co/t/lumberjack-java-nio-bufferoverflowexception/52337 "2016-06-09T14:13:40Z")
Hi. Sometime my logstash crash and I have this error in logstash.err : And this in logstash.log : It seems to be a problem of memory. I increased the LS\_HEAP\_SIZE to 2g but it doesn't solve the problem. Here is m… --- ## [Annoying - filter ruby code multiplication doesnt work](https://discuss.elastic.co/t/annoying-filter-ruby-code-multiplication-doesnt-work/52187)
**Author:** [@Mick\_Mahoney](https://discuss.elastic.co/u/Mick_Mahoney)\ **Replies:** 4\ **Last updated:** [June 9, 2016, 9:37am UTC](https://discuss.elastic.co/t/annoying-filter-ruby-code-multiplication-doesnt-work/52187 "2016-06-09T09:37:34Z")
Can anyone please help with this ... its annoying me no end \[:slight\_smile:\] field cpu.user\_p=0.45 This filter works and allocates cpu\_user\_p=45 ruby { code =\> "event\['cpu\_user\_p'\] = 0.45 \* 100" } This filter doe… --- ## [Logstash Filter Issue with Apache2](https://discuss.elastic.co/t/logstash-filter-issue-with-apache2/52280)
**Author:** [@harshanarayana](https://discuss.elastic.co/u/harshanarayana)\ **Replies:** 3\ **Last updated:** [June 9, 2016, 8:52am UTC](https://discuss.elastic.co/t/logstash-filter-issue-with-apache2/52280 "2016-06-09T08:52:20Z")
Hi All, I've recently started using the Log handler Trio of Logstash + ELK + Kibana to work analyze through my apache logs during my daily work activities. However, I'm facing some issues with Grok when it comes to fi… --- ## [Filebeat subdirectories and logstash multiline problems](https://discuss.elastic.co/t/filebeat-subdirectories-and-logstash-multiline-problems/52060)
**Author:** [@elkforce](https://discuss.elastic.co/u/elkforce)\ **Replies:** 2\ **Last updated:** [June 9, 2016, 8:49am UTC](https://discuss.elastic.co/t/filebeat-subdirectories-and-logstash-multiline-problems/52060 "2016-06-09T08:49:39Z")
Hello, i want ship user logs(java multiline) with filebeat to logstash, but the multiline Events get events from another Event. MY CONFIG: The user logs are: /tmp/user1/application.log /tmp/user2/application.log /tmp… --- ## [Can I use Logstash to add new fields to a document that's already in Elasticsearch?](https://discuss.elastic.co/t/can-i-use-logstash-to-add-new-fields-to-a-document-thats-already-in-elasticsearch/52238)
**Author:** [@yannakopoulos](https://discuss.elastic.co/u/yannakopoulos)\ **Replies:** 2\ **Last updated:** [June 9, 2016, 5:41am UTC](https://discuss.elastic.co/t/can-i-use-logstash-to-add-new-fields-to-a-document-thats-already-in-elasticsearch/52238 "2016-06-09T05:41:23Z")
I'm looking to consolidate several log files into one index. I have a unique ID that I can use to link a document of one type to a document of the other. Is there a way to use Logstash to update existing documents by add… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1142) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1144)