# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1160 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1161 --- ## [Configuring logstash to Check files automatically](https://discuss.elastic.co/t/configuring-logstash-to-check-files-automatically/48647)
**Author:** [@maher](https://discuss.elastic.co/u/maher)\ **Replies:** 3\ **Last updated:** [April 30, 2016, 2:52pm UTC](https://discuss.elastic.co/t/configuring-logstash-to-check-files-automatically/48647 "2016-04-30T14:52:49Z")
Hi, I am trying to configure logstash to automatically wake up when a new file apears or when a file is modified. For sure I can use a script to execute logstash periodically... But is there any other alternative? lik… --- ## [How to get field value in logstash?](https://discuss.elastic.co/t/how-to-get-field-value-in-logstash/48757)
**Author:** [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Replies:** 4\ **Last updated:** [April 30, 2016, 2:50pm UTC](https://discuss.elastic.co/t/how-to-get-field-value-in-logstash/48757 "2016-04-30T14:50:22Z")
from my output as below, "message" =\>"\<....... ", "@version" =\> "1", "@timestamp" =\> "2016-04-29T02:33:34.586Z", "timestamp" =\> "Apr 29 10:30:37", "syslog\_severity\_code" =\> 5, "syslog\_facility\_code" =\> 1, "syslog… --- ## [Multiple Logstah indexer/filter hosts causing duplicate elasticsearch documents](https://discuss.elastic.co/t/multiple-logstah-indexer-filter-hosts-causing-duplicate-elasticsearch-documents/48837)
**Author:** [@Terrac\_Skiens\_Vendin](https://discuss.elastic.co/u/Terrac_Skiens_Vendin)\ **Replies:** 1\ **Last updated:** [April 30, 2016, 2:11am UTC](https://discuss.elastic.co/t/multiple-logstah-indexer-filter-hosts-causing-duplicate-elasticsearch-documents/48837 "2016-04-30T02:11:23Z")
The setup: logstash-forwarder (v0.4.0) -\> logstash shipper (v1.5.4) -\> redis (v2.8.19) -\> logstash indexer (v1.5.4) -\> elasticsearch (1.7.5) There's one logstash shipper Original setup had one logstash indexer, but I r… --- ## [Logstash TCP Client configuration](https://discuss.elastic.co/t/logstash-tcp-client-configuration/45716)
**Author:** [@Joseph\_Schwartz](https://discuss.elastic.co/u/Joseph_Schwartz)\ **Replies:** 2\ **Last updated:** [April 29, 2016, 8:32pm UTC](https://discuss.elastic.co/t/logstash-tcp-client-configuration/45716 "2016-04-29T20:32:23Z")
I'm looking for an better example to connect to a TCP server and save the results of a CSV stream to a file or directly into elastic. Does anyone have good example to work with? Sample of CSV Data: 1ADLAA1,GSM,14582… --- ## [Multiline](https://discuss.elastic.co/t/multiline/48158)
**Author:** [@Giulio](https://discuss.elastic.co/u/Giulio)\ **Replies:** 9\ **Last updated:** [April 29, 2016, 3:34pm UTC](https://discuss.elastic.co/t/multiline/48158 "2016-04-29T15:34:38Z")
Hi Sorry if I disturb you again but I but I'm totally out of ideas and I really need advice from the community... I'm trying to parse a multiline log like this one from file. \[Tue Apr 19 10:31:27 2016\] \[debug\] ssl\_en… --- ## [Error in shipping logs from Logstash to Elasticsearch](https://discuss.elastic.co/t/error-in-shipping-logs-from-logstash-to-elasticsearch/41346)
**Author:** [@sukhada369](https://discuss.elastic.co/u/sukhada369)\ **Replies:** 6\ **Last updated:** [April 29, 2016, 3:07pm UTC](https://discuss.elastic.co/t/error-in-shipping-logs-from-logstash-to-elasticsearch/41346 "2016-04-29T15:07:15Z")
Error while forwarding filebeat indexed logs from Logstash to Elasticsearch. Following is the content ¨Beats input: the pipeline is blocked, temporary refusing new connection.¨ --- ## [\_grokparsefailure with collectd](https://discuss.elastic.co/t/-grokparsefailure-with-collectd/48762)
**Author:** [@vchav73](https://discuss.elastic.co/u/vchav73)\ **Replies:** 3\ **Last updated:** [April 29, 2016, 2:28pm UTC](https://discuss.elastic.co/t/-grokparsefailure-with-collectd/48762 "2016-04-29T14:28:27Z")
I have a simple config file: input { udp { port =\> 25826 buffer\_size =\> 1452 codec =\> collectd { } type =\> "collectd" } } output { if \[type\] == "collectd" { elasticsearch { index =\> "collectd… --- ## [Logstash pipeline: events get triplicated](https://discuss.elastic.co/t/logstash-pipeline-events-get-triplicated/48811)
**Author:** [@danygor](https://discuss.elastic.co/u/danygor)\ **Replies:** 0\ **Last updated:** [April 29, 2016, 1:59pm UTC](https://discuss.elastic.co/t/logstash-pipeline-events-get-triplicated/48811 "2016-04-29T13:59:54Z")
Hi all, I'm investigating a performance issue with my logstash implementation which I believe started with version 2.2. Made it short, my output got delayed in comparison to the input (input file plugin) and when copy-… --- ## [I do not get subtract dates and ruby logstash](https://discuss.elastic.co/t/i-do-not-get-subtract-dates-and-ruby-logstash/46212)
**Author:** [@olorasde](https://discuss.elastic.co/u/olorasde)\ **Replies:** 1\ **Last updated:** [April 29, 2016, 12:07pm UTC](https://discuss.elastic.co/t/i-do-not-get-subtract-dates-and-ruby-logstash/46212 "2016-04-29T12:07:19Z")
Hello, I'm trying to subtract two dates ISO 8601 type, to extract an XML plugin through the ruby. The goal is to get the difference in the form of an integer with which to operate later. I am using the latest version… --- ## [No results found in Kibana](https://discuss.elastic.co/t/no-results-found-in-kibana/47360)
**Author:** [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Replies:** 13\ **Last updated:** [April 29, 2016, 10:52am UTC](https://discuss.elastic.co/t/no-results-found-in-kibana/47360 "2016-04-29T10:52:24Z")
Hi, I just install all the latest kibana 4.5 / elasticsearch and logstash on a ubuntu server. I'm sending logs from syslog to the server but kibana still say that it can't find results. here is the conf file \[:sli… --- ## [Any way to compress output before sending to elasticsearch?](https://discuss.elastic.co/t/any-way-to-compress-output-before-sending-to-elasticsearch/48056)
**Author:** [@rahin88](https://discuss.elastic.co/u/rahin88)\ **Replies:** 3\ **Last updated:** [April 29, 2016, 7:01am UTC](https://discuss.elastic.co/t/any-way-to-compress-output-before-sending-to-elasticsearch/48056 "2016-04-29T07:01:02Z")
Hi All, Still new to ELK. I have logstash running on my machine generating 1mbps of plain text data(with lot of redundancy of text in logs). Is there a way to encode/compress this output so it consumes less bandwidth a… --- ## [Ruby exception occurred: undefined method \`localtime'](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-localtime/48751)
**Author:** [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Replies:** 1\ **Last updated:** [April 29, 2016, 5:33am UTC](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-localtime/48751 "2016-04-29T05:33:39Z")
hi all, I am using logstash 2.1.3 ,I want to show the @timestamp field in my timezone, like "@timestamp" =\> "2015-03-09T04:24:29.718+08:00", I try to using ruby conversion in filter. filter { ruby { co… --- ## [Howto control log output format](https://discuss.elastic.co/t/howto-control-log-output-format/48670)
**Author:** [@stefws](https://discuss.elastic.co/u/stefws)\ **Replies:** 2\ **Last updated:** [April 28, 2016, 10:45pm UTC](https://discuss.elastic.co/t/howto-control-log-output-format/48670 "2016-04-28T22:45:09Z")
I'm currently getting what appears to be json output on a single line in my logstash log file (-l §LS\_LOG\_FILE), how can I control format for this output so it might be more readable or at least line separated? --- ## [How to use Github plugin for Logstash?](https://discuss.elastic.co/t/how-to-use-github-plugin-for-logstash/48739)
**Author:** [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\ **Replies:** 0\ **Last updated:** [April 28, 2016, 9:32pm UTC](https://discuss.elastic.co/t/how-to-use-github-plugin-for-logstash/48739 "2016-04-28T21:32:21Z")
Can anyone please let me know how can I use the github plugin under Logstash input conf. ? --- ## [Date getting converted to UTC](https://discuss.elastic.co/t/date-getting-converted-to-utc/48546)
**Author:** [@AM51](https://discuss.elastic.co/u/AM51)\ **Replies:** 1\ **Last updated:** [April 28, 2016, 7:37pm UTC](https://discuss.elastic.co/t/date-getting-converted-to-utc/48546 "2016-04-28T19:37:01Z")
Date is automatically getting converted to UTC Timezone . How to specify timezone for the date fields . I am using logstash Thanks Archit --- ## [Logtash Error](https://discuss.elastic.co/t/logtash-error/48724)
**Author:** [@parthu](https://discuss.elastic.co/u/parthu)\ **Replies:** 1\ **Last updated:** [April 28, 2016, 7:30pm UTC](https://discuss.elastic.co/t/logtash-error/48724 "2016-04-28T19:30:28Z")
There's this error I am getting ever since I moved to Logstash 2.3.1. logstash-current/vendor/bundle/jruby/1.9/gems/mail-2.6.3/lib/mail/network/delivery\_methods/sendmail.rb:72 warning: unsupported popen option: err I… --- ## [Ship logs from appliances in form of syslog](https://discuss.elastic.co/t/ship-logs-from-appliances-in-form-of-syslog/48509)
**Author:** [@thyfere](https://discuss.elastic.co/u/thyfere)\ **Replies:** 4\ **Last updated:** [April 28, 2016, 7:21pm UTC](https://discuss.elastic.co/t/ship-logs-from-appliances-in-form-of-syslog/48509 "2016-04-28T19:21:32Z")
Hi, I am running ELK stack on a Windows box. We got a couple of load balancers appliances and they have the option to export logs in syslog format. How can I configure my Elasticsearch or Logstash to receive them? D… --- ## [Logstash 2.3.1 input log problem](https://discuss.elastic.co/t/logstash-2-3-1-input-log-problem/48519)
**Author:** [@joewen](https://discuss.elastic.co/u/joewen)\ **Replies:** 1\ **Last updated:** [April 28, 2016, 7:20pm UTC](https://discuss.elastic.co/t/logstash-2-3-1-input-log-problem/48519 "2016-04-28T19:20:56Z")
I used beats input logstash,but i find it use stdin{} the Content encounter double quotes appear escape。 is it bug？ --- ## [Logstash - How to format xml tags to show in Kibana](https://discuss.elastic.co/t/logstash-how-to-format-xml-tags-to-show-in-kibana/48708)
**Author:** [@Robert1](https://discuss.elastic.co/u/Robert1)\ **Replies:** 0\ **Last updated:** [April 28, 2016, 4:33pm UTC](https://discuss.elastic.co/t/logstash-how-to-format-xml-tags-to-show-in-kibana/48708 "2016-04-28T16:33:25Z")
I have the follow xml logs : \ \ **Author:** [@Cipher](https://discuss.elastic.co/u/Cipher)\ **Replies:** 1\ **Last updated:** [April 28, 2016, 5:34pm UTC](https://discuss.elastic.co/t/logstash-log-error-unsupported-field-netflow-question/48649 "2016-04-28T17:34:28Z") Hi all Please go easy on me \[:slight\_smile:\] - I've just literally installed ELK on a CentOS7 VM in order to trial the Netflow support for Cisco devices (to start with, then moving on to what else this collection of p… --- ## [Geoip.location field latitude/longitude coordinates are reversed](https://discuss.elastic.co/t/geoip-location-field-latitude-longitude-coordinates-are-reversed/48715)
**Author:** [@Ryan\_Groten](https://discuss.elastic.co/u/Ryan_Groten)\ **Replies:** 0\ **Last updated:** [April 28, 2016, 5:23pm UTC](https://discuss.elastic.co/t/geoip-location-field-latitude-longitude-coordinates-are-reversed/48715 "2016-04-28T17:23:29Z")
my logstash.conf: geoip { source =\> "client\_ip" target =\> "geoip" } mutate { add\_field =\> \[ "mylocation", "%{\[geoip\]\[location\]}" \] } In the error below notice that g… --- ## [Issue with Logstash Kafka Input Missing Fields](https://discuss.elastic.co/t/issue-with-logstash-kafka-input-missing-fields/48596)
**Author:** [@landonix](https://discuss.elastic.co/u/landonix)\ **Replies:** 3\ **Last updated:** [April 28, 2016, 4:55pm UTC](https://discuss.elastic.co/t/issue-with-logstash-kafka-input-missing-fields/48596 "2016-04-28T16:55:05Z")
Logstash Kafka Consumer/Input Hello! I'm attempting use the kakfa input plugin and thought I would start with some basic testing first. From the looks of it, I'm losing three fields (app, pid, and msgid) when logstash … --- ## [Logstash S3 input plugin doesn't work](https://discuss.elastic.co/t/logstash-s3-input-plugin-doesnt-work/48298)
**Author:** [@tuxknight](https://discuss.elastic.co/u/tuxknight)\ **Replies:** 2\ **Last updated:** [April 28, 2016, 3:52pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-doesnt-work/48298 "2016-04-28T15:52:24Z")
This is what I got in logstash.log {:timestamp=\>"2016-04-25T17:26:13.863000+0800", :message=\>"A plugin had an unrecoverable error. Will restart this plugin.\\n Plugin: \"xxx-elb-log", acces… --- ## [Logstash crashes during index roll over](https://discuss.elastic.co/t/logstash-crashes-during-index-roll-over/48698)
**Author:** [@cvaddey](https://discuss.elastic.co/u/cvaddey)\ **Replies:** 0\ **Last updated:** [April 28, 2016, 3:11pm UTC](https://discuss.elastic.co/t/logstash-crashes-during-index-roll-over/48698 "2016-04-28T15:11:55Z")
Logstash crashes everynight around same time . It goes OOM . It happens even with minimum load. No errors noted in logstash. ES status shows it went RED at exact same time \[2016-04-27 20:00:01,309\]\[INFO \]\[cluster.met… --- ## [SOLVED: Logstash issue with shield - "action \[indices:data/write/bulk\] is unauthorized for user \[logstash\]"](https://discuss.elastic.co/t/solved-logstash-issue-with-shield-action-indices-data-write-bulk-is-unauthorized-for-user-logstash/48410)
**Author:** [@Clement\_Ros](https://discuss.elastic.co/u/Clement_Ros)\ **Replies:** 14\ **Last updated:** [April 28, 2016, 2:26pm UTC](https://discuss.elastic.co/t/solved-logstash-issue-with-shield-action-indices-data-write-bulk-is-unauthorized-for-user-logstash/48410 "2016-04-28T14:26:20Z")
Hi, I am about to install shield on my cluster and when I want to configure logstash to communicate with ES i have the following problem : I followed the documentation and when i did this configuration the authenticat… --- ## [Lumberjack input issues](https://discuss.elastic.co/t/lumberjack-input-issues/48678)
**Author:** [@YLombardi](https://discuss.elastic.co/u/YLombardi)\ **Replies:** 0\ **Last updated:** [April 28, 2016, 1:18pm UTC](https://discuss.elastic.co/t/lumberjack-input-issues/48678 "2016-04-28T13:18:52Z")
Hi. I have two servers with logstash-forwarder. First server send logs in a json format. Second server send logs in a plain format. On a third server, I have ELK. I configure Logstash like this : input { Get file… --- ## [How to exclude empty (0 KB) files in logstash](https://discuss.elastic.co/t/how-to-exclude-empty-0-kb-files-in-logstash/48648)
**Author:** [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\ **Replies:** 0\ **Last updated:** [April 28, 2016, 8:45am UTC](https://discuss.elastic.co/t/how-to-exclude-empty-0-kb-files-in-logstash/48648 "2016-04-28T08:45:01Z")
Frequently logstash crashed with the following error {:timestamp=\>"2016-04-28T12:49:47.540000+0530", :message=\>"A plugin had an unrecoverable error. Will restart this plugin.\\n Plugin: \[\\"filepath\\"\], start\_position=\>\\"… --- ## [2.1.3: Parse Error in Logs](https://discuss.elastic.co/t/2-1-3-parse-error-in-logs/48638)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [April 28, 2016, 7:42am UTC](https://discuss.elastic.co/t/2-1-3-parse-error-in-logs/48638 "2016-04-28T07:42:28Z")
Hi, After upgrading to 2.1.3 I'm seeing the following error reported in the logs on my test VM: {:timestamp=\>"2016-04-28T08:09:37.070000+0100", :message=\>"JSON parse failure. Falling back to plain-text", :error=\># at \[… --- ## [Encounter a problem when write two config file](https://discuss.elastic.co/t/encounter-a-problem-when-write-two-config-file/48619)
**Author:** [@gfafei](https://discuss.elastic.co/u/gfafei)\ **Replies:** 2\ **Last updated:** [April 28, 2016, 5:40am UTC](https://discuss.elastic.co/t/encounter-a-problem-when-write-two-config-file/48619 "2016-04-28T05:40:48Z")
In the directory "conf.d",I wrote two config file --f1 an f2.when I write data to f1's input and the f2's output recive the data too.How can I avoid this?I expect that the two config file have no effect on each other. --- ## [Help with Grok Parsing](https://discuss.elastic.co/t/help-with-grok-parsing/48582)
**Author:** [@RKEL](https://discuss.elastic.co/u/RKEL)\ **Replies:** 1\ **Last updated:** [April 28, 2016, 5:31am UTC](https://discuss.elastic.co/t/help-with-grok-parsing/48582 "2016-04-28T05:31:57Z")
hi, any help is appreciated. spent quite a bit of time learning and trying to parse below formats in the log file \[Tue Mar 8 00:00:00 2015\]instance remoteserver 0.207631 200 GET /rest/v1/api/xyz/abc12345678?role=LEARN&… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1159) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1161)