# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=879 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 880 --- ## [Why Logstash reload the same data many times?](https://discuss.elastic.co/t/why-logstash-reload-the-same-data-many-times/106833)
**Author:** [@Chemse](https://discuss.elastic.co/u/Chemse)\ **Replies:** 2\ **Last updated:** [November 8, 2017, 10:25am UTC](https://discuss.elastic.co/t/why-logstash-reload-the-same-data-many-times/106833 "2017-11-08T10:25:02Z")
Hello, I have an issue with Logstash, it loads the same data many times. this is my config, Logstash is running as a service on Windows. input { file { path =\>\["D:/exploit/tmp/nuitAppELK/annuel/C… --- ## [Dateadd feature in logstash](https://discuss.elastic.co/t/dateadd-feature-in-logstash/105392)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 1\ **Last updated:** [November 8, 2017, 9:37am UTC](https://discuss.elastic.co/t/dateadd-feature-in-logstash/105392 "2017-11-08T09:37:32Z")
Hi, We have some log files which is dynamic in pattern. We would like to filter the logdate field as, whenver the value is null, then that value should be equal to current timestamp less 6hrs. In sql, its the same as I… --- ## [Kibana info different when sending to ES directly or with logstash](https://discuss.elastic.co/t/kibana-info-different-when-sending-to-es-directly-or-with-logstash/106831)
**Author:** [@Timshs](https://discuss.elastic.co/u/Timshs)\ **Replies:** 0\ **Last updated:** [November 8, 2017, 9:28am UTC](https://discuss.elastic.co/t/kibana-info-different-when-sending-to-es-directly-or-with-logstash/106831 "2017-11-08T09:28:09Z")
Hi For some time I have use Serilog =\> ES =\> Kibana ... and all is good. But if I use Logstash like Serilog =\> Logstash =\> ES = Kibana I do not get the same output: ELASTICSEARCH: add key="serilog:using" value="Seril… --- ## [How to update the multiline log](https://discuss.elastic.co/t/how-to-update-the-multiline-log/106827)
**Author:** [@admanheart](https://discuss.elastic.co/u/admanheart)\ **Replies:** 0\ **Last updated:** [November 8, 2017, 9:23am UTC](https://discuss.elastic.co/t/how-to-update-the-multiline-log/106827 "2017-11-08T09:23:18Z")
hi guys： i have a log like this : 2017-11-08T16:40:40Z 192.168.1.1 logbody\\nlogbody\\nlogbody i want to filter this log by \\n, and each message field had a field name source\_ip:192.168.1.1 ，how can i do with the fil… --- ## [Logstash how to set field index type from json input](https://discuss.elastic.co/t/logstash-how-to-set-field-index-type-from-json-input/106653)
**Author:** [@imqishi](https://discuss.elastic.co/u/imqishi)\ **Replies:** 2\ **Last updated:** [November 8, 2017, 6:35am UTC](https://discuss.elastic.co/t/logstash-how-to-set-field-index-type-from-json-input/106653 "2017-11-08T06:35:21Z")
I want save data to elasticsearch by a json input, but when I created it I found that all my field type is string... I try to format my data and it can achieve number and string two types' indices like this: \[image\] B… --- ## [Logstash jdbc plugin not installed](https://discuss.elastic.co/t/logstash-jdbc-plugin-not-installed/106800)
**Author:** [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)\ **Replies:** 2\ **Last updated:** [November 8, 2017, 6:33am UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-not-installed/106800 "2017-11-08T06:33:59Z")
How can I install logstash jdbc plugin.... I enter the logstash directory like below: cd /etc/logstash and then there I write this command bin/logstash-plugin install logstash-input-jdbc as mention in elastic doc ur… --- ## [Timestamp difference calculation](https://discuss.elastic.co/t/timestamp-difference-calculation/105866)
**Author:** [@paulk4633](https://discuss.elastic.co/u/paulk4633)\ **Replies:** 5\ **Last updated:** [November 8, 2017, 6:21am UTC](https://discuss.elastic.co/t/timestamp-difference-calculation/105866 "2017-11-08T06:21:39Z")
Hello, I have a problem with the calculation of a time difference between two timestamps: "time1" =\> "31/10/2017:10:18:19.928 +0200", "time2" =\> "31/10/2017:10:19:00.190 +0200" As a result I want to have a field in Lo… --- ## [Kibana Map Visualization with Lat Long fields](https://discuss.elastic.co/t/kibana-map-visualization-with-lat-long-fields/106762)
**Author:** [@shashwat](https://discuss.elastic.co/u/shashwat)\ **Replies:** 3\ **Last updated:** [November 7, 2017, 11:47pm UTC](https://discuss.elastic.co/t/kibana-map-visualization-with-lat-long-fields/106762 "2017-11-07T23:47:40Z")
Hi, I have some csv files which contains the Latitude and Longitude column. I am trying to put these coordinates on Kibana map but not able to do so. Following is my logstash conf: input { file { path … --- ## [Trouble with json codec and/or date filter](https://discuss.elastic.co/t/trouble-with-json-codec-and-or-date-filter/106770)
**Author:** [@tcouto](https://discuss.elastic.co/u/tcouto)\ **Replies:** 0\ **Last updated:** [November 7, 2017, 11:36pm UTC](https://discuss.elastic.co/t/trouble-with-json-codec-and-or-date-filter/106770 "2017-11-07T23:36:28Z")
Hi, I'm using ELK for centralized logging and I'm having trouble with a log. I just changed this log from plain text to json format. After I added the JSON codec, I can no longer see logs in kibana. In the logstash debu… --- ## [Create Dynamic Index in Elasticsearch](https://discuss.elastic.co/t/create-dynamic-index-in-elasticsearch/106579)
**Author:** [@Ferdous\_Shibly](https://discuss.elastic.co/u/Ferdous_Shibly)\ **Replies:** 6\ **Last updated:** [November 7, 2017, 10:20pm UTC](https://discuss.elastic.co/t/create-dynamic-index-in-elasticsearch/106579 "2017-11-07T22:20:39Z")
I am trying to configure logstash to create dynamic indexes in Elasticsearch. Here are my configuration input{ kafka { bootstrap\_servers =\> \["192.168.56.2:9092,192.168.56.23:9092,192.168.56.24:9092"\] topics … --- ## [Combining multiple events](https://discuss.elastic.co/t/combining-multiple-events/106756)
**Author:** [@vgaas](https://discuss.elastic.co/u/vgaas)\ **Replies:** 0\ **Last updated:** [November 7, 2017, 9:04pm UTC](https://discuss.elastic.co/t/combining-multiple-events/106756 "2017-11-07T21:04:13Z")
We are using the http output plugin to store logs into a database. We would like to batch multiple changes from a file into 1 JSON request like so: { docs: \[ { "event": "event"1 }, … --- ## [TZInfo::AmbiguousTime exception when parsing JDBC column](https://discuss.elastic.co/t/tzinfo-ambiguoustime-exception-when-parsing-jdbc-column/106748)
**Author:** [@obodard](https://discuss.elastic.co/u/obodard)\ **Replies:** 0\ **Last updated:** [November 7, 2017, 7:42pm UTC](https://discuss.elastic.co/t/tzinfo-ambiguoustime-exception-when-parsing-jdbc-column/106748 "2017-11-07T19:42:45Z")
I am getting this exception when getting my data with the JDBC plugin: error: 26413962 Sequel::InvalidValue TZInfo::AmbiguousTime: 2017-11-05T01:30:00+00:00 is an ambiguous local time. This is probably because I am … --- ## [Logstash Ruby DateTime.parse error](https://discuss.elastic.co/t/logstash-ruby-datetime-parse-error/106251)
**Author:** [@ash007](https://discuss.elastic.co/u/ash007)\ **Replies:** 2\ **Last updated:** [November 7, 2017, 6:22pm UTC](https://discuss.elastic.co/t/logstash-ruby-datetime-parse-error/106251 "2017-11-07T18:22:22Z")
Hi i am trying to execute the below code. filter{ if \[type\] == 'date'{ ruby { init =\> "@regex= /\[^0-9a-zA-Z\]/" code =\> " require 'date' require 'time' event.set('var\_str', DateTim… --- ## [Logstash \[elasticsearch plugin not working\]](https://discuss.elastic.co/t/logstash-elasticsearch-plugin-not-working/106706)
**Author:** [@Dima\_Khodan](https://discuss.elastic.co/u/Dima_Khodan)\ **Replies:** 0\ **Last updated:** [November 7, 2017, 2:42pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-plugin-not-working/106706 "2017-11-07T14:42:50Z")
I have a config file with elasticsearch filter. After I perform query inside block "query\_template": { "query": { "bool": { "must": \[ { "exists": { "field":"cor… --- ## [How to process event multiple times](https://discuss.elastic.co/t/how-to-process-event-multiple-times/105404)
**Author:** [@Vladimir\_Kirichenko](https://discuss.elastic.co/u/Vladimir_Kirichenko)\ **Replies:** 2\ **Last updated:** [November 7, 2017, 2:00pm UTC](https://discuss.elastic.co/t/how-to-process-event-multiple-times/105404 "2017-11-07T14:00:37Z")
Summary: In our monitoring solution we have events that are reported from CI evenironment to monitoring server using http. Each event represents build event and contains some info about builds and, what is important, s… --- ## [Parsing failedRequest IIS](https://discuss.elastic.co/t/parsing-failedrequest-iis/106571)
**Author:** [@vinicius](https://discuss.elastic.co/u/vinicius)\ **Replies:** 1\ **Last updated:** [November 7, 2017, 1:01pm UTC](https://discuss.elastic.co/t/parsing-failedrequest-iis/106571 "2017-11-07T13:01:16Z")
Hello, Anyone can help me parsing element "Buffer" with failed Request of IIS? I identifed 2 elements with same name (Buffer). If i'm run xpath (/failedRequest/tns:Event\[26\]/tns:EventData//tns:Data\[@Name='Buffer'\]/text… --- ## [Indexes creation by date issue](https://discuss.elastic.co/t/indexes-creation-by-date-issue/92233)
**Author:** [@alonsosanchezd](https://discuss.elastic.co/u/alonsosanchezd)\ **Replies:** 7\ **Last updated:** [November 7, 2017, 11:02am UTC](https://discuss.elastic.co/t/indexes-creation-by-date-issue/92233 "2017-11-07T11:02:15Z")
Hi all! Firstly, let me thank you all for your help! I have an strange behaviour of filebeat/logstash and elasticsearch. Let me explain: I have to index log files of last month and I'm forcing logstash to replace proce… --- ## [Logstash.outputs.elasticsearch: Elasticsearch appears to be unreachable or down!](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-elasticsearch-appears-to-be-unreachable-or-down/106473)
**Author:** [@alexhuang](https://discuss.elastic.co/u/alexhuang)\ **Replies:** 2\ **Last updated:** [November 7, 2017, 10:54am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-elasticsearch-appears-to-be-unreachable-or-down/106473 "2017-11-07T10:54:22Z")
I'm using the latest version of ELK(logstash 5.6.0, elasticsearch 5.6.0), and sometimes i will get the following error message in logstash logfile, but the elasticsearch cluster is really heathly. here is the error mes… --- ## [Logstash service is not running](https://discuss.elastic.co/t/logstash-service-is-not-running/106499)
**Author:** [@enes](https://discuss.elastic.co/u/enes)\ **Replies:** 2\ **Last updated:** [November 7, 2017, 6:13am UTC](https://discuss.elastic.co/t/logstash-service-is-not-running/106499 "2017-11-07T06:13:30Z")
Hi, I started to setup ELK. Elastic Search is ok. The problem on the logstash. when I started Logstash service got this error. \[image\] In the event log, I see this error details. \[error2\] The directory is valid on … --- ## [Logstash CSV plugin behaviour for realtime CSV files](https://discuss.elastic.co/t/logstash-csv-plugin-behaviour-for-realtime-csv-files/106396)
**Author:** [@shashwat](https://discuss.elastic.co/u/shashwat)\ **Replies:** 2\ **Last updated:** [November 6, 2017, 9:59pm UTC](https://discuss.elastic.co/t/logstash-csv-plugin-behaviour-for-realtime-csv-files/106396 "2017-11-06T21:59:55Z")
Hi , I am facing some issue with logstash (5.6.2) CSV plugin's behaviour. Following is my scenarion : I am receving .csv file every minute. I send this file to logstash. Intially around 10 mins everyhing works fine but… --- ## [Can we omit filter in this case?](https://discuss.elastic.co/t/can-we-omit-filter-in-this-case/106432)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 5\ **Last updated:** [November 6, 2017, 9:34pm UTC](https://discuss.elastic.co/t/can-we-omit-filter-in-this-case/106432 "2017-11-06T21:34:20Z")
Hi Guys, If I am accepting input over beats can I completely omit filter? I mean just input { beats port 5044} and then output { elasticsearch { hosts =\> \[192.168.1.12"9200"\] } } --- ## [How to extract rows with logstash](https://discuss.elastic.co/t/how-to-extract-rows-with-logstash/106236)
**Author:** [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\ **Replies:** 11\ **Last updated:** [November 6, 2017, 6:06pm UTC](https://discuss.elastic.co/t/how-to-extract-rows-with-logstash/106236 "2017-11-06T18:06:09Z")
hi i have this log file 2017-06-08 12:10:12,924 | INFO | nager-plugin-out | ExtraDataServices | 302 - 2017-06-08 12:10:12,925 | DEBUG | nager-plugin-out | deleteByExample | 285 - 2017-06-08 12:10:12,925 | DEBU… --- ## [Logstash clone filter overwriting user data](https://discuss.elastic.co/t/logstash-clone-filter-overwriting-user-data/105863)
**Author:** [@Dan\_Markhasin](https://discuss.elastic.co/u/Dan_Markhasin)\ **Replies:** 1\ **Last updated:** [November 6, 2017, 4:45pm UTC](https://discuss.elastic.co/t/logstash-clone-filter-overwriting-user-data/105863 "2017-11-06T16:45:51Z")
There is (IMO) a critical bug in the clone filter: Since it is hard-coded to do: clone.set("type", type) It will overwrite whatever was in the original "type" field in the document, causing data corruption/loss. I… --- ## [Filter for message field with specific pattern](https://discuss.elastic.co/t/filter-for-message-field-with-specific-pattern/106543)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 2\ **Last updated:** [November 6, 2017, 3:45pm UTC](https://discuss.elastic.co/t/filter-for-message-field-with-specific-pattern/106543 "2017-11-06T15:45:02Z")
I am getting data to Logstash from STOMP input, where my message prints as: "message" =\> " \< Name\>\\n \\\n \ **Author:** [@mPanasiewicz](https://discuss.elastic.co/u/mPanasiewicz)\ **Replies:** 4\ **Last updated:** [November 6, 2017, 3:34pm UTC](https://discuss.elastic.co/t/parse-array-of-json-objects-in-json/106518 "2017-11-06T15:34:15Z") Hello, I've got sth like: \[ {"aaa": "bbb"}, {"ccc": "ddd"}, {"eee": "fff"} \] I've tried split input plugin but I think it would not work becouse I need parent key of the array. In this case there is no key because ar… --- ## [Ruby plugin and timezone issue](https://discuss.elastic.co/t/ruby-plugin-and-timezone-issue/106138)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 1\ **Last updated:** [November 6, 2017, 3:07pm UTC](https://discuss.elastic.co/t/ruby-plugin-and-timezone-issue/106138 "2017-11-06T15:07:25Z")
Hi all, I had to calculate a new date field: given a timestamp and a offset I need to set a new field with the value timestamp+offset I solved the problem but I'm sure that there is a better way to do it. I use ruby … --- ## [Logstash starting queries](https://discuss.elastic.co/t/logstash-starting-queries/106532)
**Author:** [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\ **Replies:** 5\ **Last updated:** [November 6, 2017, 3:00pm UTC](https://discuss.elastic.co/t/logstash-starting-queries/106532 "2017-11-06T15:00:43Z")
Hi All, I have been using ELastic stack for parsing syslog and store it in elasticsearch, but i have problem in starting the service the logstash, When i start the logstash like , service logstash start , it crashes bu… --- ## [Logstash.agent - Error in reactor loop escaped: Bad file descriptor](https://discuss.elastic.co/t/logstash-agent-error-in-reactor-loop-escaped-bad-file-descriptor/106519)
**Author:** [@stefws](https://discuss.elastic.co/u/stefws)\ **Replies:** 1\ **Last updated:** [November 6, 2017, 1:48pm UTC](https://discuss.elastic.co/t/logstash-agent-error-in-reactor-loop-escaped-bad-file-descriptor/106519 "2017-11-06T13:48:50Z")
I've upgraded to 5.6.3 from 2.1.3 patched my config for breaking changes: # /usr/share/logstash/bin/logstash -t --path.config /etc/logstash/conf.d --path.settings /etc/logstash Sending Logstash's logs to /var/log/logst… --- ## [Logstash after upgrade to 5.6.3 from 2.1.3, ES output module: continues to do health checking](https://discuss.elastic.co/t/logstash-after-upgrade-to-5-6-3-from-2-1-3-es-output-module-continues-to-do-health-checking/106484)
**Author:** [@stefws](https://discuss.elastic.co/u/stefws)\ **Replies:** 1\ **Last updated:** [November 6, 2017, 1:20pm UTC](https://discuss.elastic.co/t/logstash-after-upgrade-to-5-6-3-from-2-1-3-es-output-module-continues-to-do-health-checking/106484 "2017-11-06T13:20:15Z")
wondering why logstash after been upgraded to 5.6.3 from 2.1.3 seems to continuously every few seconds to do this: \[2017-11-06T09:20:07,880\]\[INFO \]\[logstash.outputs.elasticsearch\] Elasticsearch pool URLs updated {:chan… --- ## [Logstash/rsyslog/json](https://discuss.elastic.co/t/logstash-rsyslog-json/106009)
**Author:** [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\ **Replies:** 7\ **Last updated:** [November 6, 2017, 1:17pm UTC](https://discuss.elastic.co/t/logstash-rsyslog-json/106009 "2017-11-06T13:17:30Z")
Hi All, I was trying to setup rsyslog in elasticstack ,so I was trying to follow this article, my logstash conf looks exactly similar to the article. My logstash conf ,,, input { udp { host =\> "private ip" … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=878) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=880)