# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=990 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 991 --- ## [How to filter out internal IPs and localhost.localdomain for geoip?](https://discuss.elastic.co/t/how-to-filter-out-internal-ips-and-localhost-localdomain-for-geoip/84184)
**Author:** [@dfinn](https://discuss.elastic.co/u/dfinn)\ **Replies:** 9\ **Last updated:** [May 3, 2017, 5:29pm UTC](https://discuss.elastic.co/t/how-to-filter-out-internal-ips-and-localhost-localdomain-for-geoip/84184 "2017-05-03T17:29:35Z")
My logs are full of errors like so: \[2017-05-01T15:11:18,901\]\[ERROR\]\[logstash.filters.geoip \] IP Field contained invalid IP address or hostname {:exception=\>java.net.UnknownHostException: localhost.localdomain: Name … --- ## [How to push two inputs data as a single event(docment) in logstash.ym](https://discuss.elastic.co/t/how-to-push-two-inputs-data-as-a-single-event-docment-in-logstash-ym/84274)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 3\ **Last updated:** [May 3, 2017, 4:27pm UTC](https://discuss.elastic.co/t/how-to-push-two-inputs-data-as-a-single-event-docment-in-logstash-ym/84274 "2017-05-03T16:27:23Z")
Hi...all How can i push two inputs data as a single event(docment) in logstash.yml for example input{ file{ type =\> "dummylog" path =\> \[/var/log/messages\] } file{ type =\> "log" … --- ## [Logstash - wrong codec in tcp input plugin?](https://discuss.elastic.co/t/logstash-wrong-codec-in-tcp-input-plugin/83364)
**Author:** [@leaner](https://discuss.elastic.co/u/leaner)\ **Replies:** 2\ **Last updated:** [May 3, 2017, 3:46pm UTC](https://discuss.elastic.co/t/logstash-wrong-codec-in-tcp-input-plugin/83364 "2017-05-03T15:46:24Z")
I'm using the newest version of Logstash 5.3.1. While I am using a tcp input plugins, it shows that the real codec being used is not the same one with my config file. To confirm this question, I add a log output in th… --- ## [What to the stats in pipeline.events represent?](https://discuss.elastic.co/t/what-to-the-stats-in-pipeline-events-represent/84435)
**Author:** [@rwood](https://discuss.elastic.co/u/rwood)\ **Replies:** 0\ **Last updated:** [May 3, 2017, 3:48pm UTC](https://discuss.elastic.co/t/what-to-the-stats-in-pipeline-events-represent/84435 "2017-05-03T15:48:15Z")
Hi, The output from /\_node/stats/pipeline has a section for events, and sections for each plugin. What do the stats in the events section represent? pipeline.events.duration\_in\_millis doesn't equal the sum of duration\_in… --- ## [Nothing view in elasticsearch and kibana when import csv file](https://discuss.elastic.co/t/nothing-view-in-elasticsearch-and-kibana-when-import-csv-file/83822)
**Author:** [@zegdene](https://discuss.elastic.co/u/zegdene)\ **Replies:** 24\ **Last updated:** [May 3, 2017, 3:25pm UTC](https://discuss.elastic.co/t/nothing-view-in-elasticsearch-and-kibana-when-import-csv-file/83822 "2017-05-03T15:25:49Z")
Hi, I create a file.config for import csv file to elasticsearch but the connection between ES and Logstash is warning . I have this Warn: \> Sending Logstash's logs to C:/Project/elk/logstash/logs which is now configur… --- ## [Logstash: Accessing the environment variable from rule config file through "dynamic key" reference](https://discuss.elastic.co/t/logstash-accessing-the-environment-variable-from-rule-config-file-through-dynamic-key-reference/84350)
**Author:** [@rkatyalin](https://discuss.elastic.co/u/rkatyalin)\ **Replies:** 4\ **Last updated:** [May 3, 2017, 3:22pm UTC](https://discuss.elastic.co/t/logstash-accessing-the-environment-variable-from-rule-config-file-through-dynamic-key-reference/84350 "2017-05-03T15:22:31Z")
We have an requirement to read environment variable from rule config file through dynamic key reference instead of static key reference. However, It seems Logstash does not support reading Environment variable through dy… --- ## [Logstash Kafka Avro Codec Error with Doubles](https://discuss.elastic.co/t/logstash-kafka-avro-codec-error-with-doubles/79483)
**Author:** [@sjzzzzz](https://discuss.elastic.co/u/sjzzzzz)\ **Replies:** 1\ **Last updated:** [May 3, 2017, 2:43pm UTC](https://discuss.elastic.co/t/logstash-kafka-avro-codec-error-with-doubles/79483 "2017-05-03T14:43:55Z")
Hi, we hit a snag with the logstash kafka avro codec. We are unable to transmit/receive doubles across a topic with this codec. Included is a test case along with data to reproduce the problem. We're using Logstash 5.… --- ## [Logstash/Grok help](https://discuss.elastic.co/t/logstash-grok-help/83704)
**Author:** [@mduckett1](https://discuss.elastic.co/u/mduckett1)\ **Replies:** 4\ **Last updated:** [May 3, 2017, 2:32pm UTC](https://discuss.elastic.co/t/logstash-grok-help/83704 "2017-05-03T14:32:03Z")
Hi, I am trying to pull in some logs using filebeat and having some trouble (mainly because of lack of understanding I think). The log file has multiple lines like this: 2017-04-09 12:53:10 INFO: InsertedAt=2017-04… --- ## [Logstash getstarted](https://discuss.elastic.co/t/logstash-getstarted/84264)
**Author:** [@Chemse](https://discuss.elastic.co/u/Chemse)\ **Replies:** 9\ **Last updated:** [May 3, 2017, 1:59pm UTC](https://discuss.elastic.co/t/logstash-getstarted/84264 "2017-05-03T13:59:48Z")
Hello !! I just install Logstach succesfully, and I want to test it with this command bin/logstash -e 'input { stdin { } } output { stdout {} }' My problem is I don't know where can I find this folder bin/logs… --- ## [Logstash stops compiling when given custom pattern inside a filter](https://discuss.elastic.co/t/logstash-stops-compiling-when-given-custom-pattern-inside-a-filter/84412)
**Author:** [@Darya\_Semenova](https://discuss.elastic.co/u/Darya_Semenova)\ **Replies:** 2\ **Last updated:** [May 3, 2017, 1:42pm UTC](https://discuss.elastic.co/t/logstash-stops-compiling-when-given-custom-pattern-inside-a-filter/84412 "2017-05-03T13:42:16Z")
So, the problem is this: I have a custom pattern file in ./patterns directory. It looks like this: NODELISTENUM((\[A-Za-z0-9\]{0,20})(\\-)?(\[A-Za-z0-9\]{0,20})(\\.\[A-Za-z0-9\]{0,20})?(\\,)\*(\[A-Za-z0-9\]{0,20}(\\-?\[A-Za-z0-9\]{… --- ## [Parsing IIS log Issue](https://discuss.elastic.co/t/parsing-iis-log-issue/84383)
**Author:** [@Michael\_TRAN](https://discuss.elastic.co/u/Michael_TRAN)\ **Replies:** 1\ **Last updated:** [May 3, 2017, 1:28pm UTC](https://discuss.elastic.co/t/parsing-iis-log-issue/84383 "2017-05-03T13:28:21Z")
Hello, I try to parse my IIS log but the problems is that my kibana doesn't recognize the index pattern (logstash\_iis-\*). Here the IIS logstash configuration file (/etc/logstash/conf.d/logstash\_iis.conf) input { fil… --- ## [Epoch time conversion is showing wrong time](https://discuss.elastic.co/t/epoch-time-conversion-is-showing-wrong-time/84409)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 2\ **Last updated:** [May 3, 2017, 12:28pm UTC](https://discuss.elastic.co/t/epoch-time-conversion-is-showing-wrong-time/84409 "2017-05-03T12:28:58Z")
Hello everyone here i am trying to convert epoch time into human readable format where it is parsing good but indicating wrong point of time in target field i didn't get any error also here is my code: input { exe… --- ## [Date parse error](https://discuss.elastic.co/t/date-parse-error/84269)
**Author:** [@Djelouah\_Laala](https://discuss.elastic.co/u/Djelouah_Laala)\ **Replies:** 12\ **Last updated:** [May 3, 2017, 12:18pm UTC](https://discuss.elastic.co/t/date-parse-error/84269 "2017-05-03T12:18:50Z")
Hi guys, i'm trying to parse a date like this one : 2017-04-29 00:00:00 +0200 I'm using this date filter : filter { date { match =\> \[ "txechecczhier\_date", "yyyy-MM-dd HH:mm:ss Z" \] } } I'm getting a result … --- ## [Can't implement custom pattern inside a filter](https://discuss.elastic.co/t/cant-implement-custom-pattern-inside-a-filter/84402)
**Author:** [@Darya\_Semenova](https://discuss.elastic.co/u/Darya_Semenova)\ **Replies:** 4\ **Last updated:** [May 3, 2017, 11:39am UTC](https://discuss.elastic.co/t/cant-implement-custom-pattern-inside-a-filter/84402 "2017-05-03T11:39:26Z")
Hello, For some reason in my case logstash keeps not compling when I try to use a custom pattern. So I tried to implement a needed pattern as a field (the easier one now for testing, just a letter with a '-' in front of … --- ## [Logstash array length check](https://discuss.elastic.co/t/logstash-array-length-check/39681)
**Author:** [@bubu.senapati](https://discuss.elastic.co/u/bubu.senapati)\ **Replies:** 14\ **Last updated:** [May 3, 2017, 10:22am UTC](https://discuss.elastic.co/t/logstash-array-length-check/39681 "2017-05-03T10:22:55Z")
Hi , How can i check the length of an array in logstash. I have a request that is request=/acoounts/debenhams/summary. I had split it by "/". I have to check the request.length i.e. the array size of the splitted array r… --- ## [Convert time in a lot off fields in Logstash](https://discuss.elastic.co/t/convert-time-in-a-lot-off-fields-in-logstash/84288)
**Author:** [@paracetamol](https://discuss.elastic.co/u/paracetamol)\ **Replies:** 2\ **Last updated:** [May 3, 2017, 9:03am UTC](https://discuss.elastic.co/t/convert-time-in-a-lot-off-fields-in-logstash/84288 "2017-05-03T09:03:12Z")
Hi, My input is a json with tis values: windMaxTime: "10:10" dewpointMaxTime: "10:09" date { match =\> \["stats\[sinceMidnight\]\[windMaxTime\]","HH:mm"\] target =\> "stats\[sinceMidnight\]\[windMaxTime\]" timezone =\> "UTC" } d… --- ## [Gzip error when reading file outut](https://discuss.elastic.co/t/gzip-error-when-reading-file-outut/84363)
**Author:** [@Mark\_van\_Rossum](https://discuss.elastic.co/u/Mark_van_Rossum)\ **Replies:** 0\ **Last updated:** [May 3, 2017, 7:36am UTC](https://discuss.elastic.co/t/gzip-error-when-reading-file-outut/84363 "2017-05-03T07:36:42Z")
I have a file output of: file { path =\> "/logs/%{+YYYY}/%{+MM}/%{+dd}/%{backend}" codec =\> line { format =\> "%{message}"} gzip =\> true } That writes logs to a gzipped file. However, when I attempt to re… --- ## [Logstash XML Parser](https://discuss.elastic.co/t/logstash-xml-parser/84135)
**Author:** [@Pranil\_Ambule](https://discuss.elastic.co/u/Pranil_Ambule)\ **Replies:** 2\ **Last updated:** [May 3, 2017, 5:58am UTC](https://discuss.elastic.co/t/logstash-xml-parser/84135 "2017-05-03T05:58:35Z")
I'm parsing a lot of xml files using xml filter. But for some files I'm getting errors like "Can't merge a non object mapping \[doc.Body.EnumerateResponse.Items.DCIM\_BaseMetricValue.Duration\] with an object mapping"\` a… --- ## [Split xml xpath](https://discuss.elastic.co/t/split-xml-xpath/84336)
**Author:** [@daian.martinho](https://discuss.elastic.co/u/daian.martinho)\ **Replies:** 1\ **Last updated:** [May 3, 2017, 5:40am UTC](https://discuss.elastic.co/t/split-xml-xpath/84336 "2017-05-03T05:40:29Z")
Hi, when parsing 4 xml files, sometimes the first and sometimes the last one has parse error. (cant split nillclass) . Am I doing something wrong? input { beats { port =\> "5044" } } filter { ## i… --- ## [Guide for Generating valid Certs for Logstash / Filebeat SSL Setup](https://discuss.elastic.co/t/guide-for-generating-valid-certs-for-logstash-filebeat-ssl-setup/82965)
**Author:** [@ninjada](https://discuss.elastic.co/u/ninjada)\ **Replies:** 3\ **Last updated:** [May 3, 2017, 5:13am UTC](https://discuss.elastic.co/t/guide-for-generating-valid-certs-for-logstash-filebeat-ssl-setup/82965 "2017-05-03T05:13:03Z")
hi guys, this stumps me every time i setup an ELK stack for some reason.. I had it sussed out last time i was deploying multiple ELK stacks on version 2.x have just finished testing 5.x and am readying to get everyth… --- ## [How to implement listener for log file and database](https://discuss.elastic.co/t/how-to-implement-listener-for-log-file-and-database/84152)
**Author:** [@sonal410](https://discuss.elastic.co/u/sonal410)\ **Replies:** 1\ **Last updated:** [May 3, 2017, 5:07am UTC](https://discuss.elastic.co/t/how-to-implement-listener-for-log-file-and-database/84152 "2017-05-03T05:07:48Z")
I have to update elasticsearch with each new entry in log file and each new record inserted in rdbms database, is their any way for that. I was able to migrate data once but now sure how to do it for any further change i… --- ## [Logstash in docker start failed](https://discuss.elastic.co/t/logstash-in-docker-start-failed/84347)
**Author:** [@xiaotie](https://discuss.elastic.co/u/xiaotie)\ **Replies:** 0\ **Last updated:** [May 3, 2017, 4:24am UTC](https://discuss.elastic.co/t/logstash-in-docker-start-failed/84347 "2017-05-03T04:24:50Z")
Hello, everyone I use logstash with custom docker image following this doc: https://www.elastic.co/guide/en/logstash/current/docker.html. I only change content of pipeline config as follow: input { redi… --- ## [Number of Kafka topics for ELK configuration](https://discuss.elastic.co/t/number-of-kafka-topics-for-elk-configuration/84335)
**Author:** [@John\_06](https://discuss.elastic.co/u/John_06)\ **Replies:** 0\ **Last updated:** [May 3, 2017, 12:25am UTC](https://discuss.elastic.co/t/number-of-kafka-topics-for-elk-configuration/84335 "2017-05-03T00:25:24Z")
Hi, I am using Filebeats, Kafka, Logstash and ES in my configuration. And I am going to add Metricbeats as well. Would it be a good idea to create a separate topic for Metricbeats in Kafka before sending data to the sam… --- ## [Why is my conf not running](https://discuss.elastic.co/t/why-is-my-conf-not-running/84309)
**Author:** [@Paul\_Spedding](https://discuss.elastic.co/u/Paul_Spedding)\ **Replies:** 1\ **Last updated:** [May 2, 2017, 10:58pm UTC](https://discuss.elastic.co/t/why-is-my-conf-not-running/84309 "2017-05-02T22:58:51Z")
Hi, I'm using this line to input data /etc/logstash/conf.d# /usr/share/logstash/bin/logstash -f mySql.conf --deb --debug-config Now here is my config file: simple-out.conf input { jdbc { # Postgres jdbc c… --- ## [Unable to install the logstash-filter-truncate](https://discuss.elastic.co/t/unable-to-install-the-logstash-filter-truncate/84155)
**Author:** [@ktomu](https://discuss.elastic.co/u/ktomu)\ **Replies:** 9\ **Last updated:** [May 2, 2017, 8:49pm UTC](https://discuss.elastic.co/t/unable-to-install-the-logstash-filter-truncate/84155 "2017-05-02T20:49:58Z")
Hello all when I try to install the filter I get the message: bin/logstash-plugin install --no-verify file:///tmp/logstash-filter-truncate-master.zip Installing file: /tmp/logstash-filter-truncate-master.zip ERROR: Inv… --- ## [Problem with sincedb](https://discuss.elastic.co/t/problem-with-sincedb/83420)
**Author:** [@ktomu](https://discuss.elastic.co/u/ktomu)\ **Replies:** 4\ **Last updated:** [May 2, 2017, 7:11pm UTC](https://discuss.elastic.co/t/problem-with-sincedb/83420 "2017-05-02T19:11:47Z")
Hello, I'm using the file input with Logstash - just plane text files (pipeline processing) output to elasticsearch and without any filters. Sometimes I have an issue, when some files have processed but there are no ne… --- ## [ES output filter hangs at health check](https://discuss.elastic.co/t/es-output-filter-hangs-at-health-check/84310)
**Author:** [@addyk](https://discuss.elastic.co/u/addyk)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 6:15pm UTC](https://discuss.elastic.co/t/es-output-filter-hangs-at-health-check/84310 "2017-05-02T18:15:45Z")
I posted a similar post at about a month ago but was not able to get any answers. Please read that link before reading the rest of this post. The short version of my issue is ES output filter hangs at health check … --- ## [Very slow movement of data](https://discuss.elastic.co/t/very-slow-movement-of-data/84278)
**Author:** [@jagan](https://discuss.elastic.co/u/jagan)\ **Replies:** 1\ **Last updated:** [May 2, 2017, 6:07pm UTC](https://discuss.elastic.co/t/very-slow-movement-of-data/84278 "2017-05-02T18:07:30Z")
Hi All, Recently i had upgraded all(ELK) versions to 5.3.2. Everything is working fine but today when I am trying to rebuild a new index it got terminated due to long running time. So after that the data from Filebeat … --- ## [Elasticsearch Input Plugin on Timer](https://discuss.elastic.co/t/elasticsearch-input-plugin-on-timer/84145)
**Author:** [@c.pentasuglia](https://discuss.elastic.co/u/c.pentasuglia)\ **Replies:** 3\ **Last updated:** [May 2, 2017, 3:51pm UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-on-timer/84145 "2017-05-02T15:51:07Z")
Is it possible to have the Elasticsearch input plugin fire on a timer? Ex. I would like it to fire once every hour. Or would I have to orchestrate this somehow outside of Logstash? Thanks! --- ## [Logstash process as startup process](https://discuss.elastic.co/t/logstash-process-as-startup-process/84205)
**Author:** [@Kishore](https://discuss.elastic.co/u/Kishore)\ **Replies:** 7\ **Last updated:** [May 2, 2017, 2:30pm UTC](https://discuss.elastic.co/t/logstash-process-as-startup-process/84205 "2017-05-02T14:30:42Z")
We are using logstash 2.1.1 version and running 5 different logstash process on RHEL, my concern is that whenever the servers goes down these process are getting stopped and these process are require to start manually wh… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=989) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=991)