# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1040 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1041 --- ## [Logstash 5.2 not working with Kafka 9 and AVRO](https://discuss.elastic.co/t/logstash-5-2-not-working-with-kafka-9-and-avro/73810)
**Author:** [@Zeeshan\_Alam](https://discuss.elastic.co/u/Zeeshan_Alam)\ **Replies:** 1\ **Last updated:** [February 8, 2017, 9:45am UTC](https://discuss.elastic.co/t/logstash-5-2-not-working-with-kafka-9-and-avro/73810 "2017-02-08T09:45:34Z")
Logstash 5.2 not working with Kafka 9 and AVRO I have installed Logstash 5.2.0 with logstash-input-kafka 4.1.1 and logstash-codec-avro 3.0.0 and trying to read data from Cloudera Kafka 9 but I am getting the gollowing … --- ## [Logstash json\_parse\_exception](https://discuss.elastic.co/t/logstash-json-parse-exception/74338)
**Author:** [@swumao](https://discuss.elastic.co/u/swumao)\ **Replies:** 2\ **Last updated:** [February 8, 2017, 9:44am UTC](https://discuss.elastic.co/t/logstash-json-parse-exception/74338 "2017-02-08T09:44:21Z")
"error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse \[offset\]", "caused\_by"=\>{"type"=\>"json\_parse\_exception", "reason"=\>"Numeric value (2599608067) out of range of int\\n at \[Source: org.elasticsearc… --- ## [Missing logstash-output-kinesis for Logstash 5.2?](https://discuss.elastic.co/t/missing-logstash-output-kinesis-for-logstash-5-2/73492)
**Author:** [@NirReuveny](https://discuss.elastic.co/u/NirReuveny)\ **Replies:** 4\ **Last updated:** [February 8, 2017, 9:20am UTC](https://discuss.elastic.co/t/missing-logstash-output-kinesis-for-logstash-5-2/73492 "2017-02-08T09:20:24Z")
There is no mention of such output plugin in the list of output plugins in the documentation... and when trying to install it I can the error below... Can't seem to find too much info online on this, only on the input p… --- ## [Flatten json array in Logstash](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399)
**Author:** [@rmoff](https://discuss.elastic.co/u/rmoff)\ **Replies:** 7\ **Last updated:** [February 8, 2017, 8:52am UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399 "2017-02-08T08:52:17Z")
Is there a way to flatten a json array in Logstash? Use-case is twitter input plugin, wanting to make array data usable in Kibana, c.f. https://discuss.elastic.co/t/show-images-from-urls-in-data/337/4?u=rmoff thanks. --- ## [Syslogs Data to LogStash](https://discuss.elastic.co/t/syslogs-data-to-logstash/74320)
**Author:** [@prakash1243](https://discuss.elastic.co/u/prakash1243)\ **Replies:** 4\ **Last updated:** [February 8, 2017, 7:26am UTC](https://discuss.elastic.co/t/syslogs-data-to-logstash/74320 "2017-02-08T07:26:26Z")
Hi- Am pretty new to the Logstash, I wanted to run the syslogs and publish the data to the Logstash. I've installed the required syslogs plugins on the Logstash. I would like to know, which syslogs application needs to… --- ## ['less/greater than or equal to' forgotten?](https://discuss.elastic.co/t/less-greater-than-or-equal-to-forgotten/74298)
**Author:** [@wols](https://discuss.elastic.co/u/wols)\ **Replies:** 2\ **Last updated:** [February 8, 2017, 6:48am UTC](https://discuss.elastic.co/t/less-greater-than-or-equal-to-forgotten/74298 "2017-02-08T06:48:32Z")
A first issue here: https://github.com/elastic/logstash/issues/6652 After update to logstash-5.1.2 now the question here. Config File: filter { if \[score\] \>= 10 { mutate { ... } } } Sample Data: {"@timestamp":… --- ## [Logstash keeps nonstop indexing data from redis to elasticsearch](https://discuss.elastic.co/t/logstash-keeps-nonstop-indexing-data-from-redis-to-elasticsearch/74213)
**Author:** [@daaakeee](https://discuss.elastic.co/u/daaakeee)\ **Replies:** 1\ **Last updated:** [February 8, 2017, 4:59am UTC](https://discuss.elastic.co/t/logstash-keeps-nonstop-indexing-data-from-redis-to-elasticsearch/74213 "2017-02-08T04:59:31Z")
Hi. I am having troubles at indexing logs from Redis, which takes data from log file as an input.(i.e. trying to use shipper to Redis). shipper.conf input{ file{ type =\> "filelog" path =\> \["directory/to/lo… --- ## [What all files need to be present in Logstash settings folder if custom settings path is specified in startup command line](https://discuss.elastic.co/t/what-all-files-need-to-be-present-in-logstash-settings-folder-if-custom-settings-path-is-specified-in-startup-command-line/74221)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 0\ **Last updated:** [February 7, 2017, 12:39pm UTC](https://discuss.elastic.co/t/what-all-files-need-to-be-present-in-logstash-settings-folder-if-custom-settings-path-is-specified-in-startup-command-line/74221 "2017-02-07T12:39:51Z")
Hi, I am using Logstasg GA 5.0.0. I would like to start 2 Logstash instances with different API http ports in a single machine, so I decided to make a duplicate config folder. Now I have the original config folder and c… --- ## [Can you import geoip coordinates from outside? Mine are getting reversed](https://discuss.elastic.co/t/can-you-import-geoip-coordinates-from-outside-mine-are-getting-reversed/74312)
**Author:** [@dannypatel](https://discuss.elastic.co/u/dannypatel)\ **Replies:** 3\ **Last updated:** [February 8, 2017, 3:27am UTC](https://discuss.elastic.co/t/can-you-import-geoip-coordinates-from-outside-mine-are-getting-reversed/74312 "2017-02-08T03:27:08Z")
Hello, I'm using ElasticSearch 5.0.1 and logstash 5.0.1. I'm ingesting web traffic logs which also already contain the longitude and latitude coordinates. I would like to use them to create a geoip geohash. Here is my … --- ## [Logstash email hyperlink length limit](https://discuss.elastic.co/t/logstash-email-hyperlink-length-limit/74271)
**Author:** [@bo9354](https://discuss.elastic.co/u/bo9354)\ **Replies:** 9\ **Last updated:** [February 7, 2017, 11:18pm UTC](https://discuss.elastic.co/t/logstash-email-hyperlink-length-limit/74271 "2017-02-07T23:18:30Z")
The email plug is working fine. I would like to add a hyperlink to Kibana to allow the user to display a Visualization. All of that works, except the hyperlink is truncated in the Outlook email. I have investigated diffe… --- ## [Starting Logstash without any configuration](https://discuss.elastic.co/t/starting-logstash-without-any-configuration/74159)
**Author:** [@ronakg](https://discuss.elastic.co/u/ronakg)\ **Replies:** 2\ **Last updated:** [February 7, 2017, 10:27pm UTC](https://discuss.elastic.co/t/starting-logstash-without-any-configuration/74159 "2017-02-07T22:27:40Z")
Hi, I'd like to know if there's a way to start logstash without any configuration. We're deploying logstash in kubernetes pod. When the pod comes up, the configuration is not ready yet. We're starting logstash with aut… --- ## [JMX logs filter](https://discuss.elastic.co/t/jmx-logs-filter/74292)
**Author:** [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Replies:** 0\ **Last updated:** [February 7, 2017, 10:11pm UTC](https://discuss.elastic.co/t/jmx-logs-filter/74292 "2017-02-07T22:11:03Z")
E, \[2017-02-07T21:26:34.425421\] ERROR -- : The collection MetricData must not have a size greater than 20. (Aws::CloudWatch::Errors::InvalidParameterValue) /usr/local/share/ruby/gems/2.0/gems/aws-sdk-core-2.6.49/lib/seah… --- ## [Logstash 5.1.2 connecting to elasticsearch in cloud apparantly has no template](https://discuss.elastic.co/t/logstash-5-1-2-connecting-to-elasticsearch-in-cloud-apparantly-has-no-template/74132)
**Author:** [@walbag](https://discuss.elastic.co/u/walbag)\ **Replies:** 3\ **Last updated:** [February 7, 2017, 9:27pm UTC](https://discuss.elastic.co/t/logstash-5-1-2-connecting-to-elasticsearch-in-cloud-apparantly-has-no-template/74132 "2017-02-07T21:27:43Z")
When logstash initializes it shows no mapping template in the logs. When it goes to send data there are "can't convert nil to Array" errors. Note that this is using stunnel for https to the elastic cloud node. Here is … --- ## [JBoss EAP 6.4 and Logstash using log4j](https://discuss.elastic.co/t/jboss-eap-6-4-and-logstash-using-log4j/74286)
**Author:** [@skiss](https://discuss.elastic.co/u/skiss)\ **Replies:** 0\ **Last updated:** [February 7, 2017, 8:37pm UTC](https://discuss.elastic.co/t/jboss-eap-6-4-and-logstash-using-log4j/74286 "2017-02-07T20:37:54Z")
I am trying to consolidate my Jboss logs into ElasticSearch via Logstash (log4j). So far: - I've built and installed logstash-input-log4j plugin (https://www.elastic.co/guide/en/logstash/current/plugins-inputs-log4j.h… --- ## [Logstash Date Filter Silently Failing, Never Passes Events to ES](https://discuss.elastic.co/t/logstash-date-filter-silently-failing-never-passes-events-to-es/73905)
**Author:** [@seth.yes](https://discuss.elastic.co/u/seth.yes)\ **Replies:** 2\ **Last updated:** [February 7, 2017, 7:22pm UTC](https://discuss.elastic.co/t/logstash-date-filter-silently-failing-never-passes-events-to-es/73905 "2017-02-07T19:22:58Z")
I'm using Logstash 2.4.1, Filebeat 5.1.1, ES 2.4.3. I'm trying to use the 'date' filter in Logstash in order to replace the @timestamp value and index on the time reported within the event, rather than the time of inde… --- ## [Filebeat on client server sending logs to Logstash - Timing out](https://discuss.elastic.co/t/filebeat-on-client-server-sending-logs-to-logstash-timing-out/73965)
**Author:** [@jairaman12](https://discuss.elastic.co/u/jairaman12)\ **Replies:** 2\ **Last updated:** [February 7, 2017, 7:13pm UTC](https://discuss.elastic.co/t/filebeat-on-client-server-sending-logs-to-logstash-timing-out/73965 "2017-02-07T19:13:16Z")
Hi Everyone, I've been working on setting up Elk to pull Jenkins build logs into Kibana dashboard. I setup elasticsearch, kibana & logstash on one of AWS boxes and installed filebeat on the Jenkins master box. I'm actu… --- ## [Logstash sysconfig file missing](https://discuss.elastic.co/t/logstash-sysconfig-file-missing/74115)
**Author:** [@Rinat](https://discuss.elastic.co/u/Rinat)\ **Replies:** 5\ **Last updated:** [February 7, 2017, 5:42pm UTC](https://discuss.elastic.co/t/logstash-sysconfig-file-missing/74115 "2017-02-07T17:42:58Z")
Hi, I've been reading many posts about adjusting /etc/sysconfig/logstash or /etc/init.d/logstash files for some settings. I can't find any of those files in any of those directories. When I start Logstash it works. M… --- ## [Looking for Previous/Old Compatibility Matrix for v1.x](https://discuss.elastic.co/t/looking-for-previous-old-compatibility-matrix-for-v1-x/74253)
**Author:** [@TomG](https://discuss.elastic.co/u/TomG)\ **Replies:** 0\ **Last updated:** [February 7, 2017, 4:55pm UTC](https://discuss.elastic.co/t/looking-for-previous-old-compatibility-matrix-for-v1-x/74253 "2017-02-07T16:55:08Z")
Hi, I just checked Compatibility Matrix today and I discovered that all Logstash and Elasticsearch 1.x versions references have been removed. We still use those versions (I know that's bad) and sadly I did not save it… --- ## [DNS Plugin Logs](https://discuss.elastic.co/t/dns-plugin-logs/74245)
**Author:** [@kopacko](https://discuss.elastic.co/u/kopacko)\ **Replies:** 0\ **Last updated:** [February 7, 2017, 4:16pm UTC](https://discuss.elastic.co/t/dns-plugin-logs/74245 "2017-02-07T16:16:45Z")
I got started configuring some DNS reverse lookups for my firewall logs. I am seeings these in the logs: \[2017-02-07T10:11:18,041\]\[ERROR\]\[logstash.filters.dns \] DNS: timeout on resolving address. {:field=\>"destina… --- ## [Parsing syslog entries](https://discuss.elastic.co/t/parsing-syslog-entries/24653)
**Author:** [@eacoimbra](https://discuss.elastic.co/u/eacoimbra)\ **Replies:** 5\ **Last updated:** [February 7, 2017, 4:05pm UTC](https://discuss.elastic.co/t/parsing-syslog-entries/24653 "2017-02-07T16:05:18Z")
I'm trying to structure my mail logs using the pattern %SYSLOGLINE (https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/linux-syslog). The parse is ok and I'm not getting grokparsefailure, b… --- ## [Logstash - Datadog metrics - Unable to send them to Datadog](https://discuss.elastic.co/t/logstash-datadog-metrics-unable-to-send-them-to-datadog/74240)
**Author:** [@david-niceact](https://discuss.elastic.co/u/david-niceact)\ **Replies:** 1\ **Last updated:** [February 7, 2017, 3:33pm UTC](https://discuss.elastic.co/t/logstash-datadog-metrics-unable-to-send-them-to-datadog/74240 "2017-02-07T15:33:17Z")
Hello everyone, I am trying to use the Datadog metric plugin for logstash. I have configured it to use a CSV file as an input and then perform the parse with grok. Unfortately I am not able to take these metrics and p… --- ## [Problem with S3 as Input](https://discuss.elastic.co/t/problem-with-s3-as-input/38280)
**Author:** [@Nitz](https://discuss.elastic.co/u/Nitz)\ **Replies:** 2\ **Last updated:** [February 7, 2017, 1:41pm UTC](https://discuss.elastic.co/t/problem-with-s3-as-input/38280 "2017-02-07T13:41:10Z")
Hi all, I'm having the following configuration: input { s3 { type =\> "cloudtrail" bucket =\> $BUCKET\_NAME prefix =\> $PREFIX access\_key\_id =\> $ACCESS\_KEY\_ID secret\_access\_key =\> $SECRET region… --- ## [User\_agent and geoip filters does not work](https://discuss.elastic.co/t/user-agent-and-geoip-filters-does-not-work/74219)
**Author:** [@jonasv](https://discuss.elastic.co/u/jonasv)\ **Replies:** 11\ **Last updated:** [February 7, 2017, 1:36pm UTC](https://discuss.elastic.co/t/user-agent-and-geoip-filters-does-not-work/74219 "2017-02-07T13:36:20Z")
Hey, I'm got a field which looks like this: Mozilla/5.0 (X11; Linux x86\_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36 OPR/42.0.2393.94 or this: curl/7.47.0 filter { mutate { remove\_… --- ## [Howto - Logstashon OpenBSD 5.9](https://discuss.elastic.co/t/howto-logstashon-openbsd-5-9/74226)
**Author:** [@srikanthchandika](https://discuss.elastic.co/u/srikanthchandika)\ **Replies:** 0\ **Last updated:** [February 7, 2017, 1:21pm UTC](https://discuss.elastic.co/t/howto-logstashon-openbsd-5-9/74226 "2017-02-07T13:21:47Z")
How to install Logstash on OpenBSD 5.9 to push OpenBSD logs to elasticsearch! --- ## [Referencing fields in Metrics output](https://discuss.elastic.co/t/referencing-fields-in-metrics-output/60869)
**Author:** [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Replies:** 6\ **Last updated:** [February 7, 2017, 12:14pm UTC](https://discuss.elastic.co/t/referencing-fields-in-metrics-output/60869 "2017-02-07T12:14:51Z")
Hi, I'm trying to output a custom formatted line which must include: Events per second metric. The LS hostname. I have this metric filter in the "filter" section: metrics { meter =\> "events" add\_tag =\> "me… --- ## [5.1.1: Plugin installation behind Proxy](https://discuss.elastic.co/t/5-1-1-plugin-installation-behind-proxy/70454)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 11\ **Last updated:** [February 7, 2017, 12:13pm UTC](https://discuss.elastic.co/t/5-1-1-plugin-installation-behind-proxy/70454 "2017-02-07T12:13:18Z")
Hi, I'm finding that plugin installation is failing from behind proxy. This has me wondering how proxying should be configured in the current version. Neither of these approaches are working for me: # export HTTP\_PROXY… --- ## [Create metrics with dynamic names](https://discuss.elastic.co/t/create-metrics-with-dynamic-names/74210)
**Author:** [@danieljamesscott](https://discuss.elastic.co/u/danieljamesscott)\ **Replies:** 0\ **Last updated:** [February 7, 2017, 11:15am UTC](https://discuss.elastic.co/t/create-metrics-with-dynamic-names/74210 "2017-02-07T11:15:11Z")
Is it possible to create metrics with a dynamic name, and output all of them, without having to list the possible values individually? For example, we have request logs from apache coming into logstash, and we would li… --- ## [Apparent bug in logstash-input-beats in SslSimpleBuilder.java](https://discuss.elastic.co/t/apparent-bug-in-logstash-input-beats-in-sslsimplebuilder-java/74206)
**Author:** [@nick-george](https://discuss.elastic.co/u/nick-george)\ **Replies:** 0\ **Last updated:** [February 7, 2017, 10:36am UTC](https://discuss.elastic.co/t/apparent-bug-in-logstash-input-beats-in-sslsimplebuilder-java/74206 "2017-02-07T10:36:35Z")
Hi, If the user has set verify\_mode to NONE in the logstash beats input configuration, it is overridden default value set by the code in SslSimpleBuilder.java, see below: SslClientVerifyMode verifyMode = SslClientV… --- ## [\_grokparsefailure even if pattern matches](https://discuss.elastic.co/t/-grokparsefailure-even-if-pattern-matches/73839)
**Author:** [@jonasv](https://discuss.elastic.co/u/jonasv)\ **Replies:** 4\ **Last updated:** [February 7, 2017, 10:28am UTC](https://discuss.elastic.co/t/-grokparsefailure-even-if-pattern-matches/73839 "2017-02-07T10:28:32Z")
I've got this pattern "03/Feb/2017:11:14:44 +0000" and I've got this regexp NGDATE (^....................) which if I try on any online regexp tool match 03/Feb/2017:11:14:44 but for some reason I get \_grokparsefailure … --- ## [If Else in Logstash](https://discuss.elastic.co/t/if-else-in-logstash/74008)
**Author:** [@bini](https://discuss.elastic.co/u/bini)\ **Replies:** 10\ **Last updated:** [February 7, 2017, 9:13am UTC](https://discuss.elastic.co/t/if-else-in-logstash/74008 "2017-02-07T09:13:50Z")
Hi , i am trying to create an index when the condition is if \[fs.mount\_point\] == "C:\\" and \[fs.used\] == "87264018432" i am trying to push data from topbeat to logstash. and then i am trying to give the if condition. But… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1039) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1041)