# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1039 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1040 --- ## [Issue with GeoIP location](https://discuss.elastic.co/t/issue-with-geoip-location/72738)
**Author:** [@vineets928](https://discuss.elastic.co/u/vineets928)\ **Replies:** 4\ **Last updated:** [February 9, 2017, 12:00pm UTC](https://discuss.elastic.co/t/issue-with-geoip-location/72738 "2017-02-09T12:00:28Z")
Hi, I am forwarding apache log to logstash using filebeat and using GeoLite.dat database for geoip. The configuration file is: input { beats { port =\> "5044" } } The filter part of this file is commented out to in… --- ## [@timestamp shows indexed date/time and not time from log entry](https://discuss.elastic.co/t/timestamp-shows-indexed-date-time-and-not-time-from-log-entry/74472)
**Author:** [@hoagieben](https://discuss.elastic.co/u/hoagieben)\ **Replies:** 4\ **Last updated:** [February 9, 2017, 9:37am UTC](https://discuss.elastic.co/t/timestamp-shows-indexed-date-time-and-not-time-from-log-entry/74472 "2017-02-09T09:37:21Z")
Being very new to ELK I have borrowed the following logstash config which seems to work in the main. Annoyingly though the @timestamp values are not the same as the values in the WebSphere logs but the date/time in which… --- ## [Converting elapsed time 00:00:00:XXXXXXX](https://discuss.elastic.co/t/converting-elapsed-time-00-00-00-xxxxxxx/74237)
**Author:** [@dsever](https://discuss.elastic.co/u/dsever)\ **Replies:** 2\ **Last updated:** [February 9, 2017, 8:20am UTC](https://discuss.elastic.co/t/converting-elapsed-time-00-00-00-xxxxxxx/74237 "2017-02-09T08:20:56Z")
Hi there, I have small problem with small problem with preparing elapsed time for executing db procedure. For instance using grok as %{TIME:elapsed} and it matches ok, but at eleastic it provides me value as string. f… --- ## [Parsing a variable length log](https://discuss.elastic.co/t/parsing-a-variable-length-log/74436)
**Author:** [@mcnovy](https://discuss.elastic.co/u/mcnovy)\ **Replies:** 1\ **Last updated:** [February 9, 2017, 6:58am UTC](https://discuss.elastic.co/t/parsing-a-variable-length-log/74436 "2017-02-09T06:58:16Z")
Hi, Sorry is this if too obvious, but I feel like I have been browsing through all of google now :slight\_smile: I have a log, where I can't seem to figure out any patterns. I need the location from that line, but t… --- ## [Csv output is not working](https://discuss.elastic.co/t/csv-output-is-not-working/71554)
**Author:** [@ctangirala](https://discuss.elastic.co/u/ctangirala)\ **Replies:** 3\ **Last updated:** [February 9, 2017, 5:18am UTC](https://discuss.elastic.co/t/csv-output-is-not-working/71554 "2017-02-09T05:18:11Z")
I am trying to use elasticsearch input and CSV output to query my index and output a CSV file with data. However, the CSV file does't have the any data after logstash finishes processing. All I see is timestamp and %{mes… --- ## [GEOIP Enable](https://discuss.elastic.co/t/geoip-enable/74116)
**Author:** [@mhalatuituia](https://discuss.elastic.co/u/mhalatuituia)\ **Replies:** 24\ **Last updated:** [February 9, 2017, 3:21am UTC](https://discuss.elastic.co/t/geoip-enable/74116 "2017-02-09T03:21:56Z")
Hi Firstly i am apologise for bugging the forum but it is just that i really want this to work. I have setup the stack using this link https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logsta… --- ## [Logstash kafka output plugin sent messages without error but messages were not received](https://discuss.elastic.co/t/logstash-kafka-output-plugin-sent-messages-without-error-but-messages-were-not-received/62366)
**Author:** [@zpp](https://discuss.elastic.co/u/zpp)\ **Replies:** 2\ **Last updated:** [February 9, 2017, 12:54am UTC](https://discuss.elastic.co/t/logstash-kafka-output-plugin-sent-messages-without-error-but-messages-were-not-received/62366 "2017-02-09T00:54:18Z")
Just started testing the logstash kafka output plugin, configuration as below. output { kafka { bootstrap\_servers =\> "ServerA:6667" topic\_id =\> "test3" } I enabled debug… --- ## [How to set Log4J level on individual Logstash plugin?](https://discuss.elastic.co/t/how-to-set-log4j-level-on-individual-logstash-plugin/74306)
**Author:** [@Defaux](https://discuss.elastic.co/u/Defaux)\ **Replies:** 1\ **Last updated:** [February 8, 2017, 11:42pm UTC](https://discuss.elastic.co/t/how-to-set-log4j-level-on-individual-logstash-plugin/74306 "2017-02-08T23:42:26Z")
Background: I've been working extensively with the Logstash Netflow codec the past few days. Although it's been working fairly well, it's been filling my logs with a ton of messages like: ... \[2017-02-08T00:52:58,724\]… --- ## [Installing Logstash Plugin with spaces in the install path](https://discuss.elastic.co/t/installing-logstash-plugin-with-spaces-in-the-install-path/74443)
**Author:** [@timmy8ken](https://discuss.elastic.co/u/timmy8ken)\ **Replies:** 0\ **Last updated:** [February 8, 2017, 10:53pm UTC](https://discuss.elastic.co/t/installing-logstash-plugin-with-spaces-in-the-install-path/74443 "2017-02-08T22:53:30Z")
I am in the early stages of setting up a an elastic stack and I think I have discovered a bug, but it may be user error, so hopefully someone can point out where I am going wrong. I have installed Elasticsearch, Logsta… --- ## [New logstash 5.x does not support \`-server\` flag for JVM?](https://discuss.elastic.co/t/new-logstash-5-x-does-not-support-server-flag-for-jvm/74432)
**Author:** [@DravenJohnson](https://discuss.elastic.co/u/DravenJohnson)\ **Replies:** 3\ **Last updated:** [February 8, 2017, 9:48pm UTC](https://discuss.elastic.co/t/new-logstash-5-x-does-not-support-server-flag-for-jvm/74432 "2017-02-08T21:48:45Z")
We used to use -server flag to get better performance to logstash JVM server. Seem like this option no longer valid? We tried to set -server flag in jvm.option or startup.option, or both. None of those setting trigger… --- ## [Logstash X-Pack monitoring troubles](https://discuss.elastic.co/t/logstash-x-pack-monitoring-troubles/73578)
**Author:** [@Slurpgeit](https://discuss.elastic.co/u/Slurpgeit)\ **Replies:** 4\ **Last updated:** [February 8, 2017, 9:02pm UTC](https://discuss.elastic.co/t/logstash-x-pack-monitoring-troubles/73578 "2017-02-08T21:02:06Z")
Hello, I'm trying to setup monitoring of some logstash machines, but I'm running into a few issues: The documentation (https://www.elastic.co/guide/en/x-pack/5.2/monitoring-logstash.html) states that the "xpack.monit… --- ## [Logstash File Input Plugin and Windows Line Endings](https://discuss.elastic.co/t/logstash-file-input-plugin-and-windows-line-endings/73745)
**Author:** [@tdsacilowski](https://discuss.elastic.co/u/tdsacilowski)\ **Replies:** 1\ **Last updated:** [February 8, 2017, 7:46pm UTC](https://discuss.elastic.co/t/logstash-file-input-plugin-and-windows-line-endings/73745 "2017-02-08T19:46:55Z")
Hello, I have the following Logstash (v5.1) conf file that I'm using to ingest data and am running into an issue where it seems to not be recognizing Windows line endings as a row separator (\\r\\n): input { file {… --- ## [Read data from another tool](https://discuss.elastic.co/t/read-data-from-another-tool/74409)
**Author:** [@kmroz](https://discuss.elastic.co/u/kmroz)\ **Replies:** 0\ **Last updated:** [February 8, 2017, 4:49pm UTC](https://discuss.elastic.co/t/read-data-from-another-tool/74409 "2017-02-08T16:49:56Z")
Hello , I was wondering if there was a plugin or beat that can be used to read data from anothe tool. We are currently using opentsdb but I was curious if i can take some of the data generated there and have it be pres… --- ## [Syslog grok patterns and pipeline performance](https://discuss.elastic.co/t/syslog-grok-patterns-and-pipeline-performance/74386)
**Author:** [@imperatives](https://discuss.elastic.co/u/imperatives)\ **Replies:** 3\ **Last updated:** [February 8, 2017, 4:35pm UTC](https://discuss.elastic.co/t/syslog-grok-patterns-and-pipeline-performance/74386 "2017-02-08T16:35:46Z")
I've been working with RHEL syslogs (/var/log/secure and /var/log/message) that are being shipped to Logstash via Filebeat. I created grok patterns for each of the relevant log lines and create an additional field with … --- ## [Logstash 5.2 logstash-plugin fails with jar dependencies](https://discuss.elastic.co/t/logstash-5-2-logstash-plugin-fails-with-jar-dependencies/74141)
**Author:** [@csobchuk](https://discuss.elastic.co/u/csobchuk)\ **Replies:** 2\ **Last updated:** [February 8, 2017, 4:03pm UTC](https://discuss.elastic.co/t/logstash-5-2-logstash-plugin-fails-with-jar-dependencies/74141 "2017-02-08T16:03:22Z")
Hi I am using Logstash 5.2 on Ubuntu 14.04 and have been trying to list/install/update plugins, but I get the following error for any action. sudo bin/logstash-plugin list Gem::LoadError: You have already activate… --- ## [REST interface for logstash input plugin](https://discuss.elastic.co/t/rest-interface-for-logstash-input-plugin/73915)
**Author:** [@mkelkar](https://discuss.elastic.co/u/mkelkar)\ **Replies:** 4\ **Last updated:** [February 8, 2017, 4:03pm UTC](https://discuss.elastic.co/t/rest-interface-for-logstash-input-plugin/73915 "2017-02-08T16:03:05Z")
Hi All, I am trying to create a REST interface for my logstash input plugin to interact with it. I want to expose some REST endpoints so that I can index documents in logstash through my input plugin. I am using… --- ## [File input: ignore file portion written when output was failing](https://discuss.elastic.co/t/file-input-ignore-file-portion-written-when-output-was-failing/74391)
**Author:** [@aalesh](https://discuss.elastic.co/u/aalesh)\ **Replies:** 0\ **Last updated:** [February 8, 2017, 3:40pm UTC](https://discuss.elastic.co/t/file-input-ignore-file-portion-written-when-output-was-failing/74391 "2017-02-08T15:40:47Z")
Hi guys, We've recently faced a very worrying situation on our semi-PROD server. We have a typical setup: Logstash tails our logs, parses and pre-processes them and then sends to self-hosted Elasticsearch server. We h… --- ## [Logstash File input for file rotation](https://discuss.elastic.co/t/logstash-file-input-for-file-rotation/64479)
**Author:** [@Ankit\_Jain](https://discuss.elastic.co/u/Ankit_Jain)\ **Replies:** 1\ **Last updated:** [February 8, 2017, 3:20pm UTC](https://discuss.elastic.co/t/logstash-file-input-for-file-rotation/64479 "2017-02-08T15:20:44Z")
Hi Team, We have one use case where we need to read the server logs. We only want to read the log once at the same time don't want to loss single event. All the logging happening in X.log file and which is rolling af… --- ## [X509 : SAN ips / other erros](https://discuss.elastic.co/t/x509-san-ips-other-erros/74387)
**Author:** [@JimmyLumi](https://discuss.elastic.co/u/JimmyLumi)\ **Replies:** 0\ **Last updated:** [February 8, 2017, 3:15pm UTC](https://discuss.elastic.co/t/x509-san-ips-other-erros/74387 "2017-02-08T15:15:26Z")
Hello, Following this tutorial https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-ubuntu-16-04. Everything was going great, filebeat info are showing up (i … --- ## [Logstash-input-jdbc: hoto use the actual date in select-tablename](https://discuss.elastic.co/t/logstash-input-jdbc-hoto-use-the-actual-date-in-select-tablename/74032)
**Author:** [@asterix](https://discuss.elastic.co/u/asterix)\ **Replies:** 0\ **Last updated:** [February 6, 2017, 9:22am UTC](https://discuss.elastic.co/t/logstash-input-jdbc-hoto-use-the-actual-date-in-select-tablename/74032 "2017-02-06T09:22:23Z")
Hello there, I'm new to elasticsearch an logstahs. And I've got a little problem. Perhaps someone can help me: I want to read mysql-tables with logstash. This will work fine. But, because there are talles of the log of… --- ## [Concat the XML array value in logstash](https://discuss.elastic.co/t/concat-the-xml-array-value-in-logstash/73346)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 20\ **Last updated:** [February 8, 2017, 2:08pm UTC](https://discuss.elastic.co/t/concat-the-xml-array-value-in-logstash/73346 "2017-02-08T14:08:30Z")
HI Team, I'm just working on new type of XML file. I have some array information in that file i want to gather all those data.. This array fields are repeating with same tags due to this while processing the log the i'm … --- ## [Logstash input XML file size limit for parsing?](https://discuss.elastic.co/t/logstash-input-xml-file-size-limit-for-parsing/74375)
**Author:** [@rlv\_praveen](https://discuss.elastic.co/u/rlv_praveen)\ **Replies:** 1\ **Last updated:** [February 8, 2017, 2:02pm UTC](https://discuss.elastic.co/t/logstash-input-xml-file-size-limit-for-parsing/74375 "2017-02-08T14:02:03Z")
Hi, I am facing the issue in parsing the XML which is more than 10KB , but the same config file is parsing the file with less than 10 KB with out any errors. Please give me the some inputs how the logstash will be pa… --- ## [Logstash Split Plugin](https://discuss.elastic.co/t/logstash-split-plugin/74322)
**Author:** [@Shrawan\_Bhagwat](https://discuss.elastic.co/u/Shrawan_Bhagwat)\ **Replies:** 13\ **Last updated:** [February 8, 2017, 12:55pm UTC](https://discuss.elastic.co/t/logstash-split-plugin/74322 "2017-02-08T12:55:39Z")
Hi All, We do have JSON data in the form { "data": \[ { "appName": "DemoApp", "appVersion": "1.1", "deviceId": "1234567", "deviceName": "moto e", "deviceOSVersion": "5.1", "packageName": "com.abc.DemoApp… --- ## [Can't parse timestamp to @timestamp](https://discuss.elastic.co/t/cant-parse-timestamp-to-timestamp/74289)
**Author:** [@pgrzec2s](https://discuss.elastic.co/u/pgrzec2s)\ **Replies:** 6\ **Last updated:** [February 8, 2017, 11:33am UTC](https://discuss.elastic.co/t/cant-parse-timestamp-to-timestamp/74289 "2017-02-08T11:33:22Z")
Hello Folks... i am sorry for that Question, i saw a lot of topics about the same problem, but there isn't one which is helpfull in my case. In Logstash i would like to parse my timestamp to @timestamp but i receive a \_… --- ## [Problem with conditional in output definition](https://discuss.elastic.co/t/problem-with-conditional-in-output-definition/74302)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 1\ **Last updated:** [February 8, 2017, 11:05am UTC](https://discuss.elastic.co/t/problem-with-conditional-in-output-definition/74302 "2017-02-08T11:05:42Z")
I started experimenting with metricbeat and wanted the data in the index that the sample dashboards expect, so I tried settting the index conditionally on the input type. logstash complains that it is an invalid configur… --- ## [How to extract file name from source vairable](https://discuss.elastic.co/t/how-to-extract-file-name-from-source-vairable/74347)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 7\ **Last updated:** [February 8, 2017, 10:55am UTC](https://discuss.elastic.co/t/how-to-extract-file-name-from-source-vairable/74347 "2017-02-08T10:55:05Z")
HI Team, I need to extract the source field, when i process im getting grok parse failure please find the belwo image for your reference, Im using below grok pattern, %{PATH}/%{UUID:requestFileId} --- ## [Trying to add prefix to key in logstash config, but without period](https://discuss.elastic.co/t/trying-to-add-prefix-to-key-in-logstash-config-but-without-period/74350)
**Author:** [@LoriG](https://discuss.elastic.co/u/LoriG)\ **Replies:** 1\ **Last updated:** [February 8, 2017, 10:40am UTC](https://discuss.elastic.co/t/trying-to-add-prefix-to-key-in-logstash-config-but-without-period/74350 "2017-02-08T10:40:34Z")
From a component I am working on, I get certain information in the log that is report information. This information comes as "ReportBody". However, I want to add a prefix to some of the information (i.e. NumberOfDevices}… --- ## [Send logs to different index if grok parse fails](https://discuss.elastic.co/t/send-logs-to-different-index-if-grok-parse-fails/74353)
**Author:** [@froto](https://discuss.elastic.co/u/froto)\ **Replies:** 1\ **Last updated:** [February 8, 2017, 10:36am UTC](https://discuss.elastic.co/t/send-logs-to-different-index-if-grok-parse-fails/74353 "2017-02-08T10:36:00Z")
Hey there, at the moment i have the following problem: Many different devices are sending logs to port 514 and we dont want to reconifure all of these devices I want to make a logstash configuration which extract inf… --- ## [LS environment variable handling doesn't appear to behave as documented](https://discuss.elastic.co/t/ls-environment-variable-handling-doesnt-appear-to-behave-as-documented/73828)
**Author:** [@nick-george](https://discuss.elastic.co/u/nick-george)\ **Replies:** 12\ **Last updated:** [February 8, 2017, 10:02am UTC](https://discuss.elastic.co/t/ls-environment-variable-handling-doesnt-appear-to-behave-as-documented/73828 "2017-02-08T10:02:07Z")
Hi, LS Version: 5.1.2 (RPM) From the docs (https://www.elastic.co/guide/en/logstash/current/environment-variables.html): You can add environment variable references in any plugin option type : string, number, bool… --- ## [Logstash twitter input issue with proxy](https://discuss.elastic.co/t/logstash-twitter-input-issue-with-proxy/74351)
**Author:** [@jamal](https://discuss.elastic.co/u/jamal)\ **Replies:** 0\ **Last updated:** [February 8, 2017, 9:48am UTC](https://discuss.elastic.co/t/logstash-twitter-input-issue-with-proxy/74351 "2017-02-08T09:48:51Z")
I am using logstash 5.1.2 when i run it with out the proxy it work fine buty it does'nt work with proxy anc give me connection refused error any one could help me how to solve it ? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1038) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1040)