# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1195 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1196 --- ## [Extract certain fields from Oracle listener log](https://discuss.elastic.co/t/extract-certain-fields-from-oracle-listener-log/31992)
**Author:** [@oravirt](https://discuss.elastic.co/u/oravirt)\ **Replies:** 8\ **Last updated:** [February 3, 2016, 10:52pm UTC](https://discuss.elastic.co/t/extract-certain-fields-from-oracle-listener-log/31992 "2016-02-03T22:52:31Z")
Hi, First off, I'm new to logstash so I appreciate any help I can get with this problem. I'm trying to get some information from the Oracle listener log into logstash but I'm not entirely sure how do that. Logstash r… --- ## [12345](https://discuss.elastic.co/t/topic/27897)
**Author:** [@jamasdj123](https://discuss.elastic.co/u/jamasdj123)\ **Replies:** 3\ **Last updated:** [February 3, 2016, 10:47pm UTC](https://discuss.elastic.co/t/topic/27897 "2016-02-03T22:47:30Z")
12345 --- ## [Get rid of duplicated items](https://discuss.elastic.co/t/get-rid-of-duplicated-items/40630)
**Author:** [@nkoleff](https://discuss.elastic.co/u/nkoleff)\ **Replies:** 3\ **Last updated:** [February 3, 2016, 10:10pm UTC](https://discuss.elastic.co/t/get-rid-of-duplicated-items/40630 "2016-02-03T22:10:36Z")
Hello, I've got some duplicated items in logstash which I want to get rid of in the future with implementing the fingerprint filter. I've seen an example here - https://discuss.elastic.co/t/logstash-duplicate-messag… --- ## [Adding fields to data read by the file plugin](https://discuss.elastic.co/t/adding-fields-to-data-read-by-the-file-plugin/40302)
**Author:** [@jerrac](https://discuss.elastic.co/u/jerrac)\ **Replies:** 2\ **Last updated:** [February 3, 2016, 10:08pm UTC](https://discuss.elastic.co/t/adding-fields-to-data-read-by-the-file-plugin/40302 "2016-02-03T22:08:52Z")
Using Filebeat, I can add a field that ends up looking like: "fields": { "fieldname": "value" }, So far I can't figure out how to do the same with Logstash's file input plugin. I can't do: add\_field =\> { "fields… --- ## [Logstash 1.5.6 doesn't accept connections from logstash-forwarder under load](https://discuss.elastic.co/t/logstash-1-5-6-doesnt-accept-connections-from-logstash-forwarder-under-load/40806)
**Author:** [@Karol\_Stojek](https://discuss.elastic.co/u/Karol_Stojek)\ **Replies:** 3\ **Last updated:** [February 3, 2016, 8:09pm UTC](https://discuss.elastic.co/t/logstash-1-5-6-doesnt-accept-connections-from-logstash-forwarder-under-load/40806 "2016-02-03T20:09:24Z")
Hello! I'm having troubles with stabilizing Logstash 1.5.6 under load (30-60k tps). I've reduced the problem to a simplest possible case (to eliminate elasticsearch as an output blocker): input with lumberjack and… --- ## [Trouble with using Logstash for the first time](https://discuss.elastic.co/t/trouble-with-using-logstash-for-the-first-time/40790)
**Author:** [@hiplerG](https://discuss.elastic.co/u/hiplerG)\ **Replies:** 1\ **Last updated:** [February 3, 2016, 6:11pm UTC](https://discuss.elastic.co/t/trouble-with-using-logstash-for-the-first-time/40790 "2016-02-03T18:11:00Z")
Hi I'm trying to use Logstash for the first time and I'm following the Docs in https://www.elastic.co/guide/en/logstash/2.1/first-event.html After installing the latest 2.2 release (on Ubuntu 14.04 with Java \>= 1.8.0\_65… --- ## [Windows source grok match](https://discuss.elastic.co/t/windows-source-grok-match/40778)
**Author:** [@sd\_karthik](https://discuss.elastic.co/u/sd_karthik)\ **Replies:** 2\ **Last updated:** [February 3, 2016, 4:16pm UTC](https://discuss.elastic.co/t/windows-source-grok-match/40778 "2016-02-03T16:16:44Z")
How do I write a grok for source in windows file My source is -- "T:\\pricelinelogs\\tripair\\Xml.16-02-02.log" How do i grok for "tripair" and the log file name in the source? --- ## [Replace my @timestamp](https://discuss.elastic.co/t/replace-my-timestamp/40739)
**Author:** [@m.francia](https://discuss.elastic.co/u/m.francia)\ **Replies:** 4\ **Last updated:** [February 3, 2016, 1:07pm UTC](https://discuss.elastic.co/t/replace-my-timestamp/40739 "2016-02-03T13:07:25Z")
Hi! I have a log that looks like this "2016-02-02 14:52:18" , and I want to replace my @timestamp, I tried different ways but none works. I have read many entries but I didn´t find a solution. Actually im trying with thi… --- ## [How to load xml file using logstash in es?](https://discuss.elastic.co/t/how-to-load-xml-file-using-logstash-in-es/40839)
**Author:** [@shankarananth.th](https://discuss.elastic.co/u/shankarananth.th)\ **Replies:** 6\ **Last updated:** [February 3, 2016, 12:36pm UTC](https://discuss.elastic.co/t/how-to-load-xml-file-using-logstash-in-es/40839 "2016-02-03T12:36:20Z")
Hi , I'm very new to elk. Kindly find my data sample in attachment. How can i load that xml data using logstash. I used the below mentioned code to load it. input { stdin { type =\> "stdin-type" } file { type =\> … --- ## [Syslog date without year](https://discuss.elastic.co/t/syslog-date-without-year/29834)
**Author:** [@Miguel\_Bessa](https://discuss.elastic.co/u/Miguel_Bessa)\ **Replies:** 10\ **Last updated:** [February 3, 2016, 11:47am UTC](https://discuss.elastic.co/t/syslog-date-without-year/29834 "2016-02-03T11:47:27Z")
Hi, I'm using Logstash to analyze my syslog files. My syslog files have the date like this: Jun 30 06:34:35. I want have the year to, because I need to use date to filter visualizations on kibana. I don't use kibana @… --- ## [Use http output to send data](https://discuss.elastic.co/t/use-http-output-to-send-data/39862)
**Author:** [@punit\_naik1](https://discuss.elastic.co/u/punit_naik1)\ **Replies:** 3\ **Last updated:** [February 3, 2016, 11:30am UTC](https://discuss.elastic.co/t/use-http-output-to-send-data/39862 "2016-02-03T11:30:12Z")
At my company we have our own database. We can store data into it by making http requests to it. So I wanted to store logs to the database using http output method. Now there is a multitude of logs and its big data in it… --- ## [Time ranges alerts](https://discuss.elastic.co/t/time-ranges-alerts/40846)
**Author:** [@pmatv](https://discuss.elastic.co/u/pmatv)\ **Replies:** 0\ **Last updated:** [February 3, 2016, 9:15am UTC](https://discuss.elastic.co/t/time-ranges-alerts/40846 "2016-02-03T09:15:53Z")
Hello everyone! I've configured logstash for parsing Nginx web-server log files. There were email notifications generated in case of specific status code - 400,404,500 etc. But now it generates hundreds/thousands mails… --- ## [Troubling with add\_field in filter. Please advice](https://discuss.elastic.co/t/troubling-with-add-field-in-filter-please-advice/40848)
**Author:** [@alfsolli](https://discuss.elastic.co/u/alfsolli)\ **Replies:** 1\ **Last updated:** [February 3, 2016, 9:36am UTC](https://discuss.elastic.co/t/troubling-with-add-field-in-filter-please-advice/40848 "2016-02-03T09:36:29Z")
Hi. Fresh off the boat ELK user here. Can someone point out what I'm doing wrong here? I'm trying to translate a field (IP address) into FQDN with logstash. The data source is netflow, and I'm trying to get the IP a… --- ## [Logstash imap input plugin](https://discuss.elastic.co/t/logstash-imap-input-plugin/39336)
**Author:** [@nkoleff](https://discuss.elastic.co/u/nkoleff)\ **Replies:** 2\ **Last updated:** [February 3, 2016, 7:41am UTC](https://discuss.elastic.co/t/logstash-imap-input-plugin/39336 "2016-02-03T07:41:21Z")
Hello, For some time now my imap input is failing with End of file reached and I'm wondering what could be the problem with it. It's working for like 10 to 15 minutes and then suddenly stops with that message: {:times… --- ## [Parse Json data wrapped in "message" into event fields](https://discuss.elastic.co/t/parse-json-data-wrapped-in-message-into-event-fields/40072)
**Author:** [@sara](https://discuss.elastic.co/u/sara)\ **Replies:** 5\ **Last updated:** [February 3, 2016, 7:03am UTC](https://discuss.elastic.co/t/parse-json-data-wrapped-in-message-into-event-fields/40072 "2016-02-03T07:03:01Z")
the issue is that the log in json format, sent from filebeat to logstash, the json data got wrapped in the message field and cannot be parsed into event fields. I have read a lot of posts on the similar issue over this w… --- ## [Logstash shipper events stuck at redis](https://discuss.elastic.co/t/logstash-shipper-events-stuck-at-redis/40694)
**Author:** [@Kishore](https://discuss.elastic.co/u/Kishore)\ **Replies:** 6\ **Last updated:** [February 3, 2016, 6:38am UTC](https://discuss.elastic.co/t/logstash-shipper-events-stuck-at-redis/40694 "2016-02-03T06:38:49Z")
Kindly verify my redis input configuration for logstash indexer, below is the input config details which worked for some time later it is not working. I have upgraded to logstash 2.1.1 from logstash 1.4.2 input{ redis … --- ## [How to troubleshoot LS](https://discuss.elastic.co/t/how-to-troubleshoot-ls/40786)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 3\ **Last updated:** [February 2, 2016, 9:15pm UTC](https://discuss.elastic.co/t/how-to-troubleshoot-ls/40786 "2016-02-02T21:15:17Z")
Hi Experts, I am using LS 1.5.4 , and I want to troubleshoot LS. Can anyone tell me where I find error logs for Logstash. I do not see it the way kibana or ES save it's app logs in the directory . please suggest. T… --- ## [Label individual instances of logstash through cmdline params](https://discuss.elastic.co/t/label-individual-instances-of-logstash-through-cmdline-params/40513)
**Author:** [@biswajit86](https://discuss.elastic.co/u/biswajit86)\ **Replies:** 6\ **Last updated:** [February 2, 2016, 8:25pm UTC](https://discuss.elastic.co/t/label-individual-instances-of-logstash-through-cmdline-params/40513 "2016-02-02T20:25:26Z")
is there a way to label individual instances of logstash ? I run multiple logstash instances on a host and it works fine. But when I run topbeat on the host, it identifies that same proc.name for all the instances if t… --- ## [Sending email on some event](https://discuss.elastic.co/t/sending-email-on-some-event/40540)
**Author:** [@abinay](https://discuss.elastic.co/u/abinay)\ **Replies:** 10\ **Last updated:** [February 2, 2016, 8:23pm UTC](https://discuss.elastic.co/t/sending-email-on-some-event/40540 "2016-02-02T20:23:04Z")
Hii I am using logstash . I want to send an email on receiving a error log . This is my config - if \[loglevel\] == "local.ERROR" or \[loglevel\] == "staging.ERROR" { email { from =\>… --- ## [Connecting Kinesis to Logstash](https://discuss.elastic.co/t/connecting-kinesis-to-logstash/40681)
**Author:** [@treehouse](https://discuss.elastic.co/u/treehouse)\ **Replies:** 0\ **Last updated:** [February 1, 2016, 11:04pm UTC](https://discuss.elastic.co/t/connecting-kinesis-to-logstash/40681 "2016-02-01T23:04:12Z")
Hi! We are currently running a full production deployment of an ELK pipeline. Our current setup: Application machines -\> Log-courier -\> logstash -\> ES -\> Kibana For reasons i wont go into here, we have to drop log-cour… --- ## [Sending ganglia metric to logstash and the displaying them on kibana](https://discuss.elastic.co/t/sending-ganglia-metric-to-logstash-and-the-displaying-them-on-kibana/40567)
**Author:** [@jstar](https://discuss.elastic.co/u/jstar)\ **Replies:** 12\ **Last updated:** [February 2, 2016, 7:34pm UTC](https://discuss.elastic.co/t/sending-ganglia-metric-to-logstash-and-the-displaying-them-on-kibana/40567 "2016-02-02T19:34:18Z")
Hi All, I was able to send some metrics of ganglia to logstash. Here is my input configuration input { lumberjack { port =\> 5043 type =\> "logs" ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt" ssl\_ke… --- ## [Grok escape](https://discuss.elastic.co/t/grok-escape/40679)
**Author:** [@sd\_karthik](https://discuss.elastic.co/u/sd_karthik)\ **Replies:** 4\ **Last updated:** [February 2, 2016, 6:52pm UTC](https://discuss.elastic.co/t/grok-escape/40679 "2016-02-02T18:52:43Z")
Below is the source I have -- "source" =\> "/apps/home/cache22/logs/room\_description.16-02-01\_17.log" and my match currently is -- match =\> \["source","%{GREEDYDATA}/%{GREEDYDATA:filename}.log"\] But I need to also … --- ## [Invalid format for timestamp field](https://discuss.elastic.co/t/invalid-format-for-timestamp-field/34793)
**Author:** [@alex22](https://discuss.elastic.co/u/alex22)\ **Replies:** 6\ **Last updated:** [February 2, 2016, 3:37pm UTC](https://discuss.elastic.co/t/invalid-format-for-timestamp-field/34793 "2016-02-02T15:37:44Z")
Hi there, i'm currently facing an issue with the timestamp field after upgrading logstash from 1.5 to 2.0. The data i try to parse is: "timestamp"=\>"14/Nov/2015:10:47:15 +0100", And the date filter looks like t… --- ## [Message format in logstash output plugin](https://discuss.elastic.co/t/message-format-in-logstash-output-plugin/40614)
**Author:** [@bharath\_meka](https://discuss.elastic.co/u/bharath_meka)\ **Replies:** 2\ **Last updated:** [February 2, 2016, 11:13am UTC](https://discuss.elastic.co/t/message-format-in-logstash-output-plugin/40614 "2016-02-02T11:13:28Z")
Hi, We have a custom logstash output plugin to post some messages to Slack. If I use a static message like "Testing from logstash", we are able to post it. But if I use logstash variables like %{message} or any other cu… --- ## [Unable to install logstash via chef cookbook in aws opsworks](https://discuss.elastic.co/t/unable-to-install-logstash-via-chef-cookbook-in-aws-opsworks/40705)
**Author:** [@mazjs](https://discuss.elastic.co/u/mazjs)\ **Replies:** 1\ **Last updated:** [February 2, 2016, 10:13am UTC](https://discuss.elastic.co/t/unable-to-install-logstash-via-chef-cookbook-in-aws-opsworks/40705 "2016-02-02T10:13:43Z")
HI, I am using https://github.com/lusis/chef-logstash coobook to install logstash in aws opsworks. But the chef gives error with : Missing Cookbooks: No such cookbook: chef\_handler I do have chef-handler cookbook… --- ## [Work with output and metrics](https://discuss.elastic.co/t/work-with-output-and-metrics/40464)
**Author:** [@marcmga](https://discuss.elastic.co/u/marcmga)\ **Replies:** 4\ **Last updated:** [February 2, 2016, 7:58am UTC](https://discuss.elastic.co/t/work-with-output-and-metrics/40464 "2016-02-02T07:58:51Z")
Hi, I'm working with metrics. Here's my code, which in meter i use the hostname to isolate metrics. metrics { meter =\> \[ "%{host}" \] flush\_interval =\> 60 clear\_interval =\> 300 } In kibana i'm able to see the me… --- ## [When will Logstash 2.2 be available in the repos?](https://discuss.elastic.co/t/when-will-logstash-2-2-be-available-in-the-repos/40701)
**Author:** [@kaib](https://discuss.elastic.co/u/kaib)\ **Replies:** 0\ **Last updated:** [February 2, 2016, 7:36am UTC](https://discuss.elastic.co/t/when-will-logstash-2-2-be-available-in-the-repos/40701 "2016-02-02T07:36:43Z")
Hi Folks, i have just a general Question: How long does it take a new Version to be available in the Linux Repos? Best regards, Kai --- ## [What is the purpose of type field in Input section](https://discuss.elastic.co/t/what-is-the-purpose-of-type-field-in-input-section/39828)
**Author:** [@gaurav1424](https://discuss.elastic.co/u/gaurav1424)\ **Replies:** 17\ **Last updated:** [February 2, 2016, 7:05am UTC](https://discuss.elastic.co/t/what-is-the-purpose-of-type-field-in-input-section/39828 "2016-02-02T07:05:14Z")
Hello All, Whats the use of "type =\> " in Input section of Logstash ? Anyways I will be using grok filter in filter section to filter incoming messages ? Thanks, gaurav --- ## [Mutate a field depending on field type](https://discuss.elastic.co/t/mutate-a-field-depending-on-field-type/40536)
**Author:** [@Florin\_Andrei](https://discuss.elastic.co/u/Florin_Andrei)\ **Replies:** 2\ **Last updated:** [February 2, 2016, 12:42am UTC](https://discuss.elastic.co/t/mutate-a-field-depending-on-field-type/40536 "2016-02-02T00:42:36Z")
I'm parsing logs from a node.js app. One of the fields is a JSON document that the app inserts in the logs. I tried to parse the JSON part in a very basic way, which appeared to work fine for a while: json { sour… --- ## [Unable to download data from https://rubygems.org - certificate verify failed](https://discuss.elastic.co/t/unable-to-download-data-from-https-rubygems-org-certificate-verify-failed/40629)
**Author:** [@DK\_3](https://discuss.elastic.co/u/DK_3)\ **Replies:** 1\ **Last updated:** [February 1, 2016, 8:42pm UTC](https://discuss.elastic.co/t/unable-to-download-data-from-https-rubygems-org-certificate-verify-failed/40629 "2016-02-01T20:42:05Z")
I'm hitting the following when trying to install a Logstash plugin /logstash/bin/plugin install logstash-input-courier Validating logstash-input-courier Unable to download data from https://rubygems.org - certificate … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1194) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1196)