# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1227 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1228 --- ## [Using logstash File Input to monitor files on a network share](https://discuss.elastic.co/t/using-logstash-file-input-to-monitor-files-on-a-network-share/182)
**Author:** [@markus](https://discuss.elastic.co/u/markus)\ **Replies:** 2\ **Last updated:** [October 27, 2015, 2:18pm UTC](https://discuss.elastic.co/t/using-logstash-file-input-to-monitor-files-on-a-network-share/182 "2015-10-27T14:18:17Z")
Hi, we are using Logstash on Windows to process logfiles residing on several windows servers. Is it possible / advisible to setup network shares to the various logfiles from the shipper machine and ship all log files fro… --- ## [Import table with compound key](https://discuss.elastic.co/t/import-table-with-compound-key/32979)
**Author:** [@lore6673](https://discuss.elastic.co/u/lore6673)\ **Replies:** 4\ **Last updated:** [October 27, 2015, 9:42am UTC](https://discuss.elastic.co/t/import-table-with-compound-key/32979 "2015-10-27T09:42:32Z")
Hi there I need to import a table from Oracle using logstash and jdbc input with a compound primary key, Product\_id and Language\_id. This is my conf file input { jdbc { jdbc\_connection\_string =\> "jdbc:… --- ## [IAM credentials not recognised/used for s3 - plugins used input-s3 + codec-cloudtrail](https://discuss.elastic.co/t/iam-credentials-not-recognised-used-for-s3-plugins-used-input-s3-codec-cloudtrail/32464)
**Author:** [@plonka2000](https://discuss.elastic.co/u/plonka2000)\ **Replies:** 12\ **Last updated:** [October 27, 2015, 9:27am UTC](https://discuss.elastic.co/t/iam-credentials-not-recognised-used-for-s3-plugins-used-input-s3-codec-cloudtrail/32464 "2015-10-27T09:27:12Z")
Hi, Scenario is I am trying to use Logstash to: -pull AWS CloudTrail '.json.gz' logs from an S3 bucket using logstash-input-s3 plugin -process them using the logstash-codec-cloudtrail plugin -send them to Elasticsearch… --- ## [Has anyone gotten logstash to pull from IBM WebSphere MQ?](https://discuss.elastic.co/t/has-anyone-gotten-logstash-to-pull-from-ibm-websphere-mq/24729)
**Author:** [@whyapenny](https://discuss.elastic.co/u/whyapenny)\ **Replies:** 3\ **Last updated:** [October 27, 2015, 9:19am UTC](https://discuss.elastic.co/t/has-anyone-gotten-logstash-to-pull-from-ibm-websphere-mq/24729 "2015-10-27T09:19:27Z")
Attempting to use MQ instead of redis, not having much luck. --- ## [Logstash - clone event (ruby code)](https://discuss.elastic.co/t/logstash-clone-event-ruby-code/32816)
**Author:** [@etfeet](https://discuss.elastic.co/u/etfeet)\ **Replies:** 1\ **Last updated:** [October 27, 2015, 9:17am UTC](https://discuss.elastic.co/t/logstash-clone-event-ruby-code/32816 "2015-10-27T09:17:12Z")
I've got some log lines that have a count entry and I would like to clone the event for the value of the count in the log line. I'm using a ruby code to do a for loop for the count #. However, the ruby code is outputting… --- ## [About logstash can't use logstash-output-webhdfs](https://discuss.elastic.co/t/about-logstash-cant-use-logstash-output-webhdfs/33049)
**Author:** [@bigfootwalk](https://discuss.elastic.co/u/bigfootwalk)\ **Replies:** 0\ **Last updated:** [October 27, 2015, 7:47am UTC](https://discuss.elastic.co/t/about-logstash-cant-use-logstash-output-webhdfs/33049 "2015-10-27T07:47:34Z")
Logstash can not be installed by "bin/plugin install logstash-output-webhdfs" After via the https://github.com/logstash-plugins/logstash-output-webhdfs downloading the zip, using the "bin/plugin install logstash-output-w… --- ## [Getting different log path from API](https://discuss.elastic.co/t/getting-different-log-path-from-api/32974)
**Author:** [@Navneet\_Mathpal](https://discuss.elastic.co/u/Navneet_Mathpal)\ **Replies:** 3\ **Last updated:** [October 27, 2015, 7:12am UTC](https://discuss.elastic.co/t/getting-different-log-path-from-api/32974 "2015-10-27T07:12:32Z")
Hi, I have a requirement where I need to get the file path from an API, I am getting the API in json format , which contains file path, location, etc. ex- \[{ "filepath" : "d:/systemlogs/systemlog-12-jan-2015", "err… --- ## [Email notification for different people?](https://discuss.elastic.co/t/email-notification-for-different-people/32843)
**Author:** [@abcfy2](https://discuss.elastic.co/u/abcfy2)\ **Replies:** 6\ **Last updated:** [October 27, 2015, 1:27am UTC](https://discuss.elastic.co/t/email-notification-for-different-people/32843 "2015-10-27T01:27:48Z")
I want to email different people like this. email { if \[level\] = "low" { to =\> "somepeole@domain" } else { to =\> "otherpeople@domain" } from =\> "foo" body =\> "%{message}" ... } Bu… --- ## [Imap plugin crashes logstash when there are attachments](https://discuss.elastic.co/t/imap-plugin-crashes-logstash-when-there-are-attachments/1420)
**Author:** [@Jerry\_Hoffmeister](https://discuss.elastic.co/u/Jerry_Hoffmeister)\ **Replies:** 4\ **Last updated:** [October 26, 2015, 9:46pm UTC](https://discuss.elastic.co/t/imap-plugin-crashes-logstash-when-there-are-attachments/1420 "2015-10-26T21:46:06Z")
My issue is the same as was reported about a year ago here: https://logstash.jira.com/browse/LOGSTASH-2080 logstash silently (other than the two below log entries) dies when it tries to read an e-mail with an attachm… --- ## [Default mapping Location](https://discuss.elastic.co/t/default-mapping-location/32950)
**Author:** [@hyphen1370](https://discuss.elastic.co/u/hyphen1370)\ **Replies:** 5\ **Last updated:** [October 26, 2015, 5:32pm UTC](https://discuss.elastic.co/t/default-mapping-location/32950 "2015-10-26T17:32:46Z")
Hello, Is there anyone has experience importing MySQL database through Logstash to ElasticSearch? I have issue with the default mapping that LogStash provides. It always convert date to string and Integer to Long. For … --- ## [Filter issue with logstash beta 2.0](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662)
**Author:** [@boeriksencom](https://discuss.elastic.co/u/boeriksencom)\ **Replies:** 8\ **Last updated:** [October 26, 2015, 12:21pm UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662 "2015-10-26T12:21:42Z")
Hi, I've just installed elaticsearch 2.0.0-rc1, kibana 4.2.0-beta2 and logstash 2.0.0-beta2. I have a logstash config that picks up data from at txt file: Textfile example data: 1;127.0.0.1;2014-06-22T12:32:35 Logs… --- ## [Dell Powerconnect remote logging setup](https://discuss.elastic.co/t/dell-powerconnect-remote-logging-setup/31349)
**Author:** [@cotcadaver](https://discuss.elastic.co/u/cotcadaver)\ **Replies:** 10\ **Last updated:** [October 26, 2015, 10:49am UTC](https://discuss.elastic.co/t/dell-powerconnect-remote-logging-setup/31349 "2015-10-26T10:49:43Z")
Has anyone used ELK to setup a log server for an Dell Powerconnect stack? I'm at a loss for what to do next. I used this article to set up the server and have the filter (I believe) set up for the logs, however not sure … --- ## [Converting time into @timestamp](https://discuss.elastic.co/t/converting-time-into-timestamp/32846)
**Author:** [@zappe](https://discuss.elastic.co/u/zappe)\ **Replies:** 8\ **Last updated:** [October 26, 2015, 9:10am UTC](https://discuss.elastic.co/t/converting-time-into-timestamp/32846 "2015-10-26T09:10:08Z")
I'm trying to get my time into @timestamp but are failing. Lines looks like this "20151012 08:57:43 ..." And I use the following config: grok { match =\> { "message" =\> "(?\\d{1,8}.\\d{1,2}\[:\]?\\d{1,2}\[:\]… --- ## [Logstash, Kibana in GCP?](https://discuss.elastic.co/t/logstash-kibana-in-gcp/32848)
**Author:** [@kumarhanu](https://discuss.elastic.co/u/kumarhanu)\ **Replies:** 2\ **Last updated:** [October 25, 2015, 10:44am UTC](https://discuss.elastic.co/t/logstash-kibana-in-gcp/32848 "2015-10-25T10:44:44Z")
I know that Elasticsearch can be deployed in Google Cloud Platform. Can Logstash and Kibana deployed in GCP? --- ## [The error reported is: - logstash error](https://discuss.elastic.co/t/the-error-reported-is-logstash-error/32945)
**Author:** [@tushar.hadoop](https://discuss.elastic.co/u/tushar.hadoop)\ **Replies:** 3\ **Last updated:** [October 26, 2015, 5:19am UTC](https://discuss.elastic.co/t/the-error-reported-is-logstash-error/32945 "2015-10-26T05:19:29Z")
i am facing this issue for logstash 1.5.4 this is my configuration output { rabbitmq { key =\> "ambari-key" exchange =\> "ambari-exchange-type" exchange\_type =\> "direct" user =\> "rabbit" … --- ## [Logstash terminate if using sqlite plugin](https://discuss.elastic.co/t/logstash-terminate-if-using-sqlite-plugin/32432)
**Author:** [@M.Tyler](https://discuss.elastic.co/u/M.Tyler)\ **Replies:** 5\ **Last updated:** [October 26, 2015, 12:58am UTC](https://discuss.elastic.co/t/logstash-terminate-if-using-sqlite-plugin/32432 "2015-10-26T00:58:40Z")
Please let me question about sqlite plugin. Logstash version: 1.5.4 OS: Windows Vista I set logstash.conf as the below on input section after I ran "bin/plugin install logstash-input-sqlite". But Logstash terminate i… --- ## [Is there a systemd units file available for logstash 1.5 on centOS 7](https://discuss.elastic.co/t/is-there-a-systemd-units-file-available-for-logstash-1-5-on-centos-7/32937)
**Author:** [@JamesNJ](https://discuss.elastic.co/u/JamesNJ)\ **Replies:** 1\ **Last updated:** [October 25, 2015, 11:49pm UTC](https://discuss.elastic.co/t/is-there-a-systemd-units-file-available-for-logstash-1-5-on-centos-7/32937 "2015-10-25T23:49:35Z")
I loaded my CentOS 7 logstash 1.5 via yum from: http://packages.elasticsearch.org/logstash/1.5/centos However it doesn't seem to come with a systemd units file (elasticsearch has one). There is an /etc/init.d/logstash… --- ## [Representing text literals in match statement](https://discuss.elastic.co/t/representing-text-literals-in-match-statement/32623)
**Author:** [@bluethundr](https://discuss.elastic.co/u/bluethundr)\ **Replies:** 2\ **Last updated:** [October 25, 2015, 6:41pm UTC](https://discuss.elastic.co/t/representing-text-literals-in-match-statement/32623 "2015-10-25T18:41:42Z")
Guys, I'm working on a filter to match this log entry: 491 \<14\>1 2015-10-20T20:17:42.968706+00:00 loggregator 74cb190b-7ae2-4f25-8954-c18a0fd3ca37 \[App/0\] - - \\u001b\[1m\\u001b\[36mSQL (0.9ms)\\u001b\[0m \\u001b\[1mUPDAT… --- ## [Maximum and minimum @timestamp value](https://discuss.elastic.co/t/maximum-and-minimum-timestamp-value/24857)
**Author:** [@Siddhant\_Agarwal](https://discuss.elastic.co/u/Siddhant_Agarwal)\ **Replies:** 1\ **Last updated:** [October 25, 2015, 6:35pm UTC](https://discuss.elastic.co/t/maximum-and-minimum-timestamp-value/24857 "2015-10-25T18:35:41Z")
Is there a way to convert @timestamp from iso8601 to unix seconds in log stash so that i can use timer in metrics function? I want minimum and maximum @timestamp value. Also it would work if I know the first and last tim… --- ## [Is there a logstash sequence number solution?](https://discuss.elastic.co/t/is-there-a-logstash-sequence-number-solution/32934)
**Author:** [@abierbaum](https://discuss.elastic.co/u/abierbaum)\ **Replies:** 0\ **Last updated:** [October 25, 2015, 3:13pm UTC](https://discuss.elastic.co/t/is-there-a-logstash-sequence-number-solution/32934 "2015-10-25T15:13:39Z")
I am running into the issue where logstash (and there by Kibana) don't know the order of events that happened on the same millisecond even though there was an initial order at the event source. There have been discussi… --- ## [Multiple Date fields in logstash](https://discuss.elastic.co/t/multiple-date-fields-in-logstash/25213)
**Author:** [@sanju1323](https://discuss.elastic.co/u/sanju1323)\ **Replies:** 8\ **Last updated:** [October 25, 2015, 9:34am UTC](https://discuss.elastic.co/t/multiple-date-fields-in-logstash/25213 "2015-10-25T09:34:35Z")
Hi, I have a small query regarding date field. In my data file there are two date fields like submit\_date : 2012-12-22 12:53:30.000 release\_date: 2012-12-31 12:53:30.000 for this I tried using the following in the… --- ## [@timestamp not matching date parsed, apache logs](https://discuss.elastic.co/t/timestamp-not-matching-date-parsed-apache-logs/32912)
**Author:** [@JamesNJ](https://discuss.elastic.co/u/JamesNJ)\ **Replies:** 2\ **Last updated:** [October 24, 2015, 2:08pm UTC](https://discuss.elastic.co/t/timestamp-not-matching-date-parsed-apache-logs/32912 "2015-10-24T14:08:37Z")
Hello all, I'm new to log stash and probably this is a simple mistake on my part. I have a pile of old apache combined logs I want to put into elasticsearch, using logstash. I found that logstash was was not properly tra… --- ## [Converting date to YYYY-MM-DD from YYYY/MM/DD HH:MM:SS](https://discuss.elastic.co/t/converting-date-to-yyyy-mm-dd-from-yyyy-mm-dd-hh-mm-ss/31689)
**Author:** [@Deb](https://discuss.elastic.co/u/Deb)\ **Replies:** 3\ **Last updated:** [October 24, 2015, 7:16am UTC](https://discuss.elastic.co/t/converting-date-to-yyyy-mm-dd-from-yyyy-mm-dd-hh-mm-ss/31689 "2015-10-24T07:16:22Z")
Hi, I am having nginx error logs of the form:- 2015/09/30 22:19:38 \[error\] 32317#0: \*23 \[lua\] responses.lua:61: handler(): Cassandra error: Error during UNIQUE check: Cassandra error: connection refused, client: 127.0… --- ## [Logstash tcp input/output](https://discuss.elastic.co/t/logstash-tcp-input-output/32901)
**Author:** [@treehouse](https://discuss.elastic.co/u/treehouse)\ **Replies:** 0\ **Last updated:** [October 24, 2015, 5:42am UTC](https://discuss.elastic.co/t/logstash-tcp-input-output/32901 "2015-10-24T05:42:10Z")
Hey guys I know there are a couple other questions like this, but I haven't seen behavior like I am seeing surfaced before. I have two logstash instances, A and B. Here are my configs: A -- input { stdin {} } filt… --- ## [Logstash XML Parsing Using multiline](https://discuss.elastic.co/t/logstash-xml-parsing-using-multiline/32889)
**Author:** [@samnik60](https://discuss.elastic.co/u/samnik60)\ **Replies:** 0\ **Last updated:** [October 23, 2015, 9:52pm UTC](https://discuss.elastic.co/t/logstash-xml-parsing-using-multiline/32889 "2015-10-23T21:52:50Z")
I am trying to parse an xml file as shown below http://pastebin.com/SgX5gYY2 i tried to use the multiline filter of logstash as multiline { pattern =\> ".\*\" negate =\> true what =\> "previous" } I… --- ## [Geoip's dotted fields and ES2.0](https://discuss.elastic.co/t/geoips-dotted-fields-and-es2-0/32874)
**Author:** [@renevdm](https://discuss.elastic.co/u/renevdm)\ **Replies:** 2\ **Last updated:** [October 23, 2015, 8:30pm UTC](https://discuss.elastic.co/t/geoips-dotted-fields-and-es2-0/32874 "2015-10-23T20:30:58Z")
In the Logstash 2.0.0-rc1 released blog article I read: "Elasticsearch does not allow field names to have dots, beginning with version 2.0". So how about geoip? It adds dotted fields by default. I can imagine the plu… --- ## [Logstash process hanging](https://discuss.elastic.co/t/logstash-process-hanging/32883)
**Author:** [@mihir\_ray](https://discuss.elastic.co/u/mihir_ray)\ **Replies:** 0\ **Last updated:** [October 23, 2015, 7:56pm UTC](https://discuss.elastic.co/t/logstash-process-hanging/32883 "2015-10-23T19:56:18Z")
HI, My input is kafka and output is a file. Sometimes i have seen the logstash process just hangs(doing nothing). To debug this issue i enabled verbose logging and started the process. After few days of running, it h… --- ## [MasterNotDiscoveredException with RHEL 7 Firewall on LS 1.4.2 & ES 1.2.1](https://discuss.elastic.co/t/masternotdiscoveredexception-with-rhel-7-firewall-on-ls-1-4-2-es-1-2-1/32568)
**Author:** [@mdconner](https://discuss.elastic.co/u/mdconner)\ **Replies:** 8\ **Last updated:** [October 23, 2015, 5:39pm UTC](https://discuss.elastic.co/t/masternotdiscoveredexception-with-rhel-7-firewall-on-ls-1-4-2-es-1-2-1/32568 "2015-10-23T17:39:14Z")
After opening ports 9200 and 9300-9400 for TCP & UDP, I get this error from Logstash. I am running both on the same Red Hat Enterprise Linux 7.0 server and can successfully log data when the firewall (firewalld) is disab… --- ## [How to conditionally set the type?](https://discuss.elastic.co/t/how-to-conditionally-set-the-type/32862)
**Author:** [@WoJ](https://discuss.elastic.co/u/WoJ)\ **Replies:** 5\ **Last updated:** [October 23, 2015, 5:24pm UTC](https://discuss.elastic.co/t/how-to-conditionally-set-the-type/32862 "2015-10-23T17:24:31Z")
Hello I am new to logstash (using logstash 2.0 beta) so please bear with me. I have log files in a JSON format similar to {"TAGS":".source.s\_src","SOURCEIP":"127.0.0.1","SEQNUM":"2","PROGRAM":"logstash","PRIORITY":"inf… --- ## [Logstash with docker: unknown setting 'protocol', 'host' for elasticsearch](https://discuss.elastic.co/t/logstash-with-docker-unknown-setting-protocol-host-for-elasticsearch/32815)
**Author:** [@elastiguy](https://discuss.elastic.co/u/elastiguy)\ **Replies:** 4\ **Last updated:** [October 23, 2015, 4:04pm UTC](https://discuss.elastic.co/t/logstash-with-docker-unknown-setting-protocol-host-for-elasticsearch/32815 "2015-10-23T16:04:50Z")
Hello, I have a strange problem when using the official logstash image version 2.0. I am testing a basic logstash to ES configuration using the example from the documentation. I run Logstash via docker and everythin… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1226) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1228)