# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1201 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1202 --- ## [Where do we put logstash template?](https://discuss.elastic.co/t/where-do-we-put-logstash-template/39408)
**Author:** [@MaxOrelus](https://discuss.elastic.co/u/MaxOrelus)\ **Replies:** 2\ **Last updated:** [January 18, 2016, 3:51am UTC](https://discuss.elastic.co/t/where-do-we-put-logstash-template/39408 "2016-01-18T03:51:50Z")
I'm somewhat new to the ELK stack and I'm currently putting together my dev environment so I can get familiar with the different products. I already have gone through the process of setting up ElasticSearch, Logstash, an… --- ## [Rpm install logstash but while start it error!](https://discuss.elastic.co/t/rpm-install-logstash-but-while-start-it-error/39316)
**Author:** [@clifford](https://discuss.elastic.co/u/clifford)\ **Replies:** 3\ **Last updated:** [January 18, 2016, 3:21am UTC](https://discuss.elastic.co/t/rpm-install-logstash-but-while-start-it-error/39316 "2016-01-18T03:21:56Z")
\[root@ESK01 logstash\]# ./bin/plugin install logstash-input-beats Validating logstash-input-beats ERROR: Installation aborted, verification failed for logstash-input-beats start logstash that log report error: \[root@ESK01… --- ## [Record originating IP of filebeats](https://discuss.elastic.co/t/record-originating-ip-of-filebeats/39151)
**Author:** [@tun0](https://discuss.elastic.co/u/tun0)\ **Replies:** 3\ **Last updated:** [January 17, 2016, 5:02pm UTC](https://discuss.elastic.co/t/record-originating-ip-of-filebeats/39151 "2016-01-17T17:02:37Z")
I currently have a fairly simple setup consisting of running several filebeat instances on various hosts, talking to a central logstash instance, which in turn talks to a ES cluster (6 nodes total, 1 also runs logstash).… --- ## [Multiple timezones in central syslog](https://discuss.elastic.co/t/multiple-timezones-in-central-syslog/39345)
**Author:** [@nnet](https://discuss.elastic.co/u/nnet)\ **Replies:** 1\ **Last updated:** [January 17, 2016, 4:49pm UTC](https://discuss.elastic.co/t/multiple-timezones-in-central-syslog/39345 "2016-01-17T16:49:38Z")
Hi, my centralized syslog server is in timezone EST, logstash reads /var/log/messages (file input) on this server. Some of the servers sending syslog data are in timezone CST. If I do any searches 1 hour or less old, I d… --- ## [Duplicate logs - logstash-input-s3?](https://discuss.elastic.co/t/duplicate-logs-logstash-input-s3/39231)
**Author:** [@plonka2000](https://discuss.elastic.co/u/plonka2000)\ **Replies:** 8\ **Last updated:** [January 17, 2016, 4:27pm UTC](https://discuss.elastic.co/t/duplicate-logs-logstash-input-s3/39231 "2016-01-17T16:27:32Z")
Hi all, I'm using LS2.1.1 with ES2.1.1, in AWS, to log various sources within my environment. One of these sources is AWS CloudTrail. I've noticed a strange problem when trying to optimise my elasticsearch cluster, by… --- ## [Indexing structured documents using Logstash JDBC plugin](https://discuss.elastic.co/t/indexing-structured-documents-using-logstash-jdbc-plugin/39192)
**Author:** [@oshai](https://discuss.elastic.co/u/oshai)\ **Replies:** 3\ **Last updated:** [January 17, 2016, 4:24pm UTC](https://discuss.elastic.co/t/indexing-structured-documents-using-logstash-jdbc-plugin/39192 "2016-01-17T16:24:27Z")
Hi, I'm new to Logstash and i have a simple scenario that I'm trying to execute. Using the Logstash JDBC plugin I'm fetching data from MySQL and Indexing it in ES. Before using Logstash I've used the elasticsearch … --- ## [Timeout error in elasticsearch input plugin](https://discuss.elastic.co/t/timeout-error-in-elasticsearch-input-plugin/39396)
**Author:** [@samuelbr](https://discuss.elastic.co/u/samuelbr)\ **Replies:** 0\ **Last updated:** [January 17, 2016, 10:15am UTC](https://discuss.elastic.co/t/timeout-error-in-elasticsearch-input-plugin/39396 "2016-01-17T10:15:19Z")
Hi All. It's way how to set timeout for Elasticsearch input plugin in Logstash? Because when I start Logstash I got this error: ... Error: Timeout::Error", :level=\>:error} My Logstash config looks like: a input { e… --- ## [Failed to extract detail information from Windows Event log](https://discuss.elastic.co/t/failed-to-extract-detail-information-from-windows-event-log/38933)
**Author:** [@DerekTzeng](https://discuss.elastic.co/u/DerekTzeng)\ **Replies:** 2\ **Last updated:** [January 16, 2016, 1:52pm UTC](https://discuss.elastic.co/t/failed-to-extract-detail-information-from-windows-event-log/38933 "2016-01-16T13:52:16Z")
I was studying to retrieve some detailed information from Windows Event to find out logon failure records. Those detailed information is located in field "message" or "InsertionStirng".i.e. Account Domain, Account Name a… --- ## [Log Enrichment with Logstash and the Elastic Filter Plugin - How to Handle Ordering / Relationships?](https://discuss.elastic.co/t/log-enrichment-with-logstash-and-the-elastic-filter-plugin-how-to-handle-ordering-relationships/36822)
**Author:** [@Craig\_Roberts](https://discuss.elastic.co/u/Craig_Roberts)\ **Replies:** 6\ **Last updated:** [January 16, 2016, 11:16am UTC](https://discuss.elastic.co/t/log-enrichment-with-logstash-and-the-elastic-filter-plugin-how-to-handle-ordering-relationships/36822 "2016-01-16T11:16:33Z")
Howdy, We're building an analytics system, most likely using the ELK stack from start to finish, and while we understand the document-oriented nature of Elastic, we're having some headaches figuring out how to get that… --- ## [Arbitrary non standard log file. need help with grok pattern](https://discuss.elastic.co/t/arbitrary-non-standard-log-file-need-help-with-grok-pattern/39121)
**Author:** [@sharath3185](https://discuss.elastic.co/u/sharath3185)\ **Replies:** 2\ **Last updated:** [January 16, 2016, 10:54am UTC](https://discuss.elastic.co/t/arbitrary-non-standard-log-file-need-help-with-grok-pattern/39121 "2016-01-16T10:54:47Z")
Hi, I have a unstructured log file which does not have a similar pattern as below SOMEWORD: (@aaa.py:1222) Started test case 123 execution ABC\_TIME: started script aaa.py at \[20151202 17:10:28\] SOMEWORD: (@abc.pd)… --- ## [Error Flushing Buffer at interval on Output plugin Elasticsearch](https://discuss.elastic.co/t/error-flushing-buffer-at-interval-on-output-plugin-elasticsearch/39373)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 0\ **Last updated:** [January 16, 2016, 9:00am UTC](https://discuss.elastic.co/t/error-flushing-buffer-at-interval-on-output-plugin-elasticsearch/39373 "2016-01-16T09:00:11Z")
I using message queue RabbitMQ, i have 3 LS Indexer. (LS version 2.1.1 (8GB + 8Core), ES version 2.1.1 run with 3 Node (32GB RAM+16Core / node) ) I start 2 LS Indexer normal and LS-Indexer-3, run with --debug adn i got … --- ## [Double dereference variable](https://discuss.elastic.co/t/double-dereference-variable/39370)
**Author:** [@CamJN](https://discuss.elastic.co/u/CamJN)\ **Replies:** 0\ **Last updated:** [January 16, 2016, 6:50am UTC](https://discuss.elastic.co/t/double-dereference-variable/39370 "2016-01-16T06:50:37Z")
Is there a faster way to do the following? ruby { code =\> "event\['value'\] = event\[event\['column'\]\]" } --- ## [Horizontally scaling Logstash - Per User Guide](https://discuss.elastic.co/t/horizontally-scaling-logstash-per-user-guide/39340)
**Author:** [@jt1522](https://discuss.elastic.co/u/jt1522)\ **Replies:** 1\ **Last updated:** [January 16, 2016, 1:16am UTC](https://discuss.elastic.co/t/horizontally-scaling-logstash-per-user-guide/39340 "2016-01-16T01:16:09Z")
I have been working with the ELK stack for quite some time, but now that I am going to production I need to scale Logstash for larger throughput. I have seen the guide which shows the distributed layout: https://www.e… --- ## [Connection refused error while connecting to AWS from logstash](https://discuss.elastic.co/t/connection-refused-error-while-connecting-to-aws-from-logstash/39363)
**Author:** [@subbu.nv](https://discuss.elastic.co/u/subbu.nv)\ **Replies:** 2\ **Last updated:** [January 16, 2016, 12:56am UTC](https://discuss.elastic.co/t/connection-refused-error-while-connecting-to-aws-from-logstash/39363 "2016-01-16T00:56:38Z")
Hi All, I am connecting to AWS logs from the logstash in unix machine , it gives the below error. Connection refused {:class=\>"Manticore::SocketException", :level=\>:error} but when i tried from my windows machine, … --- ## [Can't merge a non object mapping with an object mapping](https://discuss.elastic.co/t/cant-merge-a-non-object-mapping-with-an-object-mapping/37374)
**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Replies:** 5\ **Last updated:** [January 15, 2016, 8:23pm UTC](https://discuss.elastic.co/t/cant-merge-a-non-object-mapping-with-an-object-mapping/37374 "2016-01-15T20:23:26Z")
I'm ingesting an XML document and getting the following error: Merging dynamic updates triggered a conflict: Can't merge a non object mapping \[doc.a.b\] with an object mapping \[doc.a.b\] XML: \ \ \c\ … --- ## [File input yields no stdout output](https://discuss.elastic.co/t/file-input-yields-no-stdout-output/39156)
**Author:** [@CamJN](https://discuss.elastic.co/u/CamJN)\ **Replies:** 4\ **Last updated:** [January 15, 2016, 3:24pm UTC](https://discuss.elastic.co/t/file-input-yields-no-stdout-output/39156 "2016-01-15T15:24:51Z")
Continuing the discussion from New to logstash: file input and stdout output not working: I have the same problem except that deleting the sincedb file, and setting the sincedb\_path to /dev/null and rerunning still doe… --- ## [File input not working on OSX](https://discuss.elastic.co/t/file-input-not-working-on-osx/37470)
**Author:** [@schmorgs](https://discuss.elastic.co/u/schmorgs)\ **Replies:** 4\ **Last updated:** [January 15, 2016, 1:42pm UTC](https://discuss.elastic.co/t/file-input-not-working-on-osx/37470 "2016-01-15T13:42:49Z")
I'm using the file input on my mac, and every time I update the file, it's replaying all messages from the beginning of the log. I've tried explicitly setting the sincedb\_path and start\_position vars, but no luck. .s… --- ## [Logstash input exec](https://discuss.elastic.co/t/logstash-input-exec/39312)
**Author:** [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Replies:** 1\ **Last updated:** [January 15, 2016, 9:19am UTC](https://discuss.elastic.co/t/logstash-input-exec/39312 "2016-01-15T09:19:06Z")
Hi all, i have an input config which launch the exec plugin, the program launched generates json data output , how can I put them in a index on ES? thank you, sorry for my English --- ## [Logstash capture error message](https://discuss.elastic.co/t/logstash-capture-error-message/38966)
**Author:** [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Replies:** 8\ **Last updated:** [January 15, 2016, 8:46am UTC](https://discuss.elastic.co/t/logstash-capture-error-message/38966 "2016-01-15T08:46:14Z")
Hi all, I'm trying to capture the error message that is generated for a \_dateparsefailure. an example of message is this: Failed parsing date from field {:field=\>"Data/Ora Chiusura TT", :value=\>"07/10/2015 11.04.12… --- ## [Logstash not getting any syslog](https://discuss.elastic.co/t/logstash-not-getting-any-syslog/38631)
**Author:** [@simroo](https://discuss.elastic.co/u/simroo)\ **Replies:** 10\ **Last updated:** [January 15, 2016, 7:58am UTC](https://discuss.elastic.co/t/logstash-not-getting-any-syslog/38631 "2016-01-15T07:58:52Z")
Hello I have just installed ELK on my Linux Debian, and i can access kibana. But i just seem to get No results found :(. I have config a fortigate firewall and juniper switch to syslog everything to the server. But sti… --- ## [Regarding Multiline codec](https://discuss.elastic.co/t/regarding-multiline-codec/39291)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 2\ **Last updated:** [January 15, 2016, 7:10am UTC](https://discuss.elastic.co/t/regarding-multiline-codec/39291 "2016-01-15T07:10:22Z")
Hi, If I am using the multine codec on an input which handles multiple input sources can it be configured to handle the various multiline formats of each source? This is desirable behaviour to avoid pattern matching agai… --- ## [Does Log4J input block sender?](https://discuss.elastic.co/t/does-log4j-input-block-sender/39261)
**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Replies:** 1\ **Last updated:** [January 15, 2016, 5:41am UTC](https://discuss.elastic.co/t/does-log4j-input-block-sender/39261 "2016-01-15T05:41:04Z")
I'm interested in the Log4J Logstash input plugin but does it block the socketappender during heavy loads, etc.? --- ## [How to scale input data when use logstash](https://discuss.elastic.co/t/how-to-scale-input-data-when-use-logstash/39199)
**Author:** [@NgocPham](https://discuss.elastic.co/u/NgocPham)\ **Replies:** 3\ **Last updated:** [January 14, 2016, 6:29pm UTC](https://discuss.elastic.co/t/how-to-scale-input-data-when-use-logstash/39199 "2016-01-14T18:29:26Z")
Hi . Im newbie in elasticsearch and logstash. And I have some question . Please help me \[:slightly\_smiling:\] Im trying input data from postgres database into elasticsearch. I config and it running ok. But now i want sc… --- ## [Sampling or filtering varnish logs](https://discuss.elastic.co/t/sampling-or-filtering-varnish-logs/39252)
**Author:** [@fchantrel](https://discuss.elastic.co/u/fchantrel)\ **Replies:** 0\ **Last updated:** [January 14, 2016, 5:11pm UTC](https://discuss.elastic.co/t/sampling-or-filtering-varnish-logs/39252 "2016-01-14T17:11:56Z")
Hello, I'd like to index a part of my varnish logs to spend less disk space ; is it possible by filtering on a time field or by sampling (perhaps a random in a ruby script ?) ? I have two fields with a time : \[14/Jan/201… --- ## [Parse logs](https://discuss.elastic.co/t/parse-logs/29225)
**Author:** [@silviupanaite](https://discuss.elastic.co/u/silviupanaite)\ **Replies:** 11\ **Last updated:** [January 14, 2016, 3:49pm UTC](https://discuss.elastic.co/t/parse-logs/29225 "2016-01-14T15:49:25Z")
I'm trying to parse: {"log":"86.121.176.201 - - \[09/Sep/2015:14:14:06 +0000\] \\"GET /css/style.less HTTP/1.1\\" 304 0 \\"http://developer.unificationengine.com/login\\" \\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10\_10\_5) Apple… --- ## [How to remove a xml attribute in xml filter?](https://discuss.elastic.co/t/how-to-remove-a-xml-attribute-in-xml-filter/38959)
**Author:** [@Sudeesh](https://discuss.elastic.co/u/Sudeesh)\ **Replies:** 4\ **Last updated:** [January 14, 2016, 9:45am UTC](https://discuss.elastic.co/t/how-to-remove-a-xml-attribute-in-xml-filter/38959 "2016-01-14T09:45:52Z")
XML File: i want to remove the tag test-method which contains attribute is-config=true. This tag is repeated. I only want to get the value of the name=testcaseName from the tag test-method. Conf file: --- ## [Index creation at 00:00 GMT+2](https://discuss.elastic.co/t/index-creation-at-00-00-gmt-2/39198)
**Author:** [@nkoleff](https://discuss.elastic.co/u/nkoleff)\ **Replies:** 1\ **Last updated:** [January 14, 2016, 8:52am UTC](https://discuss.elastic.co/t/index-creation-at-00-00-gmt-2/39198 "2016-01-14T08:52:15Z")
Hello I've write a custom script that is monitoring my logstashes (3 machines at the moment) - each minute I am writing to my daily index an OK message. If my script detects that there are less than 9 entries for the l… --- ## [How to config Logstash input from MongoDB](https://discuss.elastic.co/t/how-to-config-logstash-input-from-mongodb/39178)
**Author:** [@tctr715](https://discuss.elastic.co/u/tctr715)\ **Replies:** 6\ **Last updated:** [January 14, 2016, 7:42am UTC](https://discuss.elastic.co/t/how-to-config-logstash-input-from-mongodb/39178 "2016-01-14T07:42:14Z")
Is this OK？ input { mongodb { database =\> "my\_db" uri =\> "mongodb://szxtsv12066mgd:27017" codec =\> json } } output { elasticsearch { hosts =\> \["127.0.0.1:9201","127.0.0.1:9202"\] } } --- ## [Elastic Search and Logstash for SNMP](https://discuss.elastic.co/t/elastic-search-and-logstash-for-snmp/39089)
**Author:** [@Jansi](https://discuss.elastic.co/u/Jansi)\ **Replies:** 3\ **Last updated:** [January 14, 2016, 7:24am UTC](https://discuss.elastic.co/t/elastic-search-and-logstash-for-snmp/39089 "2016-01-14T07:24:23Z")
Hi, I have collected the SNMP from router in Cent OS using Logstash also imported those into ES. Now I need a clarification on this. I have to separate MIB name and value from received message. also have to do som… --- ## [What is the need of logstash when we can directly forward things to elastic search](https://discuss.elastic.co/t/what-is-the-need-of-logstash-when-we-can-directly-forward-things-to-elastic-search/39067)
**Author:** [@elastic-fan](https://discuss.elastic.co/u/elastic-fan)\ **Replies:** 2\ **Last updated:** [January 14, 2016, 6:05am UTC](https://discuss.elastic.co/t/what-is-the-need-of-logstash-when-we-can-directly-forward-things-to-elastic-search/39067 "2016-01-14T06:05:16Z")
What is the need of logstash when we can directly forward things to elastic search, is there any specific reason that we want to use logstash? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1200) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1202)