# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1091 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1092 --- ## [Logstash not sending logs to elastic search](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic-search/63120)
**Author:** [@red888](https://discuss.elastic.co/u/red888)\ **Replies:** 4\ **Last updated:** [October 16, 2016, 3:44pm UTC](https://discuss.elastic.co/t/logstash-not-sending-logs-to-elastic-search/63120 "2016-10-16T15:44:02Z")
CentOS Linux release 7.2.1511 (Core) Using the latest Logstash and ES versions: Name : elasticsearch Arch : noarch Version : 2.4.0 Release : 1 Name : logstash Arch : noarch Epoch … --- ## [Update document id issue](https://discuss.elastic.co/t/update-document-id-issue/62643)
**Author:** [@ggp](https://discuss.elastic.co/u/ggp)\ **Replies:** 1\ **Last updated:** [October 16, 2016, 3:17pm UTC](https://discuss.elastic.co/t/update-document-id-issue/62643 "2016-10-16T15:17:50Z")
I am trying to update one of the document id's "name" field value and don't see the document getting updated, any inputs would be very helpful. Here is the log stash config i have input { jdbc { # Postgres jdbc conne… --- ## [Logstash - Reading from Kafka and Inserting into ElasticSearch cluster - kafka topic lag not going down](https://discuss.elastic.co/t/logstash-reading-from-kafka-and-inserting-into-elasticsearch-cluster-kafka-topic-lag-not-going-down/62072)
**Author:** [@narendra.agashe](https://discuss.elastic.co/u/narendra.agashe)\ **Replies:** 4\ **Last updated:** [October 15, 2016, 4:36pm UTC](https://discuss.elastic.co/t/logstash-reading-from-kafka-and-inserting-into-elasticsearch-cluster-kafka-topic-lag-not-going-down/62072 "2016-10-15T16:36:35Z")
Team, My ELK stack is as follows ES Version :- 2.3.5 (3 servers, 2 Master Eligible nodes and 1 Client, where Logstash is pointed to for inserting messages into ES) Kafka :- 0.8 (Cluster of 3) topic has 3 partitions. Lo… --- ## [Logstash ability to check field value against file contents](https://discuss.elastic.co/t/logstash-ability-to-check-field-value-against-file-contents/63113)
**Author:** [@Justin\_Henderson](https://discuss.elastic.co/u/Justin_Henderson)\ **Replies:** 1\ **Last updated:** [October 15, 2016, 3:52pm UTC](https://discuss.elastic.co/t/logstash-ability-to-check-field-value-against-file-contents/63113 "2016-10-15T15:52:56Z")
Does anyone know of a plugin that would allow checking a field's value against the contents of a file? I'm looking to do a simple boolean check and if true have the capability to add a tag or field. This would allow t… --- ## [Logstash service behaving different than command line](https://discuss.elastic.co/t/logstash-service-behaving-different-than-command-line/63104)
**Author:** [@chrisan](https://discuss.elastic.co/u/chrisan)\ **Replies:** 2\ **Last updated:** [October 15, 2016, 2:45pm UTC](https://discuss.elastic.co/t/logstash-service-behaving-different-than-command-line/63104 "2016-10-15T14:45:52Z")
Hello, I have 2 cloudfront distributions I am looking to analyze. The system is a fresh VM of Ubuntu 16.04 with latest Elasticsearch and Logstash installed via apt-get I have the following 2 configs: In both cas… --- ## [Logstash consuming messages slow](https://discuss.elastic.co/t/logstash-consuming-messages-slow/56900)
**Author:** [@mrunalgosar](https://discuss.elastic.co/u/mrunalgosar)\ **Replies:** 13\ **Last updated:** [October 15, 2016, 2:34pm UTC](https://discuss.elastic.co/t/logstash-consuming-messages-slow/56900 "2016-10-15T14:34:17Z")
Hi All I have recently setup ELK stack as belows. Kibana \<-- Elasticsearch \<-- Logstash \<-- Redis \<-- Filebeat. Here my Filebeat is producing log events faster than Logstash can consume..resulting in pile up on redis.… --- ## [Fetching logs for logstash](https://discuss.elastic.co/t/fetching-logs-for-logstash/63023)
**Author:** [@Rajat\_Singh](https://discuss.elastic.co/u/Rajat_Singh)\ **Replies:** 2\ **Last updated:** [October 15, 2016, 9:05am UTC](https://discuss.elastic.co/t/fetching-logs-for-logstash/63023 "2016-10-15T09:05:37Z")
Hi folks , I need to regularly fetch the logs from the remote server and parse it in the logstash after every five minutes --- ## [No Output in logstash - Error - glob did not work](https://discuss.elastic.co/t/no-output-in-logstash-error-glob-did-not-work/62892)
**Author:** [@anujwalia](https://discuss.elastic.co/u/anujwalia)\ **Replies:** 1\ **Last updated:** [October 15, 2016, 5:40am UTC](https://discuss.elastic.co/t/no-output-in-logstash-error-glob-did-not-work/62892 "2016-10-15T05:40:26Z")
Hi, I have installed logstash on windows and trying to parse the logs using grok from an log file mentioned in the tutorial i.e. logstash-tutorial.log. On debug mode i see the below exception COmmand logstash -f "C:… --- ## [Logstash Doesn't Apply Elastic Template](https://discuss.elastic.co/t/logstash-doesnt-apply-elastic-template/63083)
**Author:** [@Mormaii](https://discuss.elastic.co/u/Mormaii)\ **Replies:** 1\ **Last updated:** [October 14, 2016, 8:55pm UTC](https://discuss.elastic.co/t/logstash-doesnt-apply-elastic-template/63083 "2016-10-14T20:55:17Z")
Logstash config: if \[application\] == "api\_xalok\_varnish" { elasticsearch { template =\> "/etc/logstash/templates/api\_xalok\_varnish.json" template\_name =\> "api\_xalok\_varnish" template\_overwri… --- ## [Logstash Intermittently Stalling](https://discuss.elastic.co/t/logstash-intermittently-stalling/62420)
**Author:** [@robinjoseph08](https://discuss.elastic.co/u/robinjoseph08)\ **Replies:** 1\ **Last updated:** [October 14, 2016, 5:44pm UTC](https://discuss.elastic.co/t/logstash-intermittently-stalling/62420 "2016-10-14T17:44:58Z")
For a while now, we've seen one of our Logstash instances periodically stalling, and it stops sending events through our outputs. It gets into a weird state where it seems like it's running just fine (sudo service logs… --- ## [Approach for receiving large amount of messages per second](https://discuss.elastic.co/t/approach-for-receiving-large-amount-of-messages-per-second/62969)
**Author:** [@Kissaki](https://discuss.elastic.co/u/Kissaki)\ **Replies:** 2\ **Last updated:** [October 14, 2016, 3:12pm UTC](https://discuss.elastic.co/t/approach-for-receiving-large-amount-of-messages-per-second/62969 "2016-10-14T15:12:44Z")
Hey, I try to set up logstash to receive a large number of UDP messages per second, up to may be 50k. Currently, for my test setup, logstash and elasticsearch is installed on my local machine (i5-4590S, 16gb RAM) and g… --- ## [Last line of CSV file not parsed/indexed](https://discuss.elastic.co/t/last-line-of-csv-file-not-parsed-indexed/62968)
**Author:** [@frederes](https://discuss.elastic.co/u/frederes)\ **Replies:** 2\ **Last updated:** [October 14, 2016, 2:10pm UTC](https://discuss.elastic.co/t/last-line-of-csv-file-not-parsed-indexed/62968 "2016-10-14T14:10:07Z")
Hello, I use logstash to index a lot of CSV files. I noticed that the last line of the files was not parsed. In fact, there is no carriage return after he last line and that is the problem. Is there a way to solve this w… --- ## [Delete a log file after indexing via logstash](https://discuss.elastic.co/t/delete-a-log-file-after-indexing-via-logstash/41239)
**Author:** [@subbu.nv](https://discuss.elastic.co/u/subbu.nv)\ **Replies:** 4\ **Last updated:** [October 14, 2016, 9:09am UTC](https://discuss.elastic.co/t/delete-a-log-file-after-indexing-via-logstash/41239 "2016-10-14T09:09:55Z")
Due to spacing constraint, i may have to delete the physical file once the indexing in done. is it possible to handle this with in the .conf file in logstash? Please help. thanks --- ## [Logstash-output Kafka: message load balancing](https://discuss.elastic.co/t/logstash-output-kafka-message-load-balancing/37804)
**Author:** [@tguan](https://discuss.elastic.co/u/tguan)\ **Replies:** 3\ **Last updated:** [October 14, 2016, 8:46am UTC](https://discuss.elastic.co/t/logstash-output-kafka-message-load-balancing/37804 "2016-10-14T08:46:50Z")
Is there any good example for specifying the message\_key for logstash Kafka output to balance the messages to be stored between different partitions on the kafka server side? I have 3 partitions in the topic, and running… --- ## [How to split xml arrays?](https://discuss.elastic.co/t/how-to-split-xml-arrays/51944)
**Author:** [@pazarr](https://discuss.elastic.co/u/pazarr)\ **Replies:** 14\ **Last updated:** [October 14, 2016, 8:35am UTC](https://discuss.elastic.co/t/how-to-split-xml-arrays/51944 "2016-10-14T08:35:55Z")
Hi Folks, I'm pretty new in ELK stack. I'm trying to parse xml and get the all elements from array send to elasticsearch. So I wrote this filter which works fine: filter { xml { source =\> "message" #ta… --- ## [Config file not getting read by logstash](https://discuss.elastic.co/t/config-file-not-getting-read-by-logstash/62630)
**Author:** [@adhikz](https://discuss.elastic.co/u/adhikz)\ **Replies:** 11\ **Last updated:** [October 14, 2016, 8:30am UTC](https://discuss.elastic.co/t/config-file-not-getting-read-by-logstash/62630 "2016-10-14T08:30:24Z")
Hi, I have set up Elk stack on my windows machine with the following : Elasticserach Logstash Kibana My logstash.conf' input { file { path =\> "\\bin\\MylogFile.log" start\_position =\> "beginning" } } output { elastic… --- ## [Any possibility of getting Single XML Tag](https://discuss.elastic.co/t/any-possibility-of-getting-single-xml-tag/62124)
**Author:** [@vinoossk](https://discuss.elastic.co/u/vinoossk)\ **Replies:** 3\ **Last updated:** [October 14, 2016, 4:13am UTC](https://discuss.elastic.co/t/any-possibility-of-getting-single-xml-tag/62124 "2016-10-14T04:13:15Z")
Is it possible to fetch single tag constantly like \ (irrespective of positions) across all XML's instead of iterating through XPATH ?? --- ## [How to send data to a http end point using LS output plugin](https://discuss.elastic.co/t/how-to-send-data-to-a-http-end-point-using-ls-output-plugin/58959)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 28\ **Last updated:** [October 14, 2016, 4:00am UTC](https://discuss.elastic.co/t/how-to-send-data-to-a-http-end-point-using-ls-output-plugin/58959 "2016-10-14T04:00:12Z")
Hi All I have a CSV file which contains three simple fields and i want post its value one after another to http endpoint using LS http output plugin The csv files contain 3 header : Name, Description, Age The end p… --- ## [Quick question on type](https://discuss.elastic.co/t/quick-question-on-type/62996)
**Author:** [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\ **Replies:** 2\ **Last updated:** [October 13, 2016, 9:55pm UTC](https://discuss.elastic.co/t/quick-question-on-type/62996 "2016-10-13T21:55:09Z")
So per 2.4.0 breaking changes type is no workie in filter/output now. I currently have a logstash config with this structure: input { file { type =\> "log" } } filter { if ( \[ty… --- ## [.NET NEST and logstash?](https://discuss.elastic.co/t/net-nest-and-logstash/62940)
**Author:** [@agonzalez](https://discuss.elastic.co/u/agonzalez)\ **Replies:** 3\ **Last updated:** [October 13, 2016, 9:46pm UTC](https://discuss.elastic.co/t/net-nest-and-logstash/62940 "2016-10-13T21:46:09Z")
is there any way to index documents using .NET NEST but using logstash as endpoint? All my clients are spread over internet and only should connect to logstash public server only, not directly to ES. --- ## [Failed parsing date from field - how to troubleshoot](https://discuss.elastic.co/t/failed-parsing-date-from-field-how-to-troubleshoot/62935)
**Author:** [@trondhindenes](https://discuss.elastic.co/u/trondhindenes)\ **Replies:** 1\ **Last updated:** [October 13, 2016, 4:43pm UTC](https://discuss.elastic.co/t/failed-parsing-date-from-field-how-to-troubleshoot/62935 "2016-10-13T16:43:30Z")
I'm having this weird problem where Logstash is throwing this error left and right: {:timestamp=\>"2016-10-13T12:08:24.357000+0000", :message=\>"Failed parsing date from field", :field=\>"log\_timestamp", :value=\>"%{Date} %… --- ## [Charset not considered on HTTP input plugin](https://discuss.elastic.co/t/charset-not-considered-on-http-input-plugin/62765)
**Author:** [@Javier\_Bravo\_Conde](https://discuss.elastic.co/u/Javier_Bravo_Conde)\ **Replies:** 2\ **Last updated:** [October 13, 2016, 4:39pm UTC](https://discuss.elastic.co/t/charset-not-considered-on-http-input-plugin/62765 "2016-10-13T16:39:45Z")
Hi! Before opening a ticket I would like to discuss my issue here. I receive a event.message string encoded in UTF-8 even though I specify codec =\> plain { charset =\> "ASCII-8BIT"} in my config. Here my config: inp… --- ## [Tracking how much logstash has read from a file or How much is its backlog](https://discuss.elastic.co/t/tracking-how-much-logstash-has-read-from-a-file-or-how-much-is-its-backlog/62962)
**Author:** [@sanilveeravu](https://discuss.elastic.co/u/sanilveeravu)\ **Replies:** 1\ **Last updated:** [October 13, 2016, 3:29pm UTC](https://discuss.elastic.co/t/tracking-how-much-logstash-has-read-from-a-file-or-how-much-is-its-backlog/62962 "2016-10-13T15:29:11Z")
Hi, I get a set of zipped log files from an external application. I have a process that would unzip the files so that logstash can forward them to elastic search. Just wanted help to know if there is anyway I can figur… --- ## [Unbounded elements in XML filter](https://discuss.elastic.co/t/unbounded-elements-in-xml-filter/62937)
**Author:** [@vinoossk](https://discuss.elastic.co/u/vinoossk)\ **Replies:** 0\ **Last updated:** [October 13, 2016, 12:41pm UTC](https://discuss.elastic.co/t/unbounded-elements-in-xml-filter/62937 "2016-10-13T12:41:39Z")
I am getting unbounded elements in below format and I want to parse specific tags?? (In the below DatasourceDetail may occur 'n' number of times). Like if DataSourceResponseDuration may occur or maynot occur. How to use… --- ## [Please help in Logstash Error](https://discuss.elastic.co/t/please-help-in-logstash-error/62926)
**Author:** [@amits](https://discuss.elastic.co/u/amits)\ **Replies:** 2\ **Last updated:** [October 13, 2016, 12:29pm UTC](https://discuss.elastic.co/t/please-help-in-logstash-error/62926 "2016-10-13T12:29:34Z")
Hi all, I am trying to read log files from other machine using Filebeat at my end in Logstash. When I am staring my logstash, it is throwing an ERROR as below: A plugin had an unrecoverable error. Will restart this p… --- ## [How to get rid of extra double quote](https://discuss.elastic.co/t/how-to-get-rid-of-extra-double-quote/62820)
**Author:** [@trondhindenes](https://discuss.elastic.co/u/trondhindenes)\ **Replies:** 2\ **Last updated:** [October 13, 2016, 12:07pm UTC](https://discuss.elastic.co/t/how-to-get-rid-of-extra-double-quote/62820 "2016-10-13T12:07:49Z")
I have this logstash filter I'm struggling with. Right now it comes out like this in rubydebug: { "type" =\> "log", "host" =\> "CY-C0D6E-006", "Date" =\> "2016-10-12", … --- ## [Log in wrong index](https://discuss.elastic.co/t/log-in-wrong-index/62740)
**Author:** [@meaglin](https://discuss.elastic.co/u/meaglin)\ **Replies:** 14\ **Last updated:** [October 13, 2016, 11:42am UTC](https://discuss.elastic.co/t/log-in-wrong-index/62740 "2016-10-13T11:42:53Z")
Hello, I've got a strange problem. I'm using logstash to get logs from syslog and ossec into elasticsearch to make it possible to search through either kibana or queries run directly on it. I have different index pa… --- ## [Logstash different index names per input?](https://discuss.elastic.co/t/logstash-different-index-names-per-input/62806)
**Author:** [@agonzalez](https://discuss.elastic.co/u/agonzalez)\ **Replies:** 6\ **Last updated:** [October 13, 2016, 11:19am UTC](https://discuss.elastic.co/t/logstash-different-index-names-per-input/62806 "2016-10-13T11:19:49Z")
I have this logstash config but beats documents get created with index "log-2016.10.12" that is the document\_tpye instead of filebeat-xxx. Any idea on how i can set the index name to filebeat and snmptrap index to snmpt… --- ## [Failed to send event to MongoDB - NoMethodError](https://discuss.elastic.co/t/failed-to-send-event-to-mongodb-nomethoderror/62912)
**Author:** [@rivaanbechan](https://discuss.elastic.co/u/rivaanbechan)\ **Replies:** 0\ **Last updated:** [October 13, 2016, 8:13am UTC](https://discuss.elastic.co/t/failed-to-send-event-to-mongodb-nomethoderror/62912 "2016-10-13T08:13:59Z")
Hi, I am able to write to a file or use other outputs. My problem is specific to the mongodb output use case. centos7 MongoDB shell version: 3.2.10 logstash-output-mongodb 2.0.5 winlogbeats 1.3.1 logstash 2.4.0 log… --- ## [Which input logstash plugin is the fastest?](https://discuss.elastic.co/t/which-input-logstash-plugin-is-the-fastest/62622)
**Author:** [@111126](https://discuss.elastic.co/u/111126)\ **Replies:** 12\ **Last updated:** [October 13, 2016, 6:45am UTC](https://discuss.elastic.co/t/which-input-logstash-plugin-is-the-fastest/62622 "2016-10-13T06:45:58Z")
Hello everybody. I need hight event rate to logstash - at least 50k, and I must also have a way of further scaling How I can forward 50k, 100k, 150k events per second? I have central rsyslog server that stores logs o… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1090) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1092)