# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1022 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1023 --- ## [Multiple server resource metrics stats](https://discuss.elastic.co/t/multiple-server-resource-metrics-stats/77288)
**Author:** [@Prateek\_Kshtriya](https://discuss.elastic.co/u/Prateek_Kshtriya)\ **Replies:** 5\ **Last updated:** [March 9, 2017, 11:54am UTC](https://discuss.elastic.co/t/multiple-server-resource-metrics-stats/77288 "2017-03-09T11:54:09Z")
Hi All , I am trying to get the resource metrics data from 5 machines forming a cluster to have a one view dashboard on kibana. There are five different config files are created for each individual machines and are in… --- ## [I cannot mutate \>\> convert multiple fields](https://discuss.elastic.co/t/i-cannot-mutate-convert-multiple-fields/1227)
**Author:** [@pescobar](https://discuss.elastic.co/u/pescobar)\ **Replies:** 4\ **Last updated:** [March 9, 2017, 10:29am UTC](https://discuss.elastic.co/t/i-cannot-mutate-convert-multiple-fields/1227 "2017-03-09T10:29:26Z")
Hi, I am trying to convert few of my fields to integer using logstash 1.5. When converting just one field it works fine doing this to convert my field "failed" to integer: mutate { convert =\> \[ "fai… --- ## [Ruby exception occurred: event\['field'\]) have been disabled](https://discuss.elastic.co/t/ruby-exception-occurred-event-field-have-been-disabled/76270)
**Author:** [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\ **Replies:** 5\ **Last updated:** [March 9, 2017, 10:10am UTC](https://discuss.elastic.co/t/ruby-exception-occurred-event-field-have-been-disabled/76270 "2017-03-09T10:10:15Z")
Hello, I use this pattern to convert hexadecimal fields to decimal fields : mutate { convert =\> { "hexa\_value" =\> "integer" } } ruby { code =\> 'event\["hexa\_value"\] = event\["hexa\_value"\].hex' … --- ## [Important delay in the appearance of logs in Kibana](https://discuss.elastic.co/t/important-delay-in-the-appearance-of-logs-in-kibana/77330)
**Author:** [@vdsm](https://discuss.elastic.co/u/vdsm)\ **Replies:** 7\ **Last updated:** [March 9, 2017, 8:08am UTC](https://discuss.elastic.co/t/important-delay-in-the-appearance-of-logs-in-kibana/77330 "2017-03-09T08:08:29Z")
Hi everyone, I see a problem about my ELK stack currently in pre-production, and I do not understand where it comes from ... When I view the logs in Kibana, these are systematically between 25 and 40 minutes behind the T… --- ## [Logstash not committing offset to Kafka](https://discuss.elastic.co/t/logstash-not-committing-offset-to-kafka/77944)
**Author:** [@Dan\_Markhasin](https://discuss.elastic.co/u/Dan_Markhasin)\ **Replies:** 0\ **Last updated:** [March 9, 2017, 8:02am UTC](https://discuss.elastic.co/t/logstash-not-committing-offset-to-kafka/77944 "2017-03-09T08:02:11Z")
We are using Logstash with Kafka input and ElasticSearch output, and noticed a strange behavior when ES is slow to respond (for example, when we are pushing multiple document updates). Kafka is constantly rebalancing … --- ## [Merge information from two different lines into single event](https://discuss.elastic.co/t/merge-information-from-two-different-lines-into-single-event/77791)
**Author:** [@Tarun\_Kalluri](https://discuss.elastic.co/u/Tarun_Kalluri)\ **Replies:** 2\ **Last updated:** [March 9, 2017, 7:07am UTC](https://discuss.elastic.co/t/merge-information-from-two-different-lines-into-single-event/77791 "2017-03-09T07:07:23Z")
I have a log file which has product id and item sale details on different lines. An example file can be useless info product id: 2455 useless info Customer Name: xyz1 Price : abc1 Profit: efg1 Customer Name: xyz2 Pric… --- ## [Unable to create the Index while using JDBC Plugin](https://discuss.elastic.co/t/unable-to-create-the-index-while-using-jdbc-plugin/77779)
**Author:** [@Thangaraj\_Muruganant](https://discuss.elastic.co/u/Thangaraj_Muruganant)\ **Replies:** 1\ **Last updated:** [March 9, 2017, 6:57am UTC](https://discuss.elastic.co/t/unable-to-create-the-index-while-using-jdbc-plugin/77779 "2017-03-09T06:57:53Z")
I am facing an Issue in Creating an Index. Meanwhile i was trying to achieve Incremental Load. here i have attached my config file. Here is my config file and the the issue displaying in cmd log, in cmd log, 12:37:31… --- ## [Logstash xml parsing](https://discuss.elastic.co/t/logstash-xml-parsing/77836)
**Author:** [@WildCard](https://discuss.elastic.co/u/WildCard)\ **Replies:** 1\ **Last updated:** [March 8, 2017, 11:26pm UTC](https://discuss.elastic.co/t/logstash-xml-parsing/77836 "2017-03-08T23:26:33Z")
Hi guys, I have a xml parsing problem... this is a very smal part of the xml (the xml's are about 500MB and a lot of different nested fields: \ \ **Author:** [@PMDubuc](https://discuss.elastic.co/u/PMDubuc)\ **Replies:** 3\ **Last updated:** [March 8, 2017, 10:42pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-for-parsing-nested-data/77732 "2017-03-08T22:42:06Z") I have log messages with field data delimited by \[ \] characters. The problem is that sometimes the data in the field contains those characters. How do I write the grok pattern to take the nesting level into account, ex… --- ## [ELK plugin for Azure eventhub](https://discuss.elastic.co/t/elk-plugin-for-azure-eventhub/77756)
**Author:** [@111148](https://discuss.elastic.co/u/111148)\ **Replies:** 2\ **Last updated:** [March 8, 2017, 10:21pm UTC](https://discuss.elastic.co/t/elk-plugin-for-azure-eventhub/77756 "2017-03-08T22:21:02Z")
Hello again , guys. I am still trying setup windows azure logs parsing in ELK. Still no luck. Tried azureblob plugin, and exec command variant. Both work with issues. Found 3-d variant, ( an article) that it can be done … --- ## [How can I tell Logstash's buffer size?](https://discuss.elastic.co/t/how-can-i-tell-logstashs-buffer-size/77907)
**Author:** [@ZillaG](https://discuss.elastic.co/u/ZillaG)\ **Replies:** 0\ **Last updated:** [March 8, 2017, 10:16pm UTC](https://discuss.elastic.co/t/how-can-i-tell-logstashs-buffer-size/77907 "2017-03-08T22:16:07Z")
I set a buffer\_size in my input plugin, and see in the logs that the value is actually being set. input { udp { port =\> 5555 buffer\_size = 65536 } } However, how can I really tell that Logstash is indeed us… --- ## [Use GeoIP2 database with geoip filter plugin](https://discuss.elastic.co/t/use-geoip2-database-with-geoip-filter-plugin/77826)
**Author:** [@jcannell](https://discuss.elastic.co/u/jcannell)\ **Replies:** 4\ **Last updated:** [March 8, 2017, 9:27pm UTC](https://discuss.elastic.co/t/use-geoip2-database-with-geoip-filter-plugin/77826 "2017-03-08T21:27:54Z")
Hello, We are thinking of purchasing a license for Maxmind's GeoIP2 database. I've learned that Logstash uses the free Geolite2 database from Maxmind that might be less accurate. Would it be possible to configure logst… --- ## [Can't get headers dropped from CSV](https://discuss.elastic.co/t/cant-get-headers-dropped-from-csv/77873)
**Author:** [@jcannell](https://discuss.elastic.co/u/jcannell)\ **Replies:** 2\ **Last updated:** [March 8, 2017, 8:25pm UTC](https://discuss.elastic.co/t/cant-get-headers-dropped-from-csv/77873 "2017-03-08T20:25:29Z")
Hi Everyone, Using logstash version 2.4.1 I'm having a lot of trouble getting the first row in my CSV dropped. My filter keeps picking it up and trying to convert it to a date. Here is the filter portion my config: f… --- ## [How to change date format when indexing](https://discuss.elastic.co/t/how-to-change-date-format-when-indexing/77847)
**Author:** [@varun1992](https://discuss.elastic.co/u/varun1992)\ **Replies:** 0\ **Last updated:** [March 8, 2017, 4:19pm UTC](https://discuss.elastic.co/t/how-to-change-date-format-when-indexing/77847 "2017-03-08T16:19:27Z")
I have 2 date field in a CSV data, formatted as ==\> MM/dd/YY HH:mm How can i change the format to ==\> YYYY/MM/dd HH:mm:ss when am indexing it using logstash ? I want to save the field as date in elastic, n… --- ## [Error when DATESTAMP pattern is used to parse the log\_timestamp](https://discuss.elastic.co/t/error-when-datestamp-pattern-is-used-to-parse-the-log-timestamp/77845)
**Author:** [@harshavardan25000](https://discuss.elastic.co/u/harshavardan25000)\ **Replies:** 1\ **Last updated:** [March 8, 2017, 4:13pm UTC](https://discuss.elastic.co/t/error-when-datestamp-pattern-is-used-to-parse-the-log-timestamp/77845 "2017-03-08T16:13:35Z")
The timestamp looks like "2017-02-06 04:01:12,757" and the pattern I chose is %{DATESTAMP:timestamp}. But when the parsing occurs , the timestamp is converted as 0017-02-06T10:42:24.923Z and it is causing issues while… --- ## [Pattern Matching](https://discuss.elastic.co/t/pattern-matching/77195)
**Author:** [@cisaksen](https://discuss.elastic.co/u/cisaksen)\ **Replies:** 7\ **Last updated:** [March 8, 2017, 1:30pm UTC](https://discuss.elastic.co/t/pattern-matching/77195 "2017-03-08T13:30:12Z")
Using grok I want to parse a log file as you normally would but i also want to pull a specific piece of of the log record in a custom field. I have 2 patterns one is a nginx\_combined with a x-forwarded field at the end… --- ## [Multiline filter - single threaded?](https://discuss.elastic.co/t/multiline-filter-single-threaded/77783)
**Author:** [@jetnet](https://discuss.elastic.co/u/jetnet)\ **Replies:** 3\ **Last updated:** [March 8, 2017, 1:18pm UTC](https://discuss.elastic.co/t/multiline-filter-single-threaded/77783 "2017-03-08T13:18:40Z")
Hi All, it looks like the multi-line filter cannot be used with more than one Logstash worker: I get the following error, when the Logstash gets started with -w N (N \> 1): 09:01:17.819 \[\[main\]\>worker0\] ERROR logstash… --- ## [Logstash dont use all available CPU-Cores](https://discuss.elastic.co/t/logstash-dont-use-all-available-cpu-cores/46538)
**Author:** [@german23](https://discuss.elastic.co/u/german23)\ **Replies:** 7\ **Last updated:** [March 8, 2017, 12:31pm UTC](https://discuss.elastic.co/t/logstash-dont-use-all-available-cpu-cores/46538 "2017-03-08T12:31:00Z")
Hey guys, sorry if this is already handled in another topic here, i did not find any matching. Last week we installed logstash on a new physical server, that have 28(+28 Multithreading) CPU Threads and 64GB RAM. Fi… --- ## [Grok pattern for highlighted log format](https://discuss.elastic.co/t/grok-pattern-for-highlighted-log-format/77804)
**Author:** [@Shweta](https://discuss.elastic.co/u/Shweta)\ **Replies:** 0\ **Last updated:** [March 8, 2017, 11:18am UTC](https://discuss.elastic.co/t/grok-pattern-for-highlighted-log-format/77804 "2017-03-08T11:18:39Z")
Hi, We are having log format as follows: \%highlight{%d{ISO8601}{UTC} %-5p \[%t\] %X{-tracking-id} %c{1} - %m%n}\ We created grok pattern for it and after parsing through logstas… --- ## [Windows logstash problem](https://discuss.elastic.co/t/windows-logstash-problem/77672)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 4\ **Last updated:** [March 8, 2017, 9:33am UTC](https://discuss.elastic.co/t/windows-logstash-problem/77672 "2017-03-08T09:33:27Z")
hi im running version 2,4.1 on a windows 2012 server, I have run the ELK stack sucessfully on linux but in order to get it in production i must migrate to windows. I thought i could just move everything to the wintel p… --- ## [Logstash parsing xml](https://discuss.elastic.co/t/logstash-parsing-xml/77785)
**Author:** [@prosenjit](https://discuss.elastic.co/u/prosenjit)\ **Replies:** 1\ **Last updated:** [March 8, 2017, 9:24am UTC](https://discuss.elastic.co/t/logstash-parsing-xml/77785 "2017-03-08T09:24:47Z")
I have a custom log that has xml contents inside, I tried using the xpath attribute. however the xml fields are not captured while the grok parsing times out sometimes for large values however, the grok filter works well… --- ## [SOLVED: Logstash 5.2.cannot install logstash-filter-anonymize in an offline environment](https://discuss.elastic.co/t/solved-logstash-5-2-cannot-install-logstash-filter-anonymize-in-an-offline-environment/76922)
**Author:** [@rodgermoore](https://discuss.elastic.co/u/rodgermoore)\ **Replies:** 3\ **Last updated:** [March 8, 2017, 9:00am UTC](https://discuss.elastic.co/t/solved-logstash-5-2-cannot-install-logstash-filter-anonymize-in-an-offline-environment/76922 "2017-03-08T09:00:08Z")
Ubuntu 14.04 LTS This works fine: sudo /usr/share/logstash/bin/logstash-plugin install logstash-filter-anonymize Validating logstash-filter-anonymize Installing logstash-filter-anonymize Installation successful Th… --- ## [Error while running logstash](https://discuss.elastic.co/t/error-while-running-logstash/77768)
**Author:** [@M.Sneha](https://discuss.elastic.co/u/M.Sneha)\ **Replies:** 0\ **Last updated:** [March 8, 2017, 6:43am UTC](https://discuss.elastic.co/t/error-while-running-logstash/77768 "2017-03-08T06:43:43Z")
I am getting this error while running logstash ....Can anyone kindly help. Thanks in advance. --- ## [Logstash connectivity error](https://discuss.elastic.co/t/logstash-connectivity-error/77767)
**Author:** [@sujitthakalanth](https://discuss.elastic.co/u/sujitthakalanth)\ **Replies:** 0\ **Last updated:** [March 8, 2017, 6:43am UTC](https://discuss.elastic.co/t/logstash-connectivity-error/77767 "2017-03-08T06:43:15Z")
Hi all, Im trying to load mysql table data to elasticsearch through logstash please find the error in the attached file --- ## [Logstash Tcp output does not working](https://discuss.elastic.co/t/logstash-tcp-output-does-not-working/77750)
**Author:** [@mjuzhj23](https://discuss.elastic.co/u/mjuzhj23)\ **Replies:** 0\ **Last updated:** [March 8, 2017, 3:21am UTC](https://discuss.elastic.co/t/logstash-tcp-output-does-not-working/77750 "2017-03-08T03:21:11Z")
Log stash 5.0.0 does not working after "Failed (Connection refused - Connection refused). Sleeping for 2.0" message. I was checking for an error situation. "A" tcp server working on 9812 port.. log stash output to A… --- ## [Http output plugin json output issue](https://discuss.elastic.co/t/http-output-plugin-json-output-issue/77757)
**Author:** [@souravbadami](https://discuss.elastic.co/u/souravbadami)\ **Replies:** 0\ **Last updated:** [March 8, 2017, 4:59am UTC](https://discuss.elastic.co/t/http-output-plugin-json-output-issue/77757 "2017-03-08T04:59:36Z")
I need to get output from the Logstash http output plugin in json format. I have configured input from the apache logs. Here's the configuration. input { file { path =\> '/var/log/apache2/access.log' } }… --- ## [Scripted field considers the last updated value for calculation](https://discuss.elastic.co/t/scripted-field-considers-the-last-updated-value-for-calculation/77105)
**Author:** [@Ruthu\_Ramya](https://discuss.elastic.co/u/Ruthu_Ramya)\ **Replies:** 3\ **Last updated:** [March 7, 2017, 10:38pm UTC](https://discuss.elastic.co/t/scripted-field-considers-the-last-updated-value-for-calculation/77105 "2017-03-07T22:38:20Z")
Consider my log example: Nov 30,15:02:42.080 Start playback Nov 30,15:02:42.082 Song 1 played Nov 30,15:02:42.085 Song 2 played Nov 30,15:02:42.089 Song 3 played Nov 30,15:02:42.090 End playback Nov 30,15:02:43.06… --- ## [How to set the scheduler of JDBC from the INPUT plugin to every 5 seconds](https://discuss.elastic.co/t/how-to-set-the-scheduler-of-jdbc-from-the-input-plugin-to-every-5-seconds/62164)
**Author:** [@xiaozhuqq48](https://discuss.elastic.co/u/xiaozhuqq48)\ **Replies:** 6\ **Last updated:** [March 7, 2017, 10:00pm UTC](https://discuss.elastic.co/t/how-to-set-the-scheduler-of-jdbc-from-the-input-plugin-to-every-5-seconds/62164 "2017-03-07T22:00:30Z")
Original doc :Schedule of when to periodically run statement, in Cron format for example: "\* \* \* \* \*" (execute query every minute, on the minute) Is there any way to narrow down the period to 5 seconds? Thanks. --- ## [Difference between json codec in input {} vs filter {}?](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575)
**Author:** [@ZillaG](https://discuss.elastic.co/u/ZillaG)\ **Replies:** 7\ **Last updated:** [March 7, 2017, 9:22pm UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575 "2017-03-07T21:22:44Z")
I have JSON-ified logs coming into my Logstash server. What's the difference between putting the json codec in input { codec =\> json } vs. filter { json =\> { source =\> "message" } } --- ## [Trying to read avro files](https://discuss.elastic.co/t/trying-to-read-avro-files/77726)
**Author:** [@kmroz](https://discuss.elastic.co/u/kmroz)\ **Replies:** 0\ **Last updated:** [March 7, 2017, 9:08pm UTC](https://discuss.elastic.co/t/trying-to-read-avro-files/77726 "2017-03-07T21:08:30Z")
I am wondering the best way to read these files. Currently I was going to send them over using filebeat, but I am not sure how ti the avro codec works? Is there an easier way to transfer the avro files into a json format… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1021) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1023)