# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1034 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1035 --- ## [Logstash @timestamp precision causing misordering of data](https://discuss.elastic.co/t/logstash-timestamp-precision-causing-misordering-of-data/75550)
**Author:** [@rbgeno7](https://discuss.elastic.co/u/rbgeno7)\ **Replies:** 4\ **Last updated:** [February 17, 2017, 6:43pm UTC](https://discuss.elastic.co/t/logstash-timestamp-precision-causing-misordering-of-data/75550 "2017-02-17T18:43:29Z")
I'm using filebeat and logstash to push Yarn Container Logs (specifically Spark) to Kinesis then ES and Kibana. I'm running these as SysV services on AWS Linux: Filebeat - 5.2.1 Logstash - 2.4.1 (Need the Kinesis Outp… --- ## [Logstash file rotation by renaming the file](https://discuss.elastic.co/t/logstash-file-rotation-by-renaming-the-file/75532)
**Author:** [@krishna\_chaitanya](https://discuss.elastic.co/u/krishna_chaitanya)\ **Replies:** 0\ **Last updated:** [February 17, 2017, 4:24pm UTC](https://discuss.elastic.co/t/logstash-file-rotation-by-renaming-the-file/75532 "2017-02-17T16:24:32Z")
I am trying to rotate files nightly, where file is being renamed from name.log to name-{yyyy-mm-dd}.log and creating a new name.log file. In logstash input config, I am setting path =\> name\*.log In the documentation abo… --- ## [How to add a pattern variable for all messages](https://discuss.elastic.co/t/how-to-add-a-pattern-variable-for-all-messages/75418)
**Author:** [@gcranshaw](https://discuss.elastic.co/u/gcranshaw)\ **Replies:** 1\ **Last updated:** [February 17, 2017, 4:18pm UTC](https://discuss.elastic.co/t/how-to-add-a-pattern-variable-for-all-messages/75418 "2017-02-17T16:18:39Z")
OK, I'm at a loss on how I can accomplish this. I am using the following logstash 5.2 filter for Cisco ASA's. filter { if \[type\] == "cisco-asa" { grok { patterns\_dir =\> \["/opt/logstash/patterns"\] matc… --- ## [Boolean Mapping Issue](https://discuss.elastic.co/t/boolean-mapping-issue/75478)
**Author:** [@matt.dunks](https://discuss.elastic.co/u/matt.dunks)\ **Replies:** 0\ **Last updated:** [February 17, 2017, 11:33am UTC](https://discuss.elastic.co/t/boolean-mapping-issue/75478 "2017-02-17T11:33:15Z")
Hi My problem is we have a template defined with fields set to boolean, logstash has received numeric values for these feilds this morning ( from an old application build) and the index mapping has been set to Long ign… --- ## [GMT Timezone to CST in logstash](https://discuss.elastic.co/t/gmt-timezone-to-cst-in-logstash/75424)
**Author:** [@abd.wsu](https://discuss.elastic.co/u/abd.wsu)\ **Replies:** 9\ **Last updated:** [February 17, 2017, 3:47pm UTC](https://discuss.elastic.co/t/gmt-timezone-to-cst-in-logstash/75424 "2017-02-17T15:47:58Z")
Hello, Can anyone help me convert a GMT timezone to CST? I tried some options but haven't had accurate results so far. I have a CSV file that contains fileds like below "Record Type","Record Code","Broker Name","Bro… --- ## [Getting logstash-input-jmx up and running](https://discuss.elastic.co/t/getting-logstash-input-jmx-up-and-running/73526)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 1\ **Last updated:** [February 17, 2017, 3:23pm UTC](https://discuss.elastic.co/t/getting-logstash-input-jmx-up-and-running/73526 "2017-02-17T15:23:40Z")
I am new to the entire framework and one of the things that we want to do at my company is to use this jmx plug-in to monitor the JVM Metrics on our various Tomcat servers. So I was able to successfully get the plug-in i… --- ## [Auto acknowledgement to message from RabbitMQ](https://discuss.elastic.co/t/auto-acknowledgement-to-message-from-rabbitmq/75511)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 4\ **Last updated:** [February 17, 2017, 2:40pm UTC](https://discuss.elastic.co/t/auto-acknowledgement-to-message-from-rabbitmq/75511 "2017-02-17T14:40:08Z")
Hi List-users I have a logstash consumer which is reading messages from RabbitMQ queues through rabbitmq input. I want to speed up the delivery using auto acknowledgement = true . I have checked the logstash documentat… --- ## [Sql\_last\_value ot working?](https://discuss.elastic.co/t/sql-last-value-ot-working/75410)
**Author:** [@mano\_even](https://discuss.elastic.co/u/mano_even)\ **Replies:** 1\ **Last updated:** [February 17, 2017, 2:31pm UTC](https://discuss.elastic.co/t/sql-last-value-ot-working/75410 "2017-02-17T14:31:37Z")
Hello , i just import my data from my mssql server to ES. now i am trying to use the the sql\_last\_value options ,but its not working. i have a column in my table "myDate", so i want when i update it , iget the update… --- ## [Help with monitoring MBeans with jmx plugin of logstash](https://discuss.elastic.co/t/help-with-monitoring-mbeans-with-jmx-plugin-of-logstash/63624)
**Author:** [@harperkej](https://discuss.elastic.co/u/harperkej)\ **Replies:** 2\ **Last updated:** [February 17, 2017, 2:12pm UTC](https://discuss.elastic.co/t/help-with-monitoring-mbeans-with-jmx-plugin-of-logstash/63624 "2017-02-17T14:12:46Z")
Hi everyone :slight\_smile: I am using logstash input jmx plugin to retrieve some parameters from a Java application. The idea it to get two or more attributes from on MBean through jmx plugin and to show those attribu… --- ## [Logstash Grok Filter \_tags vs \_fields](https://discuss.elastic.co/t/logstash-grok-filter--tags-vs--fields/75026)
**Author:** [@spravn789](https://discuss.elastic.co/u/spravn789)\ **Replies:** 4\ **Last updated:** [February 17, 2017, 1:19pm UTC](https://discuss.elastic.co/t/logstash-grok-filter--tags-vs--fields/75026 "2017-02-17T13:19:38Z")
Dear All, Have few queries as follows, Requirement is to remove the tag/field Bwpayload and to add message, program and path as tags in case of grokparsefailure. i am confused about either to use remove\_tag or remov… --- ## [Multiple instances of Logstash](https://discuss.elastic.co/t/multiple-instances-of-logstash/75215)
**Author:** [@adrianolimit](https://discuss.elastic.co/u/adrianolimit)\ **Replies:** 4\ **Last updated:** [February 17, 2017, 1:16pm UTC](https://discuss.elastic.co/t/multiple-instances-of-logstash/75215 "2017-02-17T13:16:54Z")
Hello everyone, I want to use Logstash with the "Twitter input" in order to track tweets about different themes. For example, I want to use one Logstash instance by theme with the following configurations : Logstash… --- ## [Stuck with Ruby SocketError](https://discuss.elastic.co/t/stuck-with-ruby-socketerror/72490)
**Author:** [@Melpheos](https://discuss.elastic.co/u/Melpheos)\ **Replies:** 1\ **Last updated:** [February 17, 2017, 11:15am UTC](https://discuss.elastic.co/t/stuck-with-ruby-socketerror/72490 "2017-02-17T11:15:14Z")
Hi, we are getting sflow information with logstash and would like to use a simple ruby code to calculate the average bandwidth used by ip. Unfortunately when we activate in our filter rule the ruby code we end up with … --- ## [Ruby filter usage to divide value and create new events](https://discuss.elastic.co/t/ruby-filter-usage-to-divide-value-and-create-new-events/75020)
**Author:** [@pavlik](https://discuss.elastic.co/u/pavlik)\ **Replies:** 1\ **Last updated:** [February 17, 2017, 8:36am UTC](https://discuss.elastic.co/t/ruby-filter-usage-to-divide-value-and-create-new-events/75020 "2017-02-17T08:36:18Z")
Hello, I have the following event: "message" =\> "2017-01-02 08:00;2017-01-02 09:00;30;Item1, Item2, Item3", "@version" =\> "1", "@timestamp" =\> "2017-01-02T08:00:00.000Z", … --- ## [Elasticsearch update the document without updating the existing field value](https://discuss.elastic.co/t/elasticsearch-update-the-document-without-updating-the-existing-field-value/75197)
**Author:** [@banu](https://discuss.elastic.co/u/banu)\ **Replies:** 1\ **Last updated:** [February 17, 2017, 8:28am UTC](https://discuss.elastic.co/t/elasticsearch-update-the-document-without-updating-the-existing-field-value/75197 "2017-02-17T08:28:14Z")
Trying to update a specific field in elasticsearch through logstash. Is it possible to update only a set of fields through logstash ? Please find the code below, input { file { path =\> "/\*\*/\*\*/logstash/bin… --- ## [Jdbc sql\_last\_value for UTC+8](https://discuss.elastic.co/t/jdbc-sql-last-value-for-utc-8/75431)
**Author:** [@Greentea](https://discuss.elastic.co/u/Greentea)\ **Replies:** 2\ **Last updated:** [February 17, 2017, 8:08am UTC](https://discuss.elastic.co/t/jdbc-sql-last-value-for-utc-8/75431 "2017-02-17T08:08:22Z")
Hi I want to ask how to set the sql\_last\_value for different time zone? according my understanding, the sql\_last\_value is using UTC time. But my database column(last\_update\_date) is using UTC+8 time. I just want to know … --- ## [How to query elasticsearch and take some fields from old data](https://discuss.elastic.co/t/how-to-query-elasticsearch-and-take-some-fields-from-old-data/75300)
**Author:** [@Vittorio](https://discuss.elastic.co/u/Vittorio)\ **Replies:** 2\ **Last updated:** [February 17, 2017, 8:08am UTC](https://discuss.elastic.co/t/how-to-query-elasticsearch-and-take-some-fields-from-old-data/75300 "2017-02-17T08:08:09Z")
Hi all, I'm new here, hope to get some help :slight\_smile: I'm working using the logstash pipeline, for instance, I have rabbitmq as input for my logs and Elastisearch as output. Fantastic! The problem is in the elast… --- ## [Failed to create monitoring event](https://discuss.elastic.co/t/failed-to-create-monitoring-event/74876)
**Author:** [@shubhrant](https://discuss.elastic.co/u/shubhrant)\ **Replies:** 7\ **Last updated:** [February 17, 2017, 6:26am UTC](https://discuss.elastic.co/t/failed-to-create-monitoring-event/74876 "2017-02-17T06:26:10Z")
after configuring the X-PACK logstash is not processing the the logs configuration of logstash input { tcp { port =\> 5044 charset =\> "ISO-8859-1" } } filter { if \[type\] == "syslog" { gr… --- ## [Could not find log4j2 configuration fileat path(../..) in wondows 7](https://discuss.elastic.co/t/could-not-find-log4j2-configuration-fileat-path-in-wondows-7/75435)
**Author:** [@shubh09](https://discuss.elastic.co/u/shubh09)\ **Replies:** 0\ **Last updated:** [February 17, 2017, 5:41am UTC](https://discuss.elastic.co/t/could-not-find-log4j2-configuration-fileat-path-in-wondows-7/75435 "2017-02-17T05:41:57Z")
Hi Team, I am getting error while executing conf file when use logstash 5.2.0 first time. can you please help me to resolve this. --- ## [Biztalk server event capture](https://discuss.elastic.co/t/biztalk-server-event-capture/75133)
**Author:** [@Prateek\_Kshtriya](https://discuss.elastic.co/u/Prateek_Kshtriya)\ **Replies:** 4\ **Last updated:** [February 17, 2017, 5:21am UTC](https://discuss.elastic.co/t/biztalk-server-event-capture/75133 "2017-02-17T05:21:55Z")
Hi All, Please suggest if we could capture biztalk event log without explicitly saving the logs in a file. PFB the sample log- Regards, Prateek --- ## [Error: undefined method \`\<=' for nil:NilClass\>,](https://discuss.elastic.co/t/error-undefined-method-for-nil-nilclass/74873)
**Author:** [@polymorfiqcoder](https://discuss.elastic.co/u/polymorfiqcoder)\ **Replies:** 1\ **Last updated:** [February 17, 2017, 4:41am UTC](https://discuss.elastic.co/t/error-undefined-method-for-nil-nilclass/74873 "2017-02-17T04:41:27Z")
try to push windows EventLog into logstash and get pereodicaly .. \`\<=' for nil:NilClass\>, Woindows 2008r2 w/Cyrilic locale input { # get events from NxLog via tcp+json tcp { type =\> "EventLog" co… --- ## [S3 output plugin - no way to change the file names](https://discuss.elastic.co/t/s3-output-plugin-no-way-to-change-the-file-names/75425)
**Author:** [@FlorinAndrei](https://discuss.elastic.co/u/FlorinAndrei)\ **Replies:** 0\ **Last updated:** [February 17, 2017, 2:19am UTC](https://discuss.elastic.co/t/s3-output-plugin-no-way-to-change-the-file-names/75425 "2017-02-17T02:19:29Z")
logstash-5.2.1 on Ubuntu 16.04 s3 { access\_key\_id =\> "XXXXXXXXXXXXXXXX" secret\_access\_key =\> "XXXXXXXXXXXXXXXXX" region =\> "us-west-2" bucket =\> "some-bucket" time\_file =\> 5 size\_file =\> 1000000000000 pre… --- ## [S3 output plugin - cleanup when exiting is a bit messy](https://discuss.elastic.co/t/s3-output-plugin-cleanup-when-exiting-is-a-bit-messy/75422)
**Author:** [@FlorinAndrei](https://discuss.elastic.co/u/FlorinAndrei)\ **Replies:** 0\ **Last updated:** [February 17, 2017, 1:50am UTC](https://discuss.elastic.co/t/s3-output-plugin-cleanup-when-exiting-is-a-bit-messy/75422 "2017-02-17T01:50:48Z")
logstash-5.2.1 on Ubuntu 16.04 s3 { access\_key\_id =\> "XXXXXXXXXXXXXXXX" secret\_access\_key =\> "XXXXXXXXXXXXXXXXX" region =\> "us-west-2" bucket =\> "some-bucket" time\_file =\> 5 size\_file =\> 1000000000000 pre… --- ## [Logstash 5.2 input kafka working but no output to elasticsearch](https://discuss.elastic.co/t/logstash-5-2-input-kafka-working-but-no-output-to-elasticsearch/74720)
**Author:** [@camilosantana](https://discuss.elastic.co/u/camilosantana)\ **Replies:** 7\ **Last updated:** [February 16, 2017, 9:18pm UTC](https://discuss.elastic.co/t/logstash-5-2-input-kafka-working-but-no-output-to-elasticsearch/74720 "2017-02-16T21:18:10Z")
Issue Summary: filebeat outputs to kafka 0.10 topic logstash indexer inputs from kafka 0.10 outputs to elasticsearch Problem: document count on elasticsearch doesn't change Note: I'm able to get filebeat=\>logstash=\>… --- ## [How to read csv data in logstash and convert column to date,integer,boolean etc](https://discuss.elastic.co/t/how-to-read-csv-data-in-logstash-and-convert-column-to-date-integer-boolean-etc/75056)
**Author:** [@varun1992](https://discuss.elastic.co/u/varun1992)\ **Replies:** 9\ **Last updated:** [February 16, 2017, 9:45pm UTC](https://discuss.elastic.co/t/how-to-read-csv-data-in-logstash-and-convert-column-to-date-integer-boolean-etc/75056 "2017-02-16T21:45:35Z")
I got error when creating the index in logstash Could not find log4j2 configuration at path /ElasticProducts/logstash-5.2.0/config/log4j2.properties. Using default config which logs to console 08:38:21.769 \[LogStash::R… --- ## [Logstash timeout on parsing data](https://discuss.elastic.co/t/logstash-timeout-on-parsing-data/75401)
**Author:** [@kmroz](https://discuss.elastic.co/u/kmroz)\ **Replies:** 0\ **Last updated:** [February 16, 2017, 9:38pm UTC](https://discuss.elastic.co/t/logstash-timeout-on-parsing-data/75401 "2017-02-16T21:38:02Z")
Hello I am using version logstash-5.1.2. It seems while shipping the data with filebeat, within the the filebeat logs I am getting a lot of timeout 's . Would this have to do with my filebeat config or logstash. I have t… --- ## [No log4j2 configuration file found error even though it was working a few days back](https://discuss.elastic.co/t/no-log4j2-configuration-file-found-error-even-though-it-was-working-a-few-days-back/75387)
**Author:** [@abd.wsu](https://discuss.elastic.co/u/abd.wsu)\ **Replies:** 1\ **Last updated:** [February 16, 2017, 9:16pm UTC](https://discuss.elastic.co/t/no-log4j2-configuration-file-found-error-even-though-it-was-working-a-few-days-back/75387 "2017-02-16T21:16:30Z")
I am trying to parse a csv file to remove many unwanted columns and get the output in a different csv file and codec output at the same time. It was working fine till last week. Now i get a Api Webserver ERROR No log4… --- ## [Get only the last record](https://discuss.elastic.co/t/get-only-the-last-record/75320)
**Author:** [@mano\_even](https://discuss.elastic.co/u/mano_even)\ **Replies:** 6\ **Last updated:** [February 16, 2017, 8:00pm UTC](https://discuss.elastic.co/t/get-only-the-last-record/75320 "2017-02-16T20:00:45Z")
Hello , Im using logstash and jdbc driver to import my data to ES. but when i look for data using ,i only find the last record? my conf: input { jdbc { jdbc\_connection\_string =\> "jdbc:sqlserver:myconnexion;da… --- ## [Logstash Poller Error - Concurrent:TimeoutError - logstash-5.0.0-beta1](https://discuss.elastic.co/t/logstash-poller-error-concurrent-timeouterror-logstash-5-0-0-beta1/62448)
**Author:** [@eli.keimig](https://discuss.elastic.co/u/eli.keimig)\ **Replies:** 14\ **Last updated:** [February 16, 2017, 5:55pm UTC](https://discuss.elastic.co/t/logstash-poller-error-concurrent-timeouterror-logstash-5-0-0-beta1/62448 "2017-02-16T17:55:44Z")
Getting this error in my logstash-plain.log. There doesn't seem to be an exact frequency from what I can tell - or exact replication steps. It is not affecting messages going the output filters. This logstash is installe… --- ## [About Increasing Logstash Performance](https://discuss.elastic.co/t/about-increasing-logstash-performance/75179)
**Author:** [@olorasde](https://discuss.elastic.co/u/olorasde)\ **Replies:** 3\ **Last updated:** [February 16, 2017, 4:27pm UTC](https://discuss.elastic.co/t/about-increasing-logstash-performance/75179 "2017-02-16T16:27:24Z")
Hi, I have a logstash agent that extracts data from redis, preprocesses it and reengines it in redis again. I have been able to incorporate in this agent the metric filter to give me an idea of ​​how fast the process… --- ## [Persistent Queue Configuration question](https://discuss.elastic.co/t/persistent-queue-configuration-question/73408)
**Author:** [@ash007](https://discuss.elastic.co/u/ash007)\ **Replies:** 12\ **Last updated:** [February 16, 2017, 4:20pm UTC](https://discuss.elastic.co/t/persistent-queue-configuration-question/73408 "2017-02-16T16:20:15Z")
Hi, This might a very simple question but i am new to logstash. I have just configured my logstash instance with the following configuration in the logstash.yml file: path.data: /var/lib/logstash queue.type: persiste… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1033) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1035)