# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=551 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 552 --- ## [Timezone problem in ElasticSearch and Logstash](https://discuss.elastic.co/t/timezone-problem-in-elasticsearch-and-logstash/180163)
**Author:** [@Ravindra\_Nath](https://discuss.elastic.co/u/Ravindra_Nath)\ **Replies:** 4\ **Last updated:** [May 9, 2019, 3:19pm UTC](https://discuss.elastic.co/t/timezone-problem-in-elasticsearch-and-logstash/180163 "2019-05-09T15:19:51Z")
I have a DATETIME field in MySQL database table. I am dumping that table data to Elastic Search through logstash. When the data is being loaded into elastic search the time field(DATETIME type) of the log is changing. An… --- ## [How to connect two servers to Filebeat?](https://discuss.elastic.co/t/how-to-connect-two-servers-to-filebeat/180162)
**Author:** [@zenimagine](https://discuss.elastic.co/u/zenimagine)\ **Replies:** 5\ **Last updated:** [May 9, 2019, 1:57pm UTC](https://discuss.elastic.co/t/how-to-connect-two-servers-to-filebeat/180162 "2019-05-09T13:57:45Z")
I have 2 servers HTTPS with let's Encrypt : monitoring.example.com with ELK (IP: 11.11.111.111) www.example.com with my site in production (IP: 22.22.222.222) To install and configure ELK, I followed this tutorial : … --- ## [Keep track of Context ID and insert into every incoming record](https://discuss.elastic.co/t/keep-track-of-context-id-and-insert-into-every-incoming-record/178538)
**Author:** [@ingxue](https://discuss.elastic.co/u/ingxue)\ **Replies:** 2\ **Last updated:** [May 9, 2019, 1:44pm UTC](https://discuss.elastic.co/t/keep-track-of-context-id-and-insert-into-every-incoming-record/178538 "2019-05-09T13:44:30Z")
I am a newbie in ELK stack. My question is: There is an event containing a context id and coming to logstash. I need to keep track of this context id and put it in all the incoming events. How can it be done? --- ## [Log Monitoring](https://discuss.elastic.co/t/log-monitoring/180143)
**Author:** [@Ashok2](https://discuss.elastic.co/u/Ashok2)\ **Replies:** 1\ **Last updated:** [May 8, 2019, 2:06pm UTC](https://discuss.elastic.co/t/log-monitoring/180143 "2019-05-08T14:06:29Z")
hi, There is a requirement to configure the error code below and check if the emails are getting generated. Error\_code: AMQ9616 Q\_manager: ESBPRT1 I have the logstash config file in which i have written the mail aler… --- ## [How to convert data types?](https://discuss.elastic.co/t/how-to-convert-data-types/180250)
**Author:** [@orx](https://discuss.elastic.co/u/orx)\ **Replies:** 3\ **Last updated:** [May 9, 2019, 1:40pm UTC](https://discuss.elastic.co/t/how-to-convert-data-types/180250 "2019-05-09T13:40:16Z")
Hi, I want to parse the following data which is in the .txt file (separated by tabs): 1 apple tree-a city1 2 banana tree-b city2 input { file { path =\> "path-to-file" start\_positi… --- ## [Parsing multiline Tomcat logs](https://discuss.elastic.co/t/parsing-multiline-tomcat-logs/178874)
**Author:** [@agentn](https://discuss.elastic.co/u/agentn)\ **Replies:** 2\ **Last updated:** [May 9, 2019, 1:38pm UTC](https://discuss.elastic.co/t/parsing-multiline-tomcat-logs/178874 "2019-05-09T13:38:16Z")
Hi, I'm trying to parse some Tomcat logs with logstash, here is an example of data : Summary25-04-19 10:16:00 WARN \[BasicResourcePool\] com.mchange.v2.resourcepool.BasicResourcePool@74f26dc -- Acquisition Attempt Faile… --- ## [Logtash Grok Parse Error](https://discuss.elastic.co/t/logtash-grok-parse-error/180357)
**Author:** [@kolten](https://discuss.elastic.co/u/kolten)\ **Replies:** 3\ **Last updated:** [May 9, 2019, 1:34pm UTC](https://discuss.elastic.co/t/logtash-grok-parse-error/180357 "2019-05-09T13:34:44Z")
Hi, I want to parse CheckPoint Firewall syslog logs but logstash gives me an error. Here is my error: 2019-05-09T15:12:18,565\]\[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineAc… --- ## [Logstash 6.7 plugin Elasticsearch Error "bignum too big to convert into \`long'"](https://discuss.elastic.co/t/logstash-6-7-plugin-elasticsearch-error-bignum-too-big-to-convert-into-long/175201)
**Author:** [@alfonso.viso](https://discuss.elastic.co/u/alfonso.viso)\ **Replies:** 13\ **Last updated:** [May 9, 2019, 1:23pm UTC](https://discuss.elastic.co/t/logstash-6-7-plugin-elasticsearch-error-bignum-too-big-to-convert-into-long/175201 "2019-05-09T13:23:25Z")
Hi community, Recently, we've just upgrade the version of logstash, from 6.6 to 6.7. Since then, we are receiving these message: \[ERROR\]\[logstash.outputs.elasticsearch\] An unknown error occurred sending a bulk request … --- ## [How to differentiate between server logs in logstash?](https://discuss.elastic.co/t/how-to-differentiate-between-server-logs-in-logstash/180139)
**Author:** [@rplus](https://discuss.elastic.co/u/rplus)\ **Replies:** 2\ **Last updated:** [May 9, 2019, 11:22am UTC](https://discuss.elastic.co/t/how-to-differentiate-between-server-logs-in-logstash/180139 "2019-05-09T11:22:22Z")
Hi, First of all i'm not sure it this is the right place to post this, but i'll give it a shot. I've several windows servers running filebeat sending IIS logs to logstash. I need some way to differentiate between them … --- ## [Logstash TCP output to logstash TCP input with SSL error](https://discuss.elastic.co/t/logstash-tcp-output-to-logstash-tcp-input-with-ssl-error/180146)
**Author:** [@thangld](https://discuss.elastic.co/u/thangld)\ **Replies:** 6\ **Last updated:** [May 9, 2019, 10:20am UTC](https://discuss.elastic.co/t/logstash-tcp-output-to-logstash-tcp-input-with-ssl-error/180146 "2019-05-09T10:20:48Z")
Hi all. I'm trying to test Logstash TCP input plugin with SSL configuration so I run two logstash instance, one for TCP output and one for TCP input. These are my settings: input.conf : input { tcp { … --- ## [Flexible interval of exec input](https://discuss.elastic.co/t/flexible-interval-of-exec-input/180302)
**Author:** [@BennyInc](https://discuss.elastic.co/u/BennyInc)\ **Replies:** 2\ **Last updated:** [May 9, 2019, 8:46am UTC](https://discuss.elastic.co/t/flexible-interval-of-exec-input/180302 "2019-05-09T08:46:25Z")
We use an exec input in a Logstash configuration to poll the HA state of our database. Currently we set this run at an interval of 60, as the data isn't expected to change much during normal operation, and we don't want … --- ## [Pass multiple inputs in logstash config. file](https://discuss.elastic.co/t/pass-multiple-inputs-in-logstash-config-file/180293)
**Author:** [@hemantkhokhar](https://discuss.elastic.co/u/hemantkhokhar)\ **Replies:** 2\ **Last updated:** [May 9, 2019, 8:11am UTC](https://discuss.elastic.co/t/pass-multiple-inputs-in-logstash-config-file/180293 "2019-05-09T08:11:30Z")
Is this the right way to give multiple input..because i am not geeting logs on kibana. due to this i have to restart logstash after some interval continuously. input { beats { port =\> 5044 } } input { cloudwatch\_… --- ## [Unable to drop lines using drop filter of logstash](https://discuss.elastic.co/t/unable-to-drop-lines-using-drop-filter-of-logstash/179770)
**Author:** [@Suchismita\_Goswami](https://discuss.elastic.co/u/Suchismita_Goswami)\ **Replies:** 11\ **Last updated:** [May 9, 2019, 7:06am UTC](https://discuss.elastic.co/t/unable-to-drop-lines-using-drop-filter-of-logstash/179770 "2019-05-09T07:06:01Z")
Sample Log file: Started by user e\[8mha:////4JmSuRY+CFCKIu6RwHXiSXZDY4Huygv7JVrvzkfRI8zdAAAAmR+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBI… --- ## [Logstahs monitoring index no created](https://discuss.elastic.co/t/logstahs-monitoring-index-no-created/179446)
**Author:** [@Felipe\_Villalobos\_Fo](https://discuss.elastic.co/u/Felipe_Villalobos_Fo)\ **Replies:** 7\ **Last updated:** [May 9, 2019, 6:38am UTC](https://discuss.elastic.co/t/logstahs-monitoring-index-no-created/179446 "2019-05-09T06:38:56Z")
I'm having the same problem as the topic describe here. I can pipeline information with the output elasticsearch plugin, but the index for monitoring does not get created. My configuration for the monitoring is enabled… --- ## [Date filter not working with AM | PM](https://discuss.elastic.co/t/date-filter-not-working-with-am-pm/180199)
**Author:** [@sonukumarsah](https://discuss.elastic.co/u/sonukumarsah)\ **Replies:** 2\ **Last updated:** [May 9, 2019, 5:44am UTC](https://discuss.elastic.co/t/date-filter-not-working-with-am-pm/180199 "2019-05-09T05:44:21Z")
I have some log file for example this: {logtimestamp: Mar 29, 2019 11:43:22:948 AM SEVERE: ############SESSION JVM } As I want to use the timestamp in the log file, I use a data filter to match it date { match =\> \["l… --- ## [Logstash 6.6.0 sends logs to ES only few hours a day](https://discuss.elastic.co/t/logstash-6-6-0-sends-logs-to-es-only-few-hours-a-day/177002)
**Author:** [@Jaepyoung\_Kim](https://discuss.elastic.co/u/Jaepyoung_Kim)\ **Replies:** 5\ **Last updated:** [May 8, 2019, 11:52pm UTC](https://discuss.elastic.co/t/logstash-6-6-0-sends-logs-to-es-only-few-hours-a-day/177002 "2019-05-08T23:52:20Z")
I have several Elastic Searches clusters in several Kubernetes Clusters. Several VMs are sending the logs through filebeat to Logstash. Some logs show very wired pattern. The following graph shows that the logs were s… --- ## [ES 32kb Field Limit - Logstash Ruby Plugin help](https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152)
**Author:** [@EmFalcon](https://discuss.elastic.co/u/EmFalcon)\ **Replies:** 5\ **Last updated:** [May 3, 2019, 9:27pm UTC](https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152 "2019-05-03T21:27:16Z")
Using Elastic Stack 6.3. Workflow : Filebeat (input logfile) -\> Logstash -\> ES A specific log file we have generates a individual message that exceeds 32kb which from what I am reading the limit of lucene for index an… --- ## [Duplicate entries into Elastic Search](https://discuss.elastic.co/t/duplicate-entries-into-elastic-search/179778)
**Author:** [@Ravindra\_Nath](https://discuss.elastic.co/u/Ravindra_Nath)\ **Replies:** 7\ **Last updated:** [May 8, 2019, 8:54pm UTC](https://discuss.elastic.co/t/duplicate-entries-into-elastic-search/179778 "2019-05-08T20:54:59Z")
Problem: I am using ELK 6.71. I am able to get logs data to Elastic Search by input plugin from MySQL database and i scheduled to execute the SQL query for every 1 minute. But, every time logstash runs the query it also… --- ## [What exactly is @timestamp in logstash](https://discuss.elastic.co/t/what-exactly-is-timestamp-in-logstash/180232)
**Author:** [@rahulkothanath](https://discuss.elastic.co/u/rahulkothanath)\ **Replies:** 1\ **Last updated:** [May 8, 2019, 8:22pm UTC](https://discuss.elastic.co/t/what-exactly-is-timestamp-in-logstash/180232 "2019-05-08T20:22:43Z")
what is the meta data value of @timestamp in logstash indicates? is it the time when logstash receives the event? or is the time when logstash started processing the event through filter plugin? thanks for reading and … --- ## [Useragent filter plugin fields don't match ECS](https://discuss.elastic.co/t/useragent-filter-plugin-fields-dont-match-ecs/179428)
**Author:** [@Andy0708](https://discuss.elastic.co/u/Andy0708)\ **Replies:** 5\ **Last updated:** [May 8, 2019, 4:43pm UTC](https://discuss.elastic.co/t/useragent-filter-plugin-fields-dont-match-ecs/179428 "2019-05-08T16:43:15Z")
I have a Logstash 6.x pipeline which processes Apache access logs and stores each line as a document in Elasticsearch 6.x (using the index template from Filebeat). I am now upgrading both Logstash and Elasticsearch to 7.… --- ## [\[Resolved\]Logstash - Issue of SUM by aggregate filter conditionally and can't get the expected csv output, how to correct?](https://discuss.elastic.co/t/resolved-logstash-issue-of-sum-by-aggregate-filter-conditionally-and-cant-get-the-expected-csv-output-how-to-correct/180111)
**Author:** [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Replies:** 4\ **Last updated:** [May 8, 2019, 4:27pm UTC](https://discuss.elastic.co/t/resolved-logstash-issue-of-sum-by-aggregate-filter-conditionally-and-cant-get-the-expected-csv-output-how-to-correct/180111 "2019-05-08T16:27:12Z")
Hi Team, I have below CSV data which are exported from ES, when,module,fa,ui,at,ci,cn,link "April 23rd 2019, 09:13:31.562",SCM,MR,cgrant1,SCM.MR.LOAD\_FILTERS,TALSCMK,TALSCMK,1 "April 23rd 2019, 22:38:31.868",SCM,TS,101… --- ## [Logstash elasticsearch ruby filter issues](https://discuss.elastic.co/t/logstash-elasticsearch-ruby-filter-issues/179820)
**Author:** [@raged](https://discuss.elastic.co/u/raged)\ **Replies:** 10\ **Last updated:** [May 8, 2019, 3:48pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-ruby-filter-issues/179820 "2019-05-08T15:48:33Z")
Good morning all, I am attempting to join some data together from a different index in logstash. I have the following configuration file: input { beats { port =\> 5044 } } filter { elasticsearch { hosts =\> \["ht… --- ## [Geo\_shape - geo link problems with coordinates](https://discuss.elastic.co/t/geo-shape-geo-link-problems-with-coordinates/179924)
**Author:** [@gofbor](https://discuss.elastic.co/u/gofbor)\ **Replies:** 8\ **Last updated:** [May 8, 2019, 2:35pm UTC](https://discuss.elastic.co/t/geo-shape-geo-link-problems-with-coordinates/179924 "2019-05-08T14:35:40Z")
Hi, I've got index with geo\_shape type field. I can POST doc via dev\_tool: For example: POST test/doc { "geo\_link" : { "type": "LineString", "coordinates": \[ \[ -87.623177, 41.881832 \], \[ -86.623177, 42.881… --- ## [How to parse MQ consle output in logstash](https://discuss.elastic.co/t/how-to-parse-mq-consle-output-in-logstash/180193)
**Author:** [@Praful\_Chandra](https://discuss.elastic.co/u/Praful_Chandra)\ **Replies:** 1\ **Last updated:** [May 8, 2019, 2:32pm UTC](https://discuss.elastic.co/t/how-to-parse-mq-consle-output-in-logstash/180193 "2019-05-08T14:32:41Z")
IBMMQ3434 SYSTEM.ADMIN.QMGR.EVENT CURDEPTH(1) MAXDEPTH(3000) SYSTEM.AUTH.DATA.QUEUE CURDEPTH(114) MAXDEPTH(999999999) SYSTEM.CHANNEL.SYNCQ … --- ## [Logstash json input {}, “” filter](https://discuss.elastic.co/t/logstash-json-input-filter/179862)
**Author:** [@xodn0812](https://discuss.elastic.co/u/xodn0812)\ **Replies:** 3\ **Last updated:** [May 8, 2019, 1:51pm UTC](https://discuss.elastic.co/t/logstash-json-input-filter/179862 "2019-05-08T13:51:51Z")
I want to get a json-style event that contains "" or {}. However, if "" and {} are included, filtering is not done properly. Is there no way?? msg: {"key1":"val1","key2":"val2", "key3": "asfqwe12\\"wef\\"qw{asd}e213524"… --- ## [Logstash Management user with default logstash\_admin rol prevents logstash from starting](https://discuss.elastic.co/t/logstash-management-user-with-default-logstash-admin-rol-prevents-logstash-from-starting/180180)
**Author:** [@searchtool](https://discuss.elastic.co/u/searchtool)\ **Replies:** 0\ **Last updated:** [May 8, 2019, 1:35pm UTC](https://discuss.elastic.co/t/logstash-management-user-with-default-logstash-admin-rol-prevents-logstash-from-starting/180180 "2019-05-08T13:35:59Z")
Hi, i noticed when using the out of the box role logstash admin for a user as described in the docs xpack.management.elasticsearch.username and xpack.management.elasticsearch.password If your Elasticsearch cluster … --- ## [Geoip with two ASN (mmdb format) database](https://discuss.elastic.co/t/geoip-with-two-asn-mmdb-format-database/180177)
**Author:** [@yasvanth](https://discuss.elastic.co/u/yasvanth)\ **Replies:** 0\ **Last updated:** [May 8, 2019, 1:26pm UTC](https://discuss.elastic.co/t/geoip-with-two-asn-mmdb-format-database/180177 "2019-05-08T13:26:54Z")
Hi Team, I am trying to generate static ASN for the IPs in local network but I couldn't get any results. My filter config: if \[@metadata\]\[asn\_lookup\] == "true" { geoip { id =\> "elast… --- ## [\[Resolved\]Logstash - what does logstash Starting flush cycle mean?](https://discuss.elastic.co/t/resolved-logstash-what-does-logstash-starting-flush-cycle-mean/180126)
**Author:** [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Replies:** 1\ **Last updated:** [May 8, 2019, 1:11pm UTC](https://discuss.elastic.co/t/resolved-logstash-what-does-logstash-starting-flush-cycle-mean/180126 "2019-05-08T13:11:49Z")
As titled. --- ## [Logstash Indexing in Elasticsearch Bytecode Problem](https://discuss.elastic.co/t/logstash-indexing-in-elasticsearch-bytecode-problem/180172)
**Author:** [@ahmetkumas1](https://discuss.elastic.co/u/ahmetkumas1)\ **Replies:** 0\ **Last updated:** [May 8, 2019, 1:08pm UTC](https://discuss.elastic.co/t/logstash-indexing-in-elasticsearch-bytecode-problem/180172 "2019-05-08T13:08:45Z")
Hello friends, I'm trying to mapping with logstash my config file works but there is a problem all data imported as bytecode how to solve this? I'm I missed something? I can index in elasticsearch but it's not valid. L… --- ## [Logstash-plugin returns error on vanilla installation](https://discuss.elastic.co/t/logstash-plugin-returns-error-on-vanilla-installation/180149)
**Author:** [@telefonen](https://discuss.elastic.co/u/telefonen)\ **Replies:** 0\ **Last updated:** [May 8, 2019, 10:09am UTC](https://discuss.elastic.co/t/logstash-plugin-returns-error-on-vanilla-installation/180149 "2019-05-08T10:09:44Z")
Using a vanilla Windows Server 2016 with the latest version of Elastic,Kibana and Logstash. java JDK 8u211 is installed and JAVA\_HOME is set. When I run the logstash-plugin it returns the following: C:\\Logstash\\bin\>lo… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=550) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=552)