# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=412 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 413 --- ## [Could not index event to Elasticsearch, failed to parse \[timestamp\]](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-failed-to-parse-timestamp/216742)
**Author:** [@klang](https://discuss.elastic.co/u/klang)\ **Replies:** 2\ **Last updated:** [January 30, 2020, 10:44pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-failed-to-parse-timestamp/216742 "2020-01-30T22:44:18Z")
Good evening, I'm pushing JSON content from filebeats to logstash and I'm receiving a 'failed to parse \[timestamp\]' error and don't understand what is wrong but to be fair, I'm, pretty green to this technology. Sample … --- ## [Lumberjack fails afeter updating to 7.5.2](https://discuss.elastic.co/t/lumberjack-fails-afeter-updating-to-7-5-2/217228)
**Author:** [@Joao\_Neves](https://discuss.elastic.co/u/Joao_Neves)\ **Replies:** 1\ **Last updated:** [January 30, 2020, 8:46pm UTC](https://discuss.elastic.co/t/lumberjack-fails-afeter-updating-to-7-5-2/217228 "2020-01-30T20:46:21Z")
Hello, After updating ELK stack from 7.4.2 to 7.5.2 lumberjack communication stopped working appropriately. Other kind of communications still works (i.e syslog). A simplified overview of my architecture: subsidiary\_be… --- ## [Persisted queue checkpoint and acknowledged inputs](https://discuss.elastic.co/t/persisted-queue-checkpoint-and-acknowledged-inputs/217219)
**Author:** [@franck.lefebure](https://discuss.elastic.co/u/franck.lefebure)\ **Replies:** 0\ **Last updated:** [January 30, 2020, 3:10pm UTC](https://discuss.elastic.co/t/persisted-queue-checkpoint-and-acknowledged-inputs/217219 "2020-01-30T15:10:38Z")
Hi, I recently had to dig in our stack to identify some missing events, and one of the candidate explanations sits in this logstash doc section : By default, PQ will force write data to disk at every 1024 received inp… --- ## [Date format representation for "2020-01-29T21:40:00.000+0000" to assign to @timestamp](https://discuss.elastic.co/t/date-format-representation-for-2020-01-29t2100-000-0000-to-assign-to-timestamp/217261)
**Author:** [@xyz2](https://discuss.elastic.co/u/xyz2)\ **Replies:** 2\ **Last updated:** [January 30, 2020, 8:15pm UTC](https://discuss.elastic.co/t/date-format-representation-for-2020-01-29t2100-000-0000-to-assign-to-timestamp/217261 "2020-01-30T20:15:02Z")
HI, I need to assign the string in format "2020-01-29T21:40:00.000+0000" to @timestamp. Used date filter . How is the format represented exactly ? date { match =\> \[ "\[sampleDate\]", "ISO8601"\] #match =\> \[ "\[sampl… --- ## [How ot deal with empty field in xml parser?](https://discuss.elastic.co/t/how-ot-deal-with-empty-field-in-xml-parser/217250)
**Author:** [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Replies:** 2\ **Last updated:** [January 30, 2020, 6:54pm UTC](https://discuss.elastic.co/t/how-ot-deal-with-empty-field-in-xml-parser/217250 "2020-01-30T18:54:18Z")
i m sorry if this is repeat question. I m struggling with xml having empty values. I m using logstash 7.5.1 XML filter. Here is my config. ##Input section input { file { path =\> "/tmp/test.xml"… --- ## [Aggregate filter plugin +](https://discuss.elastic.co/t/aggregate-filter-plugin/217015)
**Author:** [@binygreenberg](https://discuss.elastic.co/u/binygreenberg)\ **Replies:** 1\ **Last updated:** [January 30, 2020, 5:55pm UTC](https://discuss.elastic.co/t/aggregate-filter-plugin/217015 "2020-01-30T17:55:58Z")
I'm interested in using the logstash aggregate filter plugin but I was wondering how this would work in my case where I have multiple logstash nodes. also do i need to set - filter workers to 1 ( -w 1 flag) - if I don'… --- ## [Ruby or groak for string menuplation](https://discuss.elastic.co/t/ruby-or-groak-for-string-menuplation/217236)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 4\ **Last updated:** [January 30, 2020, 4:49pm UTC](https://discuss.elastic.co/t/ruby-or-groak-for-string-menuplation/217236 "2020-01-30T16:49:44Z")
I have a two different type of string example1\_string = /a/b/c/./d/e ( if found . then I just want to remove only './' ) example2\_sting = /a/b/c/../d/e ( if found .. anywhere in string then I want to just remove ' c… --- ## [Forked Pipelines](https://discuss.elastic.co/t/forked-pipelines/217151)
**Author:** [@BeMoore](https://discuss.elastic.co/u/BeMoore)\ **Replies:** 1\ **Last updated:** [January 30, 2020, 4:37pm UTC](https://discuss.elastic.co/t/forked-pipelines/217151 "2020-01-30T16:37:24Z")
Hi all, I'm working on some forked pipelines and im getting a little confused on how this should be working... and this is sort of a continuation from this thread Single data source split to different indexes So the ex… --- ## [Logstash to filter and drop all strings mentioned in exception.txt file](https://discuss.elastic.co/t/logstash-to-filter-and-drop-all-strings-mentioned-in-exception-txt-file/217107)
**Author:** [@Anirban\_Banerjee](https://discuss.elastic.co/u/Anirban_Banerjee)\ **Replies:** 4\ **Last updated:** [January 30, 2020, 4:33pm UTC](https://discuss.elastic.co/t/logstash-to-filter-and-drop-all-strings-mentioned-in-exception-txt-file/217107 "2020-01-30T16:33:56Z")
Hello We are trying to see the feasibility of using logstash and icinga (check\_logfiles plugins) in tandem to monitor postgres logs. The idea is we have a file called ignore\_alert.txt. It has 100s of strings which the … --- ## [Grok pattern matching](https://discuss.elastic.co/t/grok-pattern-matching/217078)
**Author:** [@littlejob1](https://discuss.elastic.co/u/littlejob1)\ **Replies:** 3\ **Last updated:** [January 30, 2020, 4:21pm UTC](https://discuss.elastic.co/t/grok-pattern-matching/217078 "2020-01-30T16:21:15Z")
Trying to figure out the most effective way to parse the following message being received via a syslog input. \<134\>1 2020-01-29T21:18:07Z 10.10.10.10 PulseSecure: - - - 2020-01-29 21:18:07 - ssl - \[4.4.4.4\] domain\\user(… --- ## [How i can work with kibana (Grockl)](https://discuss.elastic.co/t/how-i-can-work-with-kibana-grockl/217218)
**Author:** [@Sergey\_Serov](https://discuss.elastic.co/u/Sergey_Serov)\ **Replies:** 0\ **Last updated:** [January 30, 2020, 3:02pm UTC](https://discuss.elastic.co/t/how-i-can-work-with-kibana-grockl/217218 "2020-01-30T15:02:58Z")
I'm not sure what it's called, sort of grok maybe ) Do not judge strictly) today I am the first day with ELK. I have a problem with fields received from Windows Server. 'Execution ProcessID' = 'ProcessId' Can someone… --- ## [JSON input field](https://discuss.elastic.co/t/json-input-field/217007)
**Author:** [@Mbt\_Mbt](https://discuss.elastic.co/u/Mbt_Mbt)\ **Replies:** 13\ **Last updated:** [January 30, 2020, 1:23pm UTC](https://discuss.elastic.co/t/json-input-field/217007 "2020-01-30T13:23:32Z")
Good afternoon, I am trying to save the information contained in a field in independent fields for later analysis in kibana. I have tried actions of the json target, source plugin ... but I don't get any positive result… --- ## [Logstash on RHEL 8.x](https://discuss.elastic.co/t/logstash-on-rhel-8-x/217178)
**Author:** [@caiossouza](https://discuss.elastic.co/u/caiossouza)\ **Replies:** 0\ **Last updated:** [January 30, 2020, 12:20pm UTC](https://discuss.elastic.co/t/logstash-on-rhel-8-x/217178 "2020-01-30T12:20:30Z")
Hi, I would like to confirm if we can run Logstash on RHEL 8.x servers. Reading the Support Matrix (https://www.elastic.co/pt/support/matrix) we can see RHEL 8.X is compatible with some recent components, like beats, b… --- ## [Facing issue while installing any filter plugin such as logstash-filter-elapsed](https://discuss.elastic.co/t/facing-issue-while-installing-any-filter-plugin-such-as-logstash-filter-elapsed/216918)
**Author:** [@nishug001](https://discuss.elastic.co/u/nishug001)\ **Replies:** 3\ **Last updated:** [January 30, 2020, 10:27am UTC](https://discuss.elastic.co/t/facing-issue-while-installing-any-filter-plugin-such-as-logstash-filter-elapsed/216918 "2020-01-30T10:27:35Z")
Hi everyone, I'm trying to install logstash-filter-elapsed on my logstash environment but I'm getting below error every time. I tried installing other plugins but still observe the same issue. Can someone please help… --- ## [Error pushing logs to elasticsearch using kafka as input for logstash](https://discuss.elastic.co/t/error-pushing-logs-to-elasticsearch-using-kafka-as-input-for-logstash/214658)
**Author:** [@Monu\_Kumar](https://discuss.elastic.co/u/Monu_Kumar)\ **Replies:** 1\ **Last updated:** [January 30, 2020, 8:42am UTC](https://discuss.elastic.co/t/error-pushing-logs-to-elasticsearch-using-kafka-as-input-for-logstash/214658 "2020-01-30T08:42:47Z")
Below is my logstash config file logstash.conf. Lotstash reads logs from Kafka topic 'my\_topic' and outputs it to elasticsearch index 'es-index'. input { kafka { bootstrap\_servers =\>"kafka1.xxx:9092,kafka2.xxx:9092,k… --- ## [Logstash monitoring problem](https://discuss.elastic.co/t/logstash-monitoring-problem/215468)
**Author:** [@dennys\_84](https://discuss.elastic.co/u/dennys_84)\ **Replies:** 6\ **Last updated:** [January 30, 2020, 8:02am UTC](https://discuss.elastic.co/t/logstash-monitoring-problem/215468 "2020-01-30T08:02:33Z")
Hi all. We are trying to use stack monitoring feature. But after we enabled internal monitoring we see only Elasticsearch and Kibana instances. Entering SETUP mode show Logstash and Beats with empty values and drops err… --- ## [How to parse message with random data using logstash filter?](https://discuss.elastic.co/t/how-to-parse-message-with-random-data-using-logstash-filter/216453)
**Author:** [@Amit\_Agarwal1](https://discuss.elastic.co/u/Amit_Agarwal1)\ **Replies:** 7\ **Last updated:** [January 30, 2020, 5:59am UTC](https://discuss.elastic.co/t/how-to-parse-message-with-random-data-using-logstash-filter/216453 "2020-01-30T05:59:22Z")
I have received the below message from the producer at Axon Topic with some random ASCII data. "message" =\> "\\u0000\\u0000\\u0000\\u0002�\\u0004C1�\\u0001\\u0002�\\u0003\\u0000\\u0000\\u0000\\u0000A�\\u0003{"prdctId":"10077","id":"… --- ## [Change log level Logstash kafka input plugin?](https://discuss.elastic.co/t/change-log-level-logstash-kafka-input-plugin/214426)
**Author:** [@SKumarMN](https://discuss.elastic.co/u/SKumarMN)\ **Replies:** 3\ **Last updated:** [January 30, 2020, 4:25am UTC](https://discuss.elastic.co/t/change-log-level-logstash-kafka-input-plugin/214426 "2020-01-30T04:25:06Z")
I am using kafka input plugin and logs are getting filled with the below msgs \[2020-01-09T17:28:16,750\]\[WARN \]\[org.apache.kafka.clients.NetworkClient\] \[Consumer clientId=logstash-0, groupId=logstash\] Error while fetch… --- ## [How can i schedule logstash every second for file input plugin?](https://discuss.elastic.co/t/how-can-i-schedule-logstash-every-second-for-file-input-plugin/216605)
**Author:** [@Gunawan\_Kevin](https://discuss.elastic.co/u/Gunawan_Kevin)\ **Replies:** 4\ **Last updated:** [January 30, 2020, 2:13am UTC](https://discuss.elastic.co/t/how-can-i-schedule-logstash-every-second-for-file-input-plugin/216605 "2020-01-30T02:13:02Z")
Hi All I want to make schedule in file input like jdbc input and i already used start\_interval and discover\_interval in my file input but still didn't work. is there another way for make file input schedule in every 5… --- ## [Switching from bitnami to elastic](https://discuss.elastic.co/t/switching-from-bitnami-to-elastic/217080)
**Author:** [@griffer98](https://discuss.elastic.co/u/griffer98)\ **Replies:** 0\ **Last updated:** [January 29, 2020, 9:47pm UTC](https://discuss.elastic.co/t/switching-from-bitnami-to-elastic/217080 "2020-01-29T21:47:48Z")
Hello, I have the ELK stack from bitnami. I have learned that this does not support xpack. so when I go to the elastic docs to install the regular logstash, the binaries are from the same page as from bitnami. I only r… --- ## [Utilize Data Parsed From Nested JSON](https://discuss.elastic.co/t/utilize-data-parsed-from-nested-json/216339)
**Author:** [@mfisher](https://discuss.elastic.co/u/mfisher)\ **Replies:** 5\ **Last updated:** [January 29, 2020, 9:05pm UTC](https://discuss.elastic.co/t/utilize-data-parsed-from-nested-json/216339 "2020-01-29T21:05:26Z")
So I am parsing some files that contain data for devices that are stored as single line nested JSON. I am able to parse the data from the files using the the "json filter" logstash filter. My logstash .conf # READ - … --- ## [Extract From a Field Name and Add Fields](https://discuss.elastic.co/t/extract-from-a-field-name-and-add-fields/216911)
**Author:** [@mfisher](https://discuss.elastic.co/u/mfisher)\ **Replies:** 5\ **Last updated:** [January 29, 2020, 5:53pm UTC](https://discuss.elastic.co/t/extract-from-a-field-name-and-add-fields/216911 "2020-01-29T17:53:19Z")
I need to copy the dynamic "MODEL" from the field name and copy the value "RandomSerial" to new fields. When "MODEL" and "RandomSerial" will be different for all messages. Logstash Output: "Serial Number (MODEL)" =\> … --- ## [Apply grok pattern based on logfile name](https://discuss.elastic.co/t/apply-grok-pattern-based-on-logfile-name/216901)
**Author:** [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)\ **Replies:** 10\ **Last updated:** [January 29, 2020, 4:21pm UTC](https://discuss.elastic.co/t/apply-grok-pattern-based-on-logfile-name/216901 "2020-01-29T16:21:59Z")
Under one index I have multiple log files and one of those log file will use a different grok format than the other three log files. How can I apply grok pattern based on if statement on file name? For example, - type: … --- ## [Alerting on no log entires](https://discuss.elastic.co/t/alerting-on-no-log-entires/216955)
**Author:** [@RichardH](https://discuss.elastic.co/u/RichardH)\ **Replies:** 1\ **Last updated:** [January 29, 2020, 3:06pm UTC](https://discuss.elastic.co/t/alerting-on-no-log-entires/216955 "2020-01-29T15:06:26Z")
Hi All, I am putting this under Logstash as I am reasonably sure (but not totally sure) I will need Logstash to achieve this. We have around 170 servers all running the same application. On each of these servers there … --- ## [Query Logstash error type](https://discuss.elastic.co/t/query-logstash-error-type/216934)
**Author:** [@bruno\_aleixo](https://discuss.elastic.co/u/bruno_aleixo)\ **Replies:** 2\ **Last updated:** [January 29, 2020, 2:51pm UTC](https://discuss.elastic.co/t/query-logstash-error-type/216934 "2020-01-29T14:51:49Z")
Hi, First, sorry my english not be good, I made a jdbc connection in the logstash, collecting information from an oracle bank, the query consists of searching the size of the bank and the name of the instance in a sin… --- ## [Problem with Aggregate Filter](https://discuss.elastic.co/t/problem-with-aggregate-filter/216908)
**Author:** [@codecounselor](https://discuss.elastic.co/u/codecounselor)\ **Replies:** 5\ **Last updated:** [January 29, 2020, 2:50pm UTC](https://discuss.elastic.co/t/problem-with-aggregate-filter/216908 "2020-01-29T14:50:57Z")
I am trying to aggregate all of the log entries for database deadlocks into a single entry. Right now I'm focusing on MySQL's log output, but I need to so something very similar for MSSQL. In comments I have the first … --- ## [Logstash CSV Kibana input error](https://discuss.elastic.co/t/logstash-csv-kibana-input-error/216948)
**Author:** [@ELK4Life](https://discuss.elastic.co/u/ELK4Life)\ **Replies:** 6\ **Last updated:** [January 29, 2020, 2:32pm UTC](https://discuss.elastic.co/t/logstash-csv-kibana-input-error/216948 "2020-01-29T14:32:40Z")
Dear Logstash community, I have challenged myself to capture my network data using TShark and make custom dashboards in Kibana. I have the following setup: 1 Ubuntu 18.04 VM with ElasticSearch, Logstash and Kibana do… --- ## [Logstash beginner configuration issues](https://discuss.elastic.co/t/logstash-beginner-configuration-issues/216692)
**Author:** [@Marie-Laure\_Aix](https://discuss.elastic.co/u/Marie-Laure_Aix)\ **Replies:** 10\ **Last updated:** [January 29, 2020, 2:00pm UTC](https://discuss.elastic.co/t/logstash-beginner-configuration-issues/216692 "2020-01-29T14:00:08Z")
Hi there, I am a complete beginner with Logstash and I have issues configurating it. I am getting this error message : Thread.exclusive is deprecated, use Thread::Mutex Sending Logstash logs to /var/log/logstash which… --- ## [Not able to create index using logstash for my ES cluster](https://discuss.elastic.co/t/not-able-to-create-index-using-logstash-for-my-es-cluster/216654)
**Author:** [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Replies:** 17\ **Last updated:** [January 29, 2020, 1:24pm UTC](https://discuss.elastic.co/t/not-able-to-create-index-using-logstash-for-my-es-cluster/216654 "2020-01-29T13:24:39Z")
Hello, I am using jdbc input plugin with logstash to fetch data from database and create index in ES. But when i am trying to run config file it is not getting started. I am having 3 node cluster with one master node … --- ## [Logstash with puppet module and OpenJDK gives install error](https://discuss.elastic.co/t/logstash-with-puppet-module-and-openjdk-gives-install-error/216794)
**Author:** [@pw64](https://discuss.elastic.co/u/pw64)\ **Replies:** 2\ **Last updated:** [January 29, 2020, 9:58am UTC](https://discuss.elastic.co/t/logstash-with-puppet-module-and-openjdk-gives-install-error/216794 "2020-01-29T09:58:07Z")
We just switched from Oracle Java to OpenJDK for licensing issues. The same Puppet code now results in an installation error: Notice: /Stage\[main\]/Logstash::Service/Exec\[logstash-system-install\]/returns: Unrecognized V… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=411) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=413)