# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=156 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 157 --- ## [Logstash Date parsing error](https://discuss.elastic.co/t/logstash-date-parsing-error/298780)
**Author:** [@userR](https://discuss.elastic.co/u/userR)\ **Replies:** 11\ **Last updated:** [March 4, 2022, 10:11pm UTC](https://discuss.elastic.co/t/logstash-date-parsing-error/298780 "2022-03-04T22:11:27Z")
Hi! I am trying to parse the following date format: 2022-03-03 10:45:02,520 My current configuration for logstash to parse is: grok { match =\> \[ "message", "%{TIMESTAMP\_ISO8601:logdate}....." \] } d… --- ## [Event aggregation with logstash error due to mixed task\_id](https://discuss.elastic.co/t/event-aggregation-with-logstash-error-due-to-mixed-task-id/298705)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 3\ **Last updated:** [March 4, 2022, 4:02pm UTC](https://discuss.elastic.co/t/event-aggregation-with-logstash-error-due-to-mixed-task-id/298705 "2022-03-04T16:02:53Z")
Hi all I have a case like this: my email log has a task\_id i called it event.id due to how the log was configed now all the log are seperated and only joined by the event.id field. Now i've tried to use the aggregate … --- ## [Logstash 8.0 is adding extra field](https://discuss.elastic.co/t/logstash-8-0-is-adding-extra-field/297594)
**Author:** [@kannan\_raj](https://discuss.elastic.co/u/kannan_raj)\ **Replies:** 3\ **Last updated:** [March 4, 2022, 3:50pm UTC](https://discuss.elastic.co/t/logstash-8-0-is-adding-extra-field/297594 "2022-03-04T15:50:19Z")
logstash consuming JSON messages from Kafka and indexing them into Elasticsearch. we found that logstash version 8.0 is adding the additional field event with the value of the actual message. could you please let me kno… --- ## [Deserialization of results in the Output section](https://discuss.elastic.co/t/deserialization-of-results-in-the-output-section/298634)
**Author:** [@puneet\_makhija](https://discuss.elastic.co/u/puneet_makhija)\ **Replies:** 6\ **Last updated:** [March 4, 2022, 2:43pm UTC](https://discuss.elastic.co/t/deserialization-of-results-in-the-output-section/298634 "2022-03-04T14:43:22Z")
@DavidTurner hope you are doing well can you please assist me into this one , I am stuck in this one I successfully implemented LogStash, here below is my logstash.conf code which is working perfectly input { kafka {… --- ## [Language matters - blacklist\_ and whitelist\_](https://discuss.elastic.co/t/language-matters-blacklist-and-whitelist/298803)
**Author:** [@eclctcjp](https://discuss.elastic.co/u/eclctcjp)\ **Replies:** 2\ **Last updated:** [March 4, 2022, 1:06pm UTC](https://discuss.elastic.co/t/language-matters-blacklist-and-whitelist/298803 "2022-03-04T13:06:10Z")
For the prune filter, please change to more neutral language the 4 values: blacklist\_names blacklist\_values whitelist\_names whitelist\_values I noticed this post, but it looks like we missed the settings in "prune" --- ## [Logstash + XML + Multiple split](https://discuss.elastic.co/t/logstash-xml-multiple-split/298821)
**Author:** [@Stek\_new](https://discuss.elastic.co/u/Stek_new)\ **Replies:** 1\ **Last updated:** [March 4, 2022, 9:44am UTC](https://discuss.elastic.co/t/logstash-xml-multiple-split/298821 "2022-03-04T09:44:06Z")
Hello! Wonder if someone could help me how to divide my XML into several events for their further parsing I have the following XML file (yes, everything in one row): \\\ **Author:** [@ivanchak](https://discuss.elastic.co/u/ivanchak)\ **Replies:** 3\ **Last updated:** [March 4, 2022, 4:30am UTC](https://discuss.elastic.co/t/does-logstash-persistent-queue-support-syslog-plugin/298809 "2022-03-04T04:30:42Z") Dear all, I have a pipeline that has been configured to use the syslog plugin as its input data source. And I would like to enable the Persistent Queue feature to prevent potential data loss due to a crash on Logstash. … --- ## [+HeapDumpOnOutOfMemoryError](https://discuss.elastic.co/t/heapdumponoutofmemoryerror/298786)
**Author:** [@d6036de2b54af16665f4](https://discuss.elastic.co/u/d6036de2b54af16665f4)\ **Replies:** 1\ **Last updated:** [March 3, 2022, 11:27pm UTC](https://discuss.elastic.co/t/heapdumponoutofmemoryerror/298786 "2022-03-03T23:27:54Z")
Hello I am getting Heapdump out of memory error on my Production.. DUe to which i am getting incosistant data on kibana Can you please help me to solve this.. I will send you error messages It's Instance type is t3a.… --- ## [Problem with Logstash SSL](https://discuss.elastic.co/t/problem-with-logstash-ssl/298084)
**Author:** [@gustavoluza](https://discuss.elastic.co/u/gustavoluza)\ **Replies:** 18\ **Last updated:** [March 3, 2022, 8:50pm UTC](https://discuss.elastic.co/t/problem-with-logstash-ssl/298084 "2022-03-03T20:50:26Z")
hi, im having difficulties when changing my logstash SSL certificate, today i have a certificate that is in the /etc/logstash folder and i generated a new one through the elastic tool, using Elasticsearch-certutil and th… --- ## [LS 7.16.3 OutOfMemory jruby.RubyHash WatchedFilesCollection](https://discuss.elastic.co/t/ls-7-16-3-outofmemory-jruby-rubyhash-watchedfilescollection/298629)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 10\ **Last updated:** [March 3, 2022, 7:14pm UTC](https://discuss.elastic.co/t/ls-7-16-3-outofmemory-jruby-rubyhash-watchedfilescollection/298629 "2022-03-03T19:14:43Z")
My Logstash instances recently have started running OutOfMemory and need some help in identifying the cause. My logstash.conf has not been changed for a long time but I did do a few Logstash version upgrades in the past… --- ## [Jdbc\_static logstash plugin, multiple loaders, Parameter field not found in event](https://discuss.elastic.co/t/jdbc-static-logstash-plugin-multiple-loaders-parameter-field-not-found-in-event/298764)
**Author:** [@Q11](https://discuss.elastic.co/u/Q11)\ **Replies:** 1\ **Last updated:** [March 3, 2022, 5:28pm UTC](https://discuss.elastic.co/t/jdbc-static-logstash-plugin-multiple-loaders-parameter-field-not-found-in-event/298764 "2022-03-03T17:28:27Z")
I am using the jdbc\_static logstash plugin, and I want to enrich events with different types of data. I have multiple loaders set, but not all events have the parameter field for each loader. This shuts down the entire… --- ## [Sending log data from rsyslog to logstash](https://discuss.elastic.co/t/sending-log-data-from-rsyslog-to-logstash/298765)
**Author:** [@Khammassi\_HoussemEdd](https://discuss.elastic.co/u/Khammassi_HoussemEdd)\ **Replies:** 0\ **Last updated:** [March 3, 2022, 4:39pm UTC](https://discuss.elastic.co/t/sending-log-data-from-rsyslog-to-logstash/298765 "2022-03-03T16:39:44Z")
i wanted to know whether it's a best practice to send log data DIRECTLY with rsyslog into logstash ?? or is it better to send data from rsyslog to another rsyslog server and and then make logstash operate on this data r… --- ## [Filebeat-netflow to logstash send only specific fields](https://discuss.elastic.co/t/filebeat-netflow-to-logstash-send-only-specific-fields/298761)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 0\ **Last updated:** [March 3, 2022, 4:15pm UTC](https://discuss.elastic.co/t/filebeat-netflow-to-logstash-send-only-specific-fields/298761 "2022-03-03T16:15:57Z")
hello good morning I need your help to send the data from a filebeat-netflow to logstash and filter the fields that I need only to send it to my elastic cloud, I mean, I can specify which fields are sent only, so that I… --- ## [Help to create new field from message](https://discuss.elastic.co/t/help-to-create-new-field-from-message/298626)
**Author:** [@Panplumousse](https://discuss.elastic.co/u/Panplumousse)\ **Replies:** 2\ **Last updated:** [March 3, 2022, 12:36pm UTC](https://discuss.elastic.co/t/help-to-create-new-field-from-message/298626 "2022-03-03T12:36:46Z")
Hello every body, I m looking for a way (maybe with grok, probably with grok :smiley: ), to create new field by extracting some specifique pattern but with keeping the field message without modification after operation … --- ## [Sync Data between MongoDB to ElasticSearch](https://discuss.elastic.co/t/sync-data-between-mongodb-to-elasticsearch/298743)
**Author:** [@mzia87](https://discuss.elastic.co/u/mzia87)\ **Replies:** 0\ **Last updated:** [March 3, 2022, 12:22pm UTC](https://discuss.elastic.co/t/sync-data-between-mongodb-to-elasticsearch/298743 "2022-03-03T12:22:07Z")
Hi Folks, Need to understand something: scenario: if we have 50 million records in our mongodb, then what is the best implementation to sync all the data inside Elasticsearch, like we have to store all the data inside … --- ## [\_dateparsefailure Logstash 7.17](https://discuss.elastic.co/t/dateparsefailure-logstash-7-17/298729)
**Author:** [@Daniel\_Lopez](https://discuss.elastic.co/u/Daniel_Lopez)\ **Replies:** 1\ **Last updated:** [March 3, 2022, 11:50am UTC](https://discuss.elastic.co/t/dateparsefailure-logstash-7-17/298729 "2022-03-03T11:50:35Z")
Hi to all! The field I index to Elasticsearch is 03/03/2020 11:11:11 AM. I tried with different filters date { match =\> \["time","dd/MM/YYYY hh:mm:ss a"\] target =\> "time" } date { match =\> \["time","dd/MM/YYYY hh:m… --- ## [Duplicated message in document, grok pattern problem](https://discuss.elastic.co/t/duplicated-message-in-document-grok-pattern-problem/298704)
**Author:** [@d.silwon](https://discuss.elastic.co/u/d.silwon)\ **Replies:** 2\ **Last updated:** [March 3, 2022, 9:40am UTC](https://discuss.elastic.co/t/duplicated-message-in-document-grok-pattern-problem/298704 "2022-03-03T09:40:13Z")
Dears, Please help me with parsing such multiline log, example one event: 2022-03-03 07:34:45.971 TRACE 865 --- \[nio-5010-exec-7\] rest-server : Request { correlationId = a835e742-635a-4… --- ## [Core vs extended field regarding a mutate error](https://discuss.elastic.co/t/core-vs-extended-field-regarding-a-mutate-error/298653)
**Author:** [@itschobot](https://discuss.elastic.co/u/itschobot)\ **Replies:** 3\ **Last updated:** [March 2, 2022, 8:24pm UTC](https://discuss.elastic.co/t/core-vs-extended-field-regarding-a-mutate-error/298653 "2022-03-02T20:24:25Z")
I'm getting a mutate error where the "host" value is being set to the "device\_hostname"but was wondering if it's because one is a core field and the other is an extended field? Here is what is being applied in my filter… --- ## [Delete previous document](https://discuss.elastic.co/t/delete-previous-document/298602)
**Author:** [@Nasser](https://discuss.elastic.co/u/Nasser)\ **Replies:** 1\ **Last updated:** [March 2, 2022, 6:35pm UTC](https://discuss.elastic.co/t/delete-previous-document/298602 "2022-03-02T18:35:15Z")
Hi i have jdbc input run every min and i want to delete the previous document that inserted in the index once the schedule run how the logstash structure will be ? --- ## [Help for parsing with Grok filter](https://discuss.elastic.co/t/help-for-parsing-with-grok-filter/298635)
**Author:** [@ASRLO](https://discuss.elastic.co/u/ASRLO)\ **Replies:** 1\ **Last updated:** [March 2, 2022, 6:13pm UTC](https://discuss.elastic.co/t/help-for-parsing-with-grok-filter/298635 "2022-03-02T18:13:28Z")
Hello, I would like to split my message field on my logs but I don't master the grok filter at all, can you suggest a split? Here is the message field to cut: Windows Installer a reconfiguré le produit. Nom du produit… --- ## [There is no grok expression that can parse 100% of the following log information](https://discuss.elastic.co/t/there-is-no-grok-expression-that-can-parse-100-of-the-following-log-information/298462)
**Author:** [@jie](https://discuss.elastic.co/u/jie)\ **Replies:** 4\ **Last updated:** [March 2, 2022, 1:10pm UTC](https://discuss.elastic.co/t/there-is-no-grok-expression-that-can-parse-100-of-the-following-log-information/298462 "2022-03-02T13:10:11Z")
hello logtfile 12Jul20 10:18:09.561645: hello IT00S451.uaes.com (port 57276) 12Jul20 10:18:09.561645: info request for enable\_MULTI\_v7 from IT00S451.uaes.com (port 57276) 12Jul20 10:18:09.561645: exit IT00S451.uaes.co… --- ## [Fleet Server - To queue or not to queue, that is the question](https://discuss.elastic.co/t/fleet-server-to-queue-or-not-to-queue-that-is-the-question/298596)
**Author:** [@Luca\_SWU](https://discuss.elastic.co/u/Luca_SWU)\ **Replies:** 2\ **Last updated:** [March 2, 2022, 11:33am UTC](https://discuss.elastic.co/t/fleet-server-to-queue-or-not-to-queue-that-is-the-question/298596 "2022-03-02T11:33:25Z")
Hello Elastic Community, Currently I am doing research about how to deploy Fleet Server and Elastic Agents in a self-managed Production environment. In this environment we would like to have the Elastic Agents and Flee… --- ## [Event.set does not work](https://discuss.elastic.co/t/event-set-does-not-work/298317)
**Author:** [@stemons](https://discuss.elastic.co/u/stemons)\ **Replies:** 3\ **Last updated:** [March 2, 2022, 8:33am UTC](https://discuss.elastic.co/t/event-set-does-not-work/298317 "2022-03-02T08:33:45Z")
Hello, I'm trying to create new field in my filter, but seems that event.set does not work properly. I can't see neither errors nor the new fields in elastic. I had to add the if condition since the field is not contain… --- ## [Logstash csv file](https://discuss.elastic.co/t/logstash-csv-file/298565)
**Author:** [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\ **Replies:** 1\ **Last updated:** [March 1, 2022, 9:39pm UTC](https://discuss.elastic.co/t/logstash-csv-file/298565 "2022-03-01T21:39:32Z")
I have question, How can we tell logstash to import only the first 10 lines from a csv file Thank you --- ## [How to append a string to a value?](https://discuss.elastic.co/t/how-to-append-a-string-to-a-value/298542)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 3\ **Last updated:** [March 1, 2022, 7:53pm UTC](https://discuss.elastic.co/t/how-to-append-a-string-to-a-value/298542 "2022-03-01T19:53:57Z")
Hi, I would like to append the letter C to this key: "module-temp": 32.37 so that it looks like "module-temp": 32.37 C Tried this but didn't work: if ("\*" in \[module-temp\]) { mutate { replace =\> … --- ## [Could not execute action: PipelineAction::Create\](https://discuss.elastic.co/t/could-not-execute-action-pipelineaction-create-main/298454)
**Author:** [@sath](https://discuss.elastic.co/u/sath)\ **Replies:** 6\ **Last updated:** [March 1, 2022, 7:17pm UTC](https://discuss.elastic.co/t/could-not-execute-action-pipelineaction-create-main/298454 "2022-03-01T19:17:41Z")
Hello, Logstash isn't able to create the index, that's what my conclusion is after checking the logs. pipeline.yml - pipeline.id: main path.config: "/etc/logstash/conf.d/1.conf" 1.conf: input { beats { … --- ## [Ruby filter to whitelist nested fields](https://discuss.elastic.co/t/ruby-filter-to-whitelist-nested-fields/298424)
**Author:** [@lordcosmos1978](https://discuss.elastic.co/u/lordcosmos1978)\ **Replies:** 2\ **Last updated:** [March 1, 2022, 9:48am UTC](https://discuss.elastic.co/t/ruby-filter-to-whitelist-nested-fields/298424 "2022-03-01T09:48:27Z")
I am trying to whitelist a number of nested fields. I first tried it using the prune filter but it seems that one does not support nested fields only top level fields. I saw in this topic Logstash prune nested fields t… --- ## [Logstashからelasticsearchにデータを保存できません](https://discuss.elastic.co/t/logstash-elasticsearch/298476)
**Author:** [@tamina](https://discuss.elastic.co/u/tamina)\ **Replies:** 0\ **Last updated:** [March 1, 2022, 7:31am UTC](https://discuss.elastic.co/t/logstash-elasticsearch/298476 "2022-03-01T07:31:46Z")
logstash8.0からelasticsearch8.0にデータを保存しようとしているのですがエラーが出てしまい成功しません。どこを直したらよいでしょうか？ ■発生しているエラー ''' \[2022-03-01T16:11:11,837\]\[WARN \]\[o.e.t.TcpTransport \] \[JPC00006506\] exception caught on transport layer \[Netty4TcpCh… --- ## [Error: java.lang.ClassNotFoundException: com.dbschema.MongoJdbcDriver. Are you sure you've included the correct jdbc driver in :jdbc\_driver\_library?](https://discuss.elastic.co/t/error-java-lang-classnotfoundexception-com-dbschema-mongojdbcdriver-are-you-sure-youve-included-the-correct-jdbc-driver-in-jdbc-driver-library/298460)
**Author:** [@Amal\_Anush](https://discuss.elastic.co/u/Amal_Anush)\ **Replies:** 0\ **Last updated:** [March 1, 2022, 3:04am UTC](https://discuss.elastic.co/t/error-java-lang-classnotfoundexception-com-dbschema-mongojdbcdriver-are-you-sure-youve-included-the-correct-jdbc-driver-in-jdbc-driver-library/298460 "2022-03-01T03:04:03Z")
hi , im using dbschema mongo driver to connect to move data from mongo to logstash . but here is the error Error: java.lang.ClassNotFoundException: com.dbschema.MongoJdbcDriver. Are you sure you've included the correct… --- ## [Filebeat configuration allows multiple pipelines for a log file?](https://discuss.elastic.co/t/filebeat-configuration-allows-multiple-pipelines-for-a-log-file/298458)
**Author:** [@jie](https://discuss.elastic.co/u/jie)\ **Replies:** 0\ **Last updated:** [March 1, 2022, 1:57am UTC](https://discuss.elastic.co/t/filebeat-configuration-allows-multiple-pipelines-for-a-log-file/298458 "2022-03-01T01:57:11Z")
Specifically, I have a pipeline preprocessing log file does not match the log is not saved, I want to not match the log file is saved in the log file data loss. [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=155) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=157)