# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=76 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 77 --- ## [Parsing error in date format](https://discuss.elastic.co/t/parsing-error-in-date-format/333466)
**Author:** [@Sachchan](https://discuss.elastic.co/u/Sachchan)\ **Replies:** 2\ **Last updated:** [May 15, 2023, 4:47pm UTC](https://discuss.elastic.co/t/parsing-error-in-date-format/333466 "2023-05-15T16:47:28Z")
Hi Team getting below error in parsing the date in logstash. Kindly suggest how this can be resolved. "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse field \[ResponseTime\] of type \[date\] in docu… --- ## [Pipeline not working in logstash / very strange work of logstash](https://discuss.elastic.co/t/pipeline-not-working-in-logstash-very-strange-work-of-logstash/332818)
**Author:** [@San9](https://discuss.elastic.co/u/San9)\ **Replies:** 2\ **Last updated:** [May 15, 2023, 12:26pm UTC](https://discuss.elastic.co/t/pipeline-not-working-in-logstash-very-strange-work-of-logstash/332818 "2023-05-15T12:26:10Z")
I've encountered strange behavior of Lostash. I have a configuration that reads files locally on the server, then analyzes them and then poisons them into elastic. When I test the config everything works for me /usr/sha… --- ## [Logstash/Kibana : time field incorrect](https://discuss.elastic.co/t/logstash-kibana-time-field-incorrect/333303)
**Author:** [@JackieLaFrite](https://discuss.elastic.co/u/JackieLaFrite)\ **Replies:** 2\ **Last updated:** [May 15, 2023, 7:51am UTC](https://discuss.elastic.co/t/logstash-kibana-time-field-incorrect/333303 "2023-05-15T07:51:30Z")
I've been stuck for 4 days on this problem. The logs that appear in Kibana have their field time changed (+2 hours) + the logs that appear in kibana are logs from two hours ago. It's currently 14h10, here is my last lo… --- ## [Logstash configuration when failing to handle message](https://discuss.elastic.co/t/logstash-configuration-when-failing-to-handle-message/333406)
**Author:** [@TheZadok42](https://discuss.elastic.co/u/TheZadok42)\ **Replies:** 0\ **Last updated:** [May 15, 2023, 7:02am UTC](https://discuss.elastic.co/t/logstash-configuration-when-failing-to-handle-message/333406 "2023-05-15T07:02:10Z")
Hi! Recently I started to integrate logstash into our infrastructure, and while reading the documentation I didn’t quite understand how to handle bad messages. My current flow is as follows: Application -\> rabbitmq -\> … --- ## [Autodetect\_column\_names is not working as expected in csv filter plugin](https://discuss.elastic.co/t/autodetect-column-names-is-not-working-as-expected-in-csv-filter-plugin/333269)
**Author:** [@vladislav](https://discuss.elastic.co/u/vladislav)\ **Replies:** 2\ **Last updated:** [May 15, 2023, 6:39am UTC](https://discuss.elastic.co/t/autodetect-column-names-is-not-working-as-expected-in-csv-filter-plugin/333269 "2023-05-15T06:39:13Z")
Hi, I have this logstash .conf file: input { file { path =\> "/eee/\*.csv" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { csv { autodetect\_column\_names =\> true } } And multi… --- ## [Data enrichment using logstash with translate plugin](https://discuss.elastic.co/t/data-enrichment-using-logstash-with-translate-plugin/333403)
**Author:** [@gpandey7](https://discuss.elastic.co/u/gpandey7)\ **Replies:** 0\ **Last updated:** [May 15, 2023, 6:37am UTC](https://discuss.elastic.co/t/data-enrichment-using-logstash-with-translate-plugin/333403 "2023-05-15T06:37:29Z")
I am trying to enrich the data before it gets indexed, I have tried the below methods but both are currently not working Using the elasticsearch plugin in filter input { kafka { bootstrap\_servers =\> "x… --- ## [SnakeYAML vulnerability with latest Logstash version](https://discuss.elastic.co/t/snakeyaml-vulnerability-with-latest-logstash-version/333332)
**Author:** [@Nikhil\_Khurana](https://discuss.elastic.co/u/Nikhil_Khurana)\ **Replies:** 1\ **Last updated:** [May 14, 2023, 11:29pm UTC](https://discuss.elastic.co/t/snakeyaml-vulnerability-with-latest-logstash-version/333332 "2023-05-14T23:29:26Z")
Hi, In the latest version of Logstash, SnakeYAML dependency was bumped to 1.33 but it seems that is vulnerable as well. The vulnerability CVE-2022-1471 is a critical one with score of 9.8. Are there plans to bump it to… --- ## [Getting An unknown error occurred sending a bulk request to Elasticsearch](https://discuss.elastic.co/t/getting-an-unknown-error-occurred-sending-a-bulk-request-to-elasticsearch/333378)
**Author:** [@Manjiri](https://discuss.elastic.co/u/Manjiri)\ **Replies:** 1\ **Last updated:** [May 14, 2023, 5:33pm UTC](https://discuss.elastic.co/t/getting-an-unknown-error-occurred-sending-a-bulk-request-to-elasticsearch/333378 "2023-05-14T17:33:42Z")
After Starting the logstash the logs are fetching for 5 mins after that in logstash facing below error : An unknown error occurred sending a bulk request to Elasticsearch (will retry indefinitely) {:message=\> "incompati… --- ## [Logstash: Logevent when shutting down but not when starting up](https://discuss.elastic.co/t/logstash-logevent-when-shutting-down-but-not-when-starting-up/333146)
**Author:** [@bitnapper](https://discuss.elastic.co/u/bitnapper)\ **Replies:** 5\ **Last updated:** [May 14, 2023, 10:54am UTC](https://discuss.elastic.co/t/logstash-logevent-when-shutting-down-but-not-when-starting-up/333146 "2023-05-14T10:54:50Z")
Hi, simple question. Logstash produces a log-event when shutting down but not when starting up. Can I make it do that without activating the whol debug log? Regards --- ## [Logstash Limits](https://discuss.elastic.co/t/logstash-limits/331353)
**Author:** [@shushuu](https://discuss.elastic.co/u/shushuu)\ **Replies:** 4\ **Last updated:** [May 13, 2023, 12:32pm UTC](https://discuss.elastic.co/t/logstash-limits/331353 "2023-05-13T12:32:13Z")
Hi, We would like to use Logstash to receive log messages from multiple services (nxlog) and send them further to Elastic. i.e. using this architecture - but with nxlog instead of Beats: What are the limits of a si… --- ## [About ELK STack](https://discuss.elastic.co/t/about-elk-stack/333296)
**Author:** [@Anil\_Sai\_Pinnelli](https://discuss.elastic.co/u/Anil_Sai_Pinnelli)\ **Replies:** 1\ **Last updated:** [May 12, 2023, 3:54pm UTC](https://discuss.elastic.co/t/about-elk-stack/333296 "2023-05-12T15:54:55Z")
Commands to link Mysql DB to elasticsearch using logstash. I am having one configuration file but, it did'nt worked for me!! input { jdbc { jdbc\_driver\_library =\> "/root/mysql-connector-java-5.1.30-bin.jar" jdbc\_dri… --- ## [Parse Array of JSON object](https://discuss.elastic.co/t/parse-array-of-json-object/333034)
**Author:** [@Nurm](https://discuss.elastic.co/u/Nurm)\ **Replies:** 4\ **Last updated:** [May 12, 2023, 7:10am UTC](https://discuss.elastic.co/t/parse-array-of-json-object/333034 "2023-05-12T07:10:55Z")
input { jdbc { jdbc\_connection\_string =\> "jdbc:postgresql://localhost:5432/db" jdbc\_user =\> "user" jdbc\_password =\> "pass" jdbc\_driver\_library =\> "/usr/share/logstash/lib/postgresql-42… --- ## [I want to split from filed value using logstash](https://discuss.elastic.co/t/i-want-to-split-from-filed-value-using-logstash/333059)
**Author:** [@dharminfadia](https://discuss.elastic.co/u/dharminfadia)\ **Replies:** 7\ **Last updated:** [May 12, 2023, 6:12am UTC](https://discuss.elastic.co/t/i-want-to-split-from-filed-value-using-logstash/333059 "2023-05-12T06:12:50Z")
@warkolm @Badger help me.... Hello Everyone I am trying to split recipient-status feild first 3 digit and want to add in to new feild I tried mutate split and add filed but no luck can any one suggest how I can achiv… --- ## [Logstash error connecting to ElasticSearch](https://discuss.elastic.co/t/logstash-error-connecting-to-elasticsearch/333168)
**Author:** [@audric\_w](https://discuss.elastic.co/u/audric_w)\ **Replies:** 3\ **Last updated:** [May 11, 2023, 10:29pm UTC](https://discuss.elastic.co/t/logstash-error-connecting-to-elasticsearch/333168 "2023-05-11T22:29:15Z")
I've tried to created sidecar using beats and logstash on OpenShift. However the logstash always attempted to resurrect connection to dead ES instance (to http://elastisearch:9200), despite configs that I've done. Logst… --- ## [Version conflict, document already exists (current version \[1\])](https://discuss.elastic.co/t/version-conflict-document-already-exists-current-version-1/333107)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 11\ **Last updated:** [May 11, 2023, 8:46pm UTC](https://discuss.elastic.co/t/version-conflict-document-already-exists-current-version-1/333107 "2023-05-11T20:46:01Z")
I am running metricbeat on few system. sending that data to proxy server. proxy then sends data to two logstash servers logstash then parse this and stores records in Elasticsearch. I am creating my own \_id for each … --- ## [Question logstash | Events received vs Event emitted](https://discuss.elastic.co/t/question-logstash-events-received-vs-event-emitted/333219)
**Author:** [@ahmed\_charafouddine](https://discuss.elastic.co/u/ahmed_charafouddine)\ **Replies:** 0\ **Last updated:** [May 11, 2023, 3:20pm UTC](https://discuss.elastic.co/t/question-logstash-events-received-vs-event-emitted/333219 "2023-05-11T15:20:04Z")
Hello, On the monitoring part of my logstash instance, I see that I have 1.3b of events received against 784.7m events emitted. can the fact that I drop certain messages in my pipeline explain this phenomenon or is it r… --- ## [Fastest way to ingest CSV's with logstash to elasticsearch](https://discuss.elastic.co/t/fastest-way-to-ingest-csvs-with-logstash-to-elasticsearch/333118)
**Author:** [@Security\_Check](https://discuss.elastic.co/u/Security_Check)\ **Replies:** 8\ **Last updated:** [May 11, 2023, 3:19pm UTC](https://discuss.elastic.co/t/fastest-way-to-ingest-csvs-with-logstash-to-elasticsearch/333118 "2023-05-11T15:19:24Z")
I'm currently trying to ingest 100gb of csv files into elasticsearch through logstash. The issue is it's taking forever. I have narrowed down the columns I'm trying to filter for to 8 out of 71 but it still takes a long … --- ## [Multiple matches required](https://discuss.elastic.co/t/multiple-matches-required/333192)
**Author:** [@Jason\_Hall](https://discuss.elastic.co/u/Jason_Hall)\ **Replies:** 1\ **Last updated:** [May 11, 2023, 3:10pm UTC](https://discuss.elastic.co/t/multiple-matches-required/333192 "2023-05-11T15:10:31Z")
I am currently setting up some filters for my incoming Watchguard Firewall logs. The logs come in various different formats so i have to setup multiple match rules. My current filter is filter { #Watchguard logs filter… --- ## [Need to split in form of key & value](https://discuss.elastic.co/t/need-to-split-in-form-of-key-value/333218)
**Author:** [@ZERO\_COOL](https://discuss.elastic.co/u/ZERO_COOL)\ **Replies:** 0\ **Last updated:** [May 11, 2023, 2:40pm UTC](https://discuss.elastic.co/t/need-to-split-in-form-of-key-value/333218 "2023-05-11T14:40:09Z")
I am getting event as below. "rusage" =\> \[ \[0\] "", \[1\] "\[mem=10000,mem=5000,VCS-BASE-RUNTIME=1\]" \], I want the value of mem as res\_mem higher one among two keys with "mem" as new field. output: { res\_mem = 10000 … --- ## [If statement performance question](https://discuss.elastic.co/t/if-statement-performance-question/333210)
**Author:** [@mgotechlock](https://discuss.elastic.co/u/mgotechlock)\ **Replies:** 0\ **Last updated:** [May 11, 2023, 12:43pm UTC](https://discuss.elastic.co/t/if-statement-performance-question/333210 "2023-05-11T12:43:59Z")
Question If I use this IF statement, if ("FTNTFGTpolicyname" in \[message\]) or ("FTNTFGTlogid" in \[message\]) {, the CPU of the logstash server spikes to very high, pretty much forever. If I change it to this, CPU is … --- ## [Grok regex match after CSV filter: unable to add a new field from grok match in logstash](https://discuss.elastic.co/t/grok-regex-match-after-csv-filter-unable-to-add-a-new-field-from-grok-match-in-logstash/333206)
**Author:** [@rj.elkadmin](https://discuss.elastic.co/u/rj.elkadmin)\ **Replies:** 0\ **Last updated:** [May 11, 2023, 12:24pm UTC](https://discuss.elastic.co/t/grok-regex-match-after-csv-filter-unable-to-add-a-new-field-from-grok-match-in-logstash/333206 "2023-05-11T12:24:26Z")
Hi team, I am new to here, i apologize for any inconvenient. I am looking for some help on my issue here, kindly assist. My requirement is to process data from csv files located in s3 bucket using Logstash and ingest i… --- ## [How to combine two records into one with logstash and call a filter script before save into Elasticsearch](https://discuss.elastic.co/t/how-to-combine-two-records-into-one-with-logstash-and-call-a-filter-script-before-save-into-elasticsearch/332957)
**Author:** [@liusanyong](https://discuss.elastic.co/u/liusanyong)\ **Replies:** 3\ **Last updated:** [May 11, 2023, 12:23pm UTC](https://discuss.elastic.co/t/how-to-combine-two-records-into-one-with-logstash-and-call-a-filter-script-before-save-into-elasticsearch/332957 "2023-05-11T12:23:38Z")
Hi, I want to do some aggregation and transformation with logstash for input data stream as following steps: Combine two input metric events for a single transaction coming from transaction server and database into o… --- ## [Getting error "Could not index event to Elasticsearch" in logstash?](https://discuss.elastic.co/t/getting-error-could-not-index-event-to-elasticsearch-in-logstash/333198)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 1\ **Last updated:** [May 11, 2023, 11:47am UTC](https://discuss.elastic.co/t/getting-error-could-not-index-event-to-elasticsearch-in-logstash/333198 "2023-05-11T11:47:25Z")
Using Logstash version 7.4.3 logstash-filter-json plugin. filter { json { source =\> "message" } } logs are:- {"Event":"SparkListenerJobStart","Job ID":1,"Submission Time":1640751467318,"Stage Infos":\[{"Stage ID":… --- ## [How to disable a plugin in Logstash Configuration file](https://discuss.elastic.co/t/how-to-disable-a-plugin-in-logstash-configuration-file/333197)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 0\ **Last updated:** [May 11, 2023, 11:00am UTC](https://discuss.elastic.co/t/how-to-disable-a-plugin-in-logstash-configuration-file/333197 "2023-05-11T11:00:52Z")
Hello, I have a configuration file with multiple plugins. I want to disable all plugin and run 1 plugin for some use cases...How can I do that. My config example- input { http\_poller { urls =\> { api1=\> { … --- ## [Logstash jdbc Illegal instant due to time zone offset transition (daylight savings time 'gap'): 1979-03-21](https://discuss.elastic.co/t/logstash-jdbc-illegal-instant-due-to-time-zone-offset-transition-daylight-savings-time-gap-1979-03-21/332902)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 1\ **Last updated:** [May 11, 2023, 5:59am UTC](https://discuss.elastic.co/t/logstash-jdbc-illegal-instant-due-to-time-zone-offset-transition-daylight-savings-time-gap-1979-03-21/332902 "2023-05-11T05:59:11Z")
Hi Here is the logstash jdbc input config: Logstash conf: input { jdbc { jdbc\_driver\_library =\> "/opt/jdbc/ifxjdbc.jar" jdbc\_driver\_class =\> "com.informix.jdbc.IfxDriver" jdbc\_connection\_string =\> "jdbc:… --- ## [Can filebeat recognize .gz log files?](https://discuss.elastic.co/t/can-filebeat-recognize-gz-log-files/332961)
**Author:** [@talka](https://discuss.elastic.co/u/talka)\ **Replies:** 1\ **Last updated:** [May 11, 2023, 2:49am UTC](https://discuss.elastic.co/t/can-filebeat-recognize-gz-log-files/332961 "2023-05-11T02:49:18Z")
Hi, I'm using filebeat version 8.7.0. /var/log list the following files: -rwxrwxrwx 1 1000 1000 244631 Mar 21 06:30 cron -rwxrwxrwx 1 1000 1000 48940 Feb 26 03:37 cron-20230226.gz -rwxrwxrwx 1 1000 1000 48766 Mar … --- ## [Logtash with saml](https://discuss.elastic.co/t/logtash-with-saml/333131)
**Author:** [@Pablo\_Crosio](https://discuss.elastic.co/u/Pablo_Crosio)\ **Replies:** 2\ **Last updated:** [May 10, 2023, 9:22pm UTC](https://discuss.elastic.co/t/logtash-with-saml/333131 "2023-05-10T21:22:38Z")
Is it possible to connect with Logtash to OpenSearch using SAML authentication? We are able to integrate with SAML and Azure AD to log in to dashboards but we are unable to connect Logtash to OpenSearch with an Azure AD… --- ## [Duplicating Event To Multiple Indices](https://discuss.elastic.co/t/duplicating-event-to-multiple-indices/332955)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 3\ **Last updated:** [May 10, 2023, 4:18pm UTC](https://discuss.elastic.co/t/duplicating-event-to-multiple-indices/332955 "2023-05-10T16:18:40Z")
I have events coming in with an ID of 123. Is it possible to have this event indexed into two different indices by doing something like below? output { if \[log\] == 123 { elasticsearch { index =\> "123logs" … --- ## [Elasticsearch not updating data from Logstash](https://discuss.elastic.co/t/elasticsearch-not-updating-data-from-logstash/333097)
**Author:** [@VVlad23](https://discuss.elastic.co/u/VVlad23)\ **Replies:** 0\ **Last updated:** [May 10, 2023, 2:51pm UTC](https://discuss.elastic.co/t/elasticsearch-not-updating-data-from-logstash/333097 "2023-05-10T14:51:09Z")
Hello! So it's been some time I've spent trying to figure out what exactly is happening and why there is a problem. We're sending information from a server through Filebeat to Logstash. Logstash is installed on one of … --- ## [Elasticsearch / logstash Log time shift](https://discuss.elastic.co/t/elasticsearch-logstash-log-time-shift/332898)
**Author:** [@JackieLaFrite](https://discuss.elastic.co/u/JackieLaFrite)\ **Replies:** 6\ **Last updated:** [May 10, 2023, 12:28pm UTC](https://discuss.elastic.co/t/elasticsearch-logstash-log-time-shift/332898 "2023-05-10T12:28:54Z")
I currently have a small problem and I don't know why it happens. I have my log 2023-05-09 09:20:11 \[DEBUG\] org.apache.activemq.transport.AbstractInactivityMonitor:150 -\> WriteChecker: 10000ms elapsed since last write … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=75) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=77)