# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=130 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 131 --- ## [Regarding Date filter format](https://discuss.elastic.co/t/regarding-date-filter-format/308626)
**Author:** [@suresh\_u](https://discuss.elastic.co/u/suresh_u)\ **Replies:** 16\ **Last updated:** [July 1, 2022, 3:50pm UTC](https://discuss.elastic.co/t/regarding-date-filter-format/308626 "2022-07-01T15:50:45Z")
Hi team, I have a field having value as "updatedDate" =\> 2019-12-31T00:27:17.356Z I am trying to convert date format to updatedDate" =\> 2019-12-31 I am using below filter.But getting datetime parse error: date { m… --- ## [Does jdbc\_user get signed out after run of pipeline?](https://discuss.elastic.co/t/does-jdbc-user-get-signed-out-after-run-of-pipeline/307407)
**Author:** [@DazDotOne](https://discuss.elastic.co/u/DazDotOne)\ **Replies:** 5\ **Last updated:** [July 1, 2022, 2:18pm UTC](https://discuss.elastic.co/t/does-jdbc-user-get-signed-out-after-run-of-pipeline/307407 "2022-07-01T14:18:32Z")
Sorry if this is answered elsewhere, I had a look around the web but couldn't really find anything. After a pipeline has executed, does this plugin sign the given user out from the SQL server or retain the connection fo… --- ## [Logstash elasticsearch filter and array of string substitution](https://discuss.elastic.co/t/logstash-elasticsearch-filter-and-array-of-string-substitution/308646)
**Author:** [@Silver137](https://discuss.elastic.co/u/Silver137)\ **Replies:** 1\ **Last updated:** [July 1, 2022, 11:01am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-filter-and-array-of-string-substitution/308646 "2022-07-01T11:01:45Z")
I'm using the logstash elasticsearch filter to make a terms query where i need to substitute an array on the selected query\_template. The query template is as follows: { "size": 5, "query": { "bool": {"filter":… --- ## [Failed to execute action logstash message=\>"Expected one of \[ \\\\t\\\\r\\\\n\], \\"#\\", \\"{\\" at line 9, column 8 (byte 62) after filter {\\n json "](https://discuss.elastic.co/t/failed-to-execute-action-logstash-message-expected-one-of-t-r-n-at-line-9-column-8-byte-62-after-filter-n-json/308559)
**Author:** [@Hemabh\_Ravee](https://discuss.elastic.co/u/Hemabh_Ravee)\ **Replies:** 2\ **Last updated:** [July 1, 2022, 9:04am UTC](https://discuss.elastic.co/t/failed-to-execute-action-logstash-message-expected-one-of-t-r-n-at-line-9-column-8-byte-62-after-filter-n-json/308559 "2022-07-01T09:04:03Z")
I am new to the Elastic stack and tried using the json filter plugin to parse kibana's logs. But adding the filter started throwing this error - Failed to execute action logstash message=\>"Expected one of \[ \\\\t\\\\r\\\\n\],… --- ## [Combining Logstash and RavenDB](https://discuss.elastic.co/t/combining-logstash-and-ravendb/308631)
**Author:** [@AurelioAranzana](https://discuss.elastic.co/u/AurelioAranzana)\ **Replies:** 0\ **Last updated:** [July 1, 2022, 7:25am UTC](https://discuss.elastic.co/t/combining-logstash-and-ravendb/308631 "2022-07-01T07:25:49Z")
Hi everyone! I would like to know if anyone knows if it is possible to implement a pipeline to extract data from RavenDB to export it to another database via logstash. At the moment, I'm used to using the pluging jdbc … --- ## [Logstash Multi pipeline with beats input](https://discuss.elastic.co/t/logstash-multi-pipeline-with-beats-input/308612)
**Author:** [@MM2022](https://discuss.elastic.co/u/MM2022)\ **Replies:** 1\ **Last updated:** [July 1, 2022, 1:45am UTC](https://discuss.elastic.co/t/logstash-multi-pipeline-with-beats-input/308612 "2022-07-01T01:45:03Z")
Hi All, I have one ELK stack and multiple clients to send their data to it. Each client should have their own Index and clients should not be able to see each others data. I am using filebeats on the client servers. the… --- ## [Enable filebeat and logstash caching during unavailablity](https://discuss.elastic.co/t/enable-filebeat-and-logstash-caching-during-unavailablity/308443)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 4\ **Last updated:** [June 30, 2022, 6:25pm UTC](https://discuss.elastic.co/t/enable-filebeat-and-logstash-caching-during-unavailablity/308443 "2022-06-30T18:25:36Z")
Hello, I hope you and your loved ones are safe and healthy. I am running a cluster that collects logs from sources on the internet. I need to enable caching of logs in case the next hop is not reachable as dropping log… --- ## [Input file and input jdbc](https://discuss.elastic.co/t/input-file-and-input-jdbc/308587)
**Author:** [@Calvete](https://discuss.elastic.co/u/Calvete)\ **Replies:** 1\ **Last updated:** [June 30, 2022, 4:34pm UTC](https://discuss.elastic.co/t/input-file-and-input-jdbc/308587 "2022-06-30T16:34:52Z")
Hi I want to get the name of the csv file and then that value use it in a select of another input in database input { file { path =\> "C:/Users/a/Downloads/bucket/\*.csv" start\_position =\> "beginning" sincedb\_path =\> … --- ## [Avoid duplicate document in different Indices,Logsatsh](https://discuss.elastic.co/t/avoid-duplicate-document-in-different-indices-logsatsh/308578)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 1\ **Last updated:** [June 30, 2022, 4:29pm UTC](https://discuss.elastic.co/t/avoid-duplicate-document-in-different-indices-logsatsh/308578 "2022-06-30T16:29:53Z")
Hello All, I have a logstash configuration that uses the following in the output block in an attempt to mitigate duplicates in elastic Index. The data that logstash is fetching from is through perl script running every… --- ## [Logstash grok multiline file](https://discuss.elastic.co/t/logstash-grok-multiline-file/308509)
**Author:** [@alex\_96](https://discuss.elastic.co/u/alex_96)\ **Replies:** 3\ **Last updated:** [June 29, 2022, 9:51pm UTC](https://discuss.elastic.co/t/logstash-grok-multiline-file/308509 "2022-06-29T21:51:53Z")
Hello, I read a lot of articles here, but I still don’t understand how to properly configure grok in logstash, here is an example file: # Time: 2021-02-17T15:19:22.121290Z # User@Host: app\[app\] @ \[192.168.100.1\] Id: 4… --- ## [Logstash not using ingest pipeline with datastreams](https://discuss.elastic.co/t/logstash-not-using-ingest-pipeline-with-datastreams/308384)
**Author:** [@rzisk](https://discuss.elastic.co/u/rzisk)\ **Replies:** 2\ **Last updated:** [June 30, 2022, 1:24pm UTC](https://discuss.elastic.co/t/logstash-not-using-ingest-pipeline-with-datastreams/308384 "2022-06-30T13:24:59Z")
Hello, I recently got our Elasticsearch instances upgraded to the 8 branch am working on getting all of our Beats upgraded to 8 as well. So far there have been various gotchas with getting Logstash to play well with dat… --- ## [Codec Multiline is not working for Kafka input plugin as expected](https://discuss.elastic.co/t/codec-multiline-is-not-working-for-kafka-input-plugin-as-expected/308553)
**Author:** [@ashish.kumar2](https://discuss.elastic.co/u/ashish.kumar2)\ **Replies:** 0\ **Last updated:** [June 30, 2022, 9:56am UTC](https://discuss.elastic.co/t/codec-multiline-is-not-working-for-kafka-input-plugin-as-expected/308553 "2022-06-30T09:56:43Z")
codec =\> multiline { pattern =\> "(^UL ingress)|(^DL ingress)|(^numOfActiveUe)|(^pdcpCurrentTime\_g)" what =\> "previous" max\_lines =\> 6 } Still we are getting indiv… --- ## [Regarding Split filter in mutate](https://discuss.elastic.co/t/regarding-split-filter-in-mutate/308540)
**Author:** [@suresh\_u](https://discuss.elastic.co/u/suresh_u)\ **Replies:** 3\ **Last updated:** [June 30, 2022, 8:29am UTC](https://discuss.elastic.co/t/regarding-split-filter-in-mutate/308540 "2022-06-30T08:29:07Z")
Hi team, I have one roll number like 1071-P56790-345. First number is 1071. Remaining fields in future can be added. So I need to extract 1071 and P56790-345 into 2 seperate fields. I tried using filter by delimeter … --- ## [Parse date and time to timestamp using grok pattern](https://discuss.elastic.co/t/parse-date-and-time-to-timestamp-using-grok-pattern/308481)
**Author:** [@zain.jutt](https://discuss.elastic.co/u/zain.jutt)\ **Replies:** 5\ **Last updated:** [June 30, 2022, 7:42am UTC](https://discuss.elastic.co/t/parse-date-and-time-to-timestamp-using-grok-pattern/308481 "2022-06-30T07:42:39Z")
Hi, i am trying to extract the information to timestamp from the following log event for that I tried it as 20220628 11:44:29.887 - DEBUG - Trying to connect to the target: I created the pattern this way %{YEAR}%{MONT… --- ## [Prevent logstash from adding additional data on output](https://discuss.elastic.co/t/prevent-logstash-from-adding-additional-data-on-output/307834)
**Author:** [@mikhatanu](https://discuss.elastic.co/u/mikhatanu)\ **Replies:** 6\ **Last updated:** [June 30, 2022, 6:11am UTC](https://discuss.elastic.co/t/prevent-logstash-from-adding-additional-data-on-output/307834 "2022-06-30T06:11:12Z")
Hello, i am using the kafka output and input plugins in logstash. The log route is like this: logstash -\> kafka -\> logstash (indexing) -\> elasticsearch. logstash encodes additional data (such as file path, hostname, etc)… --- ## [Dissect failed finding my field in event](https://discuss.elastic.co/t/dissect-failed-finding-my-field-in-event/308515)
**Author:** [@vincent2mots](https://discuss.elastic.co/u/vincent2mots)\ **Replies:** 2\ **Last updated:** [June 30, 2022, 5:13am UTC](https://discuss.elastic.co/t/dissect-failed-finding-my-field-in-event/308515 "2022-06-30T05:13:22Z")
Hi experts!! I have an index mapping like this : { "talend-esb-dev-2022.06" : { "mappings" : { "properties" : { "customInfo" : { "properties" : { "camel" : { "pro… --- ## [Sending Data From Oracle to Elastic](https://discuss.elastic.co/t/sending-data-from-oracle-to-elastic/308460)
**Author:** [@Dhruvin.patel001](https://discuss.elastic.co/u/Dhruvin.patel001)\ **Replies:** 5\ **Last updated:** [June 29, 2022, 7:08pm UTC](https://discuss.elastic.co/t/sending-data-from-oracle-to-elastic/308460 "2022-06-29T19:08:47Z")
I am trying to send oracle events to Elasticsearch. In oracle DB, we have create\_date column as primary key. Here is the format of create date:- 22-JUL-19 02.22.02.918000000 PM 22-JUL-19 02.22.03.325000000 PM 22-JUL-… --- ## [Logstash: logstash-plugin install logstash-codec-protobuf, message: certificate verify failed](https://discuss.elastic.co/t/logstash-logstash-plugin-install-logstash-codec-protobuf-message-certificate-verify-failed/308504)
**Author:** [@vante](https://discuss.elastic.co/u/vante)\ **Replies:** 0\ **Last updated:** [June 29, 2022, 5:27pm UTC](https://discuss.elastic.co/t/logstash-logstash-plugin-install-logstash-codec-protobuf-message-certificate-verify-failed/308504 "2022-06-29T17:27:17Z")
Hey, i am trying to install protobuf codec plugin for my logstash but got this //ERROR: Something went wrong when installing logstash-codec-protobuf, message: certificate verify failed. Does anyone know how to go about … --- ## [Logstash pipeline for kafka message](https://discuss.elastic.co/t/logstash-pipeline-for-kafka-message/307413)
**Author:** [@harijld](https://discuss.elastic.co/u/harijld)\ **Replies:** 3\ **Last updated:** [June 29, 2022, 4:34pm UTC](https://discuss.elastic.co/t/logstash-pipeline-for-kafka-message/307413 "2022-06-29T16:34:25Z")
Hi Team, Need some guidance to prepare log stash pipeline for below message. I am getting below message from KAFKA topic and i need to stash it into Elasticsearch though log stash. I am able to do it but I want to inser… --- ## [Accessing the previous value in a field in logstash](https://discuss.elastic.co/t/accessing-the-previous-value-in-a-field-in-logstash/308462)
**Author:** [@dishant.sharma](https://discuss.elastic.co/u/dishant.sharma)\ **Replies:** 1\ **Last updated:** [June 29, 2022, 3:47pm UTC](https://discuss.elastic.co/t/accessing-the-previous-value-in-a-field-in-logstash/308462 "2022-06-29T15:47:24Z")
A detailed explanation of the issue is on the above link. --- ## [JDBC Communication error with Logstash](https://discuss.elastic.co/t/jdbc-communication-error-with-logstash/308454)
**Author:** [@Simone1](https://discuss.elastic.co/u/Simone1)\ **Replies:** 1\ **Last updated:** [June 29, 2022, 1:26pm UTC](https://discuss.elastic.co/t/jdbc-communication-error-with-logstash/308454 "2022-06-29T13:26:50Z")
I'm trying to collect data with Logstash from a remote MySQL db and i'm receving the following error: \[ERROR\] 2022-06-29 11:44:57.733 \[\[main\]\"… --- ## [Convert Array to String by ingesting three double quotes](https://discuss.elastic.co/t/convert-array-to-string-by-ingesting-three-double-quotes/308358)
**Author:** [@Elie](https://discuss.elastic.co/u/Elie)\ **Replies:** 2\ **Last updated:** [June 29, 2022, 10:40am UTC](https://discuss.elastic.co/t/convert-array-to-string-by-ingesting-three-double-quotes/308358 "2022-06-29T10:40:31Z")
Hi, I am wanting to change an Array to a String to make it easier for me to store in my Elasticsearch, but I am having trouble finding the way of doing so. There are some special cases in my data where my field "value" … --- ## [Regarding Mutate Ruby filter](https://discuss.elastic.co/t/regarding-mutate-ruby-filter/308426)
**Author:** [@suresh\_u](https://discuss.elastic.co/u/suresh_u)\ **Replies:** 2\ **Last updated:** [June 29, 2022, 9:49am UTC](https://discuss.elastic.co/t/regarding-mutate-ruby-filter/308426 "2022-06-29T09:49:43Z")
I have 1 sample transaction number 10004567-P658910-569876. I want to seperate above transaction number with index 0 as 1 seperate field and remaining as other field. Ex: 10004567-P658910-569876 Num\_field = 10004567 … --- ## [Convert iso 8601 timestamp to UNIX\_MS](https://discuss.elastic.co/t/convert-iso-8601-timestamp-to-unix-ms/306693)
**Author:** [@The-morpho](https://discuss.elastic.co/u/The-morpho)\ **Replies:** 15\ **Last updated:** [June 29, 2022, 9:38am UTC](https://discuss.elastic.co/t/convert-iso-8601-timestamp-to-unix-ms/306693 "2022-06-29T09:38:08Z")
Hello, I have json logs wich have a field timestamp in iso 8601 format I want to convert it to UNIX\_MS so I can calculate the response time between step 1 and 2, 1 and 3 for each mid. How this can be done ? This is an… --- ## [ProducerConfig values not all set](https://discuss.elastic.co/t/producerconfig-values-not-all-set/308438)
**Author:** [@mikhatanu](https://discuss.elastic.co/u/mikhatanu)\ **Replies:** 1\ **Last updated:** [June 29, 2022, 9:23am UTC](https://discuss.elastic.co/t/producerconfig-values-not-all-set/308438 "2022-06-29T09:23:11Z")
Hello, i've been using the kafka plugin and set some ssl settings in the plugin. The config checker (before starting logstash) detects my config values, but the producerConfig (in the log) is not set. my config: input … --- ## [FortiGate-logs exporten from Graylog with GELF into ELK and read by "Fortinet Logs"](https://discuss.elastic.co/t/fortigate-logs-exporten-from-graylog-with-gelf-into-elk-and-read-by-fortinet-logs/308368)
**Author:** [@j0han](https://discuss.elastic.co/u/j0han)\ **Replies:** 0\ **Last updated:** [June 28, 2022, 1:57pm UTC](https://discuss.elastic.co/t/fortigate-logs-exporten-from-graylog-with-gelf-into-elk-and-read-by-fortinet-logs/308368 "2022-06-28T13:57:24Z")
Hi all! I have a problem where I'm getting logs exported from Graylog with GELF. I can read the logs with logstash and I get the data I need exported as JSON. But I'm having big problems to get the integration "Fortin… --- ## [Output to redis with a "sorted set" datatype](https://discuss.elastic.co/t/output-to-redis-with-a-sorted-set-datatype/308401)
**Author:** [@ktomu](https://discuss.elastic.co/u/ktomu)\ **Replies:** 1\ **Last updated:** [June 28, 2022, 9:26pm UTC](https://discuss.elastic.co/t/output-to-redis-with-a-sorted-set-datatype/308401 "2022-06-28T21:26:43Z")
Hi, I wanna find a way to send output to redis with datatype as "sorted set" and in batch mode. As I see now, logstash supports only list and channel as a datatype. Is there any other way how to do it? --- ## [Issue with very long TCP messages](https://discuss.elastic.co/t/issue-with-very-long-tcp-messages/308377)
**Author:** [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\ **Replies:** 3\ **Last updated:** [June 28, 2022, 6:49pm UTC](https://discuss.elastic.co/t/issue-with-very-long-tcp-messages/308377 "2022-06-28T18:49:31Z")
We are trying to ingest data via TCP into logstash and send to elasticsearch. Most messages are fine but sometimes our tool produces very long TCP messages and we end up with the first message getting trimmed and the re… --- ## [Kafka plugin with PEM ssl](https://discuss.elastic.co/t/kafka-plugin-with-pem-ssl/308210)
**Author:** [@mikhatanu](https://discuss.elastic.co/u/mikhatanu)\ **Replies:** 5\ **Last updated:** [June 28, 2022, 4:05pm UTC](https://discuss.elastic.co/t/kafka-plugin-with-pem-ssl/308210 "2022-06-28T16:05:49Z")
Hello, is it possible to use PEM keystore type instead of JKS in the plugin --- ## [Failed to install template](https://discuss.elastic.co/t/failed-to-install-template/307425)
**Author:** [@The-morpho](https://discuss.elastic.co/u/The-morpho)\ **Replies:** 1\ **Last updated:** [June 28, 2022, 2:17pm UTC](https://discuss.elastic.co/t/failed-to-install-template/307425 "2022-06-28T14:17:32Z")
Hello, I'm new with elastic, I'm working with logs that I want to process from them a new fields based on aggregation. But I got this error: \[ERROR\]\[logstash.outputs.elasticsearch\]\[main\] Failed to install template {:m… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=129) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=131)