# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=28 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 29 --- ## [Logstash upgrade issue (Upgrade from 7.17.5 to 8.13.24](https://discuss.elastic.co/t/logstash-upgrade-issue-upgrade-from-7-17-5-to-8-13-24/361399)
**Author:** [@mansoorpn](https://discuss.elastic.co/u/mansoorpn)\ **Replies:** 2\ **Last updated:** [June 14, 2024, 10:19am UTC](https://discuss.elastic.co/t/logstash-upgrade-issue-upgrade-from-7-17-5-to-8-13-24/361399 "2024-06-14T10:19:00Z")
Hello Team, Last week I upgraded Elasticsearch from 7.XX to 8.13.4. The Logstash version is 7.17.5. Everything is working as expected. Now I tried to upgrade Logstash from 7.17.5 to 8.13.4 and it failed with the below … --- ## [Rsyslog to logstash as json -\> extract pattern from message](https://discuss.elastic.co/t/rsyslog-to-logstash-as-json-extract-pattern-from-message/361382)
**Author:** [@vasilev](https://discuss.elastic.co/u/vasilev)\ **Replies:** 2\ **Last updated:** [June 13, 2024, 12:02pm UTC](https://discuss.elastic.co/t/rsyslog-to-logstash-as-json-extract-pattern-from-message/361382 "2024-06-13T12:02:03Z")
Hello all, I am sending rsyslog data to logstash via udp. Here is the logstash configuration: input { udp { port =\> 15044 codec =\> "json" type =\> "rsyslog" } } filter { if \[srvtype\] == "test" { json { source… --- ## [Multiple pipeline on single host not working as expected](https://discuss.elastic.co/t/multiple-pipeline-on-single-host-not-working-as-expected/361330)
**Author:** [@Ajay\_Bhatnagar](https://discuss.elastic.co/u/Ajay_Bhatnagar)\ **Replies:** 2\ **Last updated:** [June 13, 2024, 12:34pm UTC](https://discuss.elastic.co/t/multiple-pipeline-on-single-host-not-working-as-expected/361330 "2024-06-13T12:34:26Z")
When defining multiple pipelines on single host with output to elasticsearch in different indices (each pipeline points its own separate config file, different pipeline id and different output index name defined in confi… --- ## [How http\_poller plugin works in logstash](https://discuss.elastic.co/t/how-http-poller-plugin-works-in-logstash/361379)
**Author:** [@upreddy](https://discuss.elastic.co/u/upreddy)\ **Replies:** 2\ **Last updated:** [June 13, 2024, 12:15pm UTC](https://discuss.elastic.co/t/how-http-poller-plugin-works-in-logstash/361379 "2024-06-13T12:15:19Z")
Hi Team, We are using http\_poller input plugin in our logstash to fetch data from opensearch perticular index and we did some filtering and sending data to kafka servers. Whenever kafka server gets some error from this … --- ## [Best practice for multiple Logstash pipelines writing to Elasticsearch](https://discuss.elastic.co/t/best-practice-for-multiple-logstash-pipelines-writing-to-elasticsearch/361389)
**Author:** [@yago82](https://discuss.elastic.co/u/yago82)\ **Replies:** 0\ **Last updated:** [June 13, 2024, 10:53am UTC](https://discuss.elastic.co/t/best-practice-for-multiple-logstash-pipelines-writing-to-elasticsearch/361389 "2024-06-13T10:53:49Z")
Hello, I'm currently working with multiple Logstash pipelines and I'm trying to determine the best approach for writing data to Elasticsearch. I would appreciate any advice or insights from the community. Here are the … --- ## [MIgrating opensearch to elasticsearch using logstash is generating more docs on destination index than there are in source index](https://discuss.elastic.co/t/migrating-opensearch-to-elasticsearch-using-logstash-is-generating-more-docs-on-destination-index-than-there-are-in-source-index/361275)
**Author:** [@Mat\_Wojdyla](https://discuss.elastic.co/u/Mat_Wojdyla)\ **Replies:** 7\ **Last updated:** [June 12, 2024, 11:32pm UTC](https://discuss.elastic.co/t/migrating-opensearch-to-elasticsearch-using-logstash-is-generating-more-docs-on-destination-index-than-there-are-in-source-index/361275 "2024-06-12T23:32:42Z")
Hello, I am in the process of migrating our opensearch cluster running ES 7.9 as the backend version to an Elasticsearch cluster I built in EC2 that is also running 7.9.3 for the time being. I have the sync running just… --- ## [How do I send Cowrie logs to ELK (with SSL) via Logstash?](https://discuss.elastic.co/t/how-do-i-send-cowrie-logs-to-elk-with-ssl-via-logstash/361159)
**Author:** [@optimuspur3](https://discuss.elastic.co/u/optimuspur3)\ **Replies:** 1\ **Last updated:** [June 12, 2024, 1:59pm UTC](https://discuss.elastic.co/t/how-do-i-send-cowrie-logs-to-elk-with-ssl-via-logstash/361159 "2024-06-12T13:59:11Z")
When I am setting up my ELK without SSL configuration, I was able to setup Cowrie logstash in my logstash-cowrie.log and inside my filebeat.yml file. The link from the ELK with SSL configuration: I have created anothe… --- ## [The error message stack trace log is being printed in separate rows instead of a single row](https://discuss.elastic.co/t/the-error-message-stack-trace-log-is-being-printed-in-separate-rows-instead-of-a-single-row/358626)
**Author:** [@kasecen637](https://discuss.elastic.co/u/kasecen637)\ **Replies:** 4\ **Last updated:** [June 12, 2024, 10:53am UTC](https://discuss.elastic.co/t/the-error-message-stack-trace-log-is-being-printed-in-separate-rows-instead-of-a-single-row/358626 "2024-06-12T10:53:53Z")
Hi Elastic Discuss Community, In kibana when i seee the logs message when an error log messges comes it print every error messge in every different line i want that the error message whole stack trace log should get in … --- ## [Intermittent data loss in kinesis logstash input plugin](https://discuss.elastic.co/t/intermittent-data-loss-in-kinesis-logstash-input-plugin/361312)
**Author:** [@Amol\_Gaitonde1](https://discuss.elastic.co/u/Amol_Gaitonde1)\ **Replies:** 0\ **Last updated:** [June 12, 2024, 9:26am UTC](https://discuss.elastic.co/t/intermittent-data-loss-in-kinesis-logstash-input-plugin/361312 "2024-06-12T09:26:55Z")
We are using kinesis input plugin in logstash to fetch data from cloudwatch. Below is the configuration setting. The issue which we are facing is intermittent data loss without any errors. We are not able to find the r… --- ## [Accidentally deleted http\_ca.crt. how to recover! urgent! please!](https://discuss.elastic.co/t/accidentally-deleted-http-ca-crt-how-to-recover-urgent-please/360965)
**Author:** [@evangelin](https://discuss.elastic.co/u/evangelin)\ **Replies:** 7\ **Last updated:** [June 11, 2024, 6:27pm UTC](https://discuss.elastic.co/t/accidentally-deleted-http-ca-crt-how-to-recover-urgent-please/360965 "2024-06-11T18:27:32Z")
hello, deleted ca.crt. I'm getting the below error in logstash \[2024-06-06T18:17:02,766\]\[WARN \]\[logstash.outputs.elasticsearch\]\[main\] Attempted to resurrect connection to dead ES instance, but got an error {:url=\>"http… --- ## [Multiple pipelines: StringIndexOutOfBoundsException: String index out of range: -1](https://discuss.elastic.co/t/multiple-pipelines-stringindexoutofboundsexception-string-index-out-of-range-1/360714)
**Author:** [@om-myc](https://discuss.elastic.co/u/om-myc)\ **Replies:** 2\ **Last updated:** [June 11, 2024, 8:14am UTC](https://discuss.elastic.co/t/multiple-pipelines-stringindexoutofboundsexception-string-index-out-of-range-1/360714 "2024-06-11T08:14:20Z")
Hello everybody, i recently set up an ELK stack and am running Logstash on a Windows host which processes input from log files and sends them to elasticsearch on another host. I use the "collector pattern" to process t… --- ## [Using proxy on datadog\_output\_logs plugin on logstash](https://discuss.elastic.co/t/using-proxy-on-datadog-output-logs-plugin-on-logstash/361219)
**Author:** [@Kalaivani\_C](https://discuss.elastic.co/u/Kalaivani_C)\ **Replies:** 1\ **Last updated:** [June 11, 2024, 7:00am UTC](https://discuss.elastic.co/t/using-proxy-on-datadog-output-logs-plugin-on-logstash/361219 "2024-06-11T07:00:14Z")
Hello Team, Problem Statement: Logstash events are not reaching datadog. HTTP connection to datadog starts but no payloads are sent. Can you confirm if the proxy parameter in the logstash pipeline is correct? Please re… --- ## [Logstash Error - Oracle"" was unexpected at this time](https://discuss.elastic.co/t/logstash-error-oracle-was-unexpected-at-this-time/361185)
**Author:** [@asandhu](https://discuss.elastic.co/u/asandhu)\ **Replies:** 3\ **Last updated:** [June 10, 2024, 7:36pm UTC](https://discuss.elastic.co/t/logstash-error-oracle-was-unexpected-at-this-time/361185 "2024-06-10T19:36:51Z")
Hello All I am trying to set up Logstash on my Local to connect to a hosted Elastic instance. Having followed the steps to set up Logstash - I get the following error when trying to run it. I thought I had multiple v… --- ## [Logstash java codec running error](https://discuss.elastic.co/t/logstash-java-codec-running-error/361160)
**Author:** [@xudl](https://discuss.elastic.co/u/xudl)\ **Replies:** 0\ **Last updated:** [June 10, 2024, 2:22pm UTC](https://discuss.elastic.co/t/logstash-java-codec-running-error/361160 "2024-06-10T14:22:55Z")
hi：i run logstash-codec-java\_codec\_example , have some error: Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"Java::JavaLang::IllegalStateException", :message=\>"Unable… --- ## [Error parsing json in Logstash](https://discuss.elastic.co/t/error-parsing-json-in-logstash/361134)
**Author:** [@optimuspur3](https://discuss.elastic.co/u/optimuspur3)\ **Replies:** 2\ **Last updated:** [June 10, 2024, 2:06pm UTC](https://discuss.elastic.co/t/error-parsing-json-in-logstash/361134 "2024-06-10T14:06:44Z")
So I have been trying to setup cowrie to link it with my ELK but apparently I have been getting the error parsing json in my configuration files. The setup for my ELK is from this link. Error when it shows on my logst… --- ## [Logstash JDBC Input Issue - Single Quotes](https://discuss.elastic.co/t/logstash-jdbc-input-issue-single-quotes/360882)
**Author:** [@OscarFilho](https://discuss.elastic.co/u/OscarFilho)\ **Replies:** 2\ **Last updated:** [June 9, 2024, 6:09pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-issue-single-quotes/360882 "2024-06-09T18:09:43Z")
I'm using a JDBC input plugin (for sql server connection) in my logstash and having a issue on statement. The error displayed is: Java::ComMicrosoftSqlserverJdbc::SQLServerException: Incorrect syntax near 'ATUALIZADO\\'… --- ## [Logstash Configuration for Multi-Database Input Monitoring and Notification](https://discuss.elastic.co/t/logstash-configuration-for-multi-database-input-monitoring-and-notification/361115)
**Author:** [@SalehEska](https://discuss.elastic.co/u/SalehEska)\ **Replies:** 0\ **Last updated:** [June 9, 2024, 6:05pm UTC](https://discuss.elastic.co/t/logstash-configuration-for-multi-database-input-monitoring-and-notification/361115 "2024-06-09T18:05:05Z")
In Logstash, I have multiple inputs, such as Oracle and MySQL databases. Each input contains a query to retrieve data. I need to check if the process reaches each database input and executes the query, If the result of t… --- ## [Question about parsing log.file.path field](https://discuss.elastic.co/t/question-about-parsing-log-file-path-field/360872)
**Author:** [@Borja](https://discuss.elastic.co/u/Borja)\ **Replies:** 3\ **Last updated:** [June 7, 2024, 7:35pm UTC](https://discuss.elastic.co/t/question-about-parsing-log-file-path-field/360872 "2024-06-07T19:35:19Z")
Hi all, we are trying to ingest from filebeat some differents apache logs from diffent apps and different directories. So we have this input structure (this are the log.file.path fields ingested) /home/E879365/logs/ap… --- ## [\_dateparsefailure when trying to overwrite @timestamp](https://discuss.elastic.co/t/dateparsefailure-when-trying-to-overwrite-timestamp/360814)
**Author:** [@Cara410](https://discuss.elastic.co/u/Cara410)\ **Replies:** 8\ **Last updated:** [June 7, 2024, 7:05pm UTC](https://discuss.elastic.co/t/dateparsefailure-when-trying-to-overwrite-timestamp/360814 "2024-06-07T19:05:55Z")
Hello, I have been trying to get the timestamp to match the log entry and not the time it was ingested. I am using logstash 8.12.2 My log lines look like this: {"Timestamp":"2024-06-03 20:18:59.2251", "Message":"BLAH"}… --- ## [Searching from keyword field in Logstash with query](https://discuss.elastic.co/t/searching-from-keyword-field-in-logstash-with-query/361048)
**Author:** [@Ma\_G](https://discuss.elastic.co/u/Ma_G)\ **Replies:** 0\ **Last updated:** [June 7, 2024, 3:20pm UTC](https://discuss.elastic.co/t/searching-from-keyword-field-in-logstash-with-query/361048 "2024-06-07T15:20:42Z")
I would like to filter out the string "Alert" in the message field of my index. The field is a keyword field. To achieve this, I have written the following: filter { mutate { add\_field =\> {"secret" =\> "\ **Author:** [@Mansoor\_Ur\_Rehman](https://discuss.elastic.co/u/Mansoor_Ur_Rehman)\ **Replies:** 13\ **Last updated:** [June 7, 2024, 3:51pm UTC](https://discuss.elastic.co/t/logstash-output-date-using-timestamp-of-filebeat-instead-of-server-time/360845 "2024-06-07T15:51:41Z") I have setup a cluster filebeat -\> logstash -\> elasticsearch, i have a lifecycle policy that makes the indexes readonly after 7 days. i have out for logsatash index =\> "log-%{\[fields\]\[log\_name\]}-%{+YYYY.MM.dd}" … --- ## [Grok parser incorrectly matches JAVACLASS](https://discuss.elastic.co/t/grok-parser-incorrectly-matches-javaclass/360951)
**Author:** [@Rade\_Pajic](https://discuss.elastic.co/u/Rade_Pajic)\ **Replies:** 4\ **Last updated:** [June 7, 2024, 5:12am UTC](https://discuss.elastic.co/t/grok-parser-incorrectly-matches-javaclass/360951 "2024-06-07T05:12:11Z")
Hello, I'm upgrading ELK from version 7.3.1 to 8.13.4. I have a problem with the Logstash pipeline as it does not work as expected, or I am missing something completely. I have Java logs, and I'm sending them to Logsta… --- ## [I try to ingest the data from S3 bucket to opensearch dashboards using logstash](https://discuss.elastic.co/t/i-try-to-ingest-the-data-from-s3-bucket-to-opensearch-dashboards-using-logstash/360781)
**Author:** [@Rathiga\_Thambu](https://discuss.elastic.co/u/Rathiga_Thambu)\ **Replies:** 16\ **Last updated:** [June 6, 2024, 4:32pm UTC](https://discuss.elastic.co/t/i-try-to-ingest-the-data-from-s3-bucket-to-opensearch-dashboards-using-logstash/360781 "2024-06-06T16:32:05Z")
\[2024-06-04T11:09:21,409\]\[INFO \]\[logstash.javapipeline \]\[main\] Pipeline terminated {"pipeline.id"=\>"main"} \[2024-06-04T11:09:21,484\]\[ERROR\]\[logstash.agent \] Failed to execute action {:id=\>:main, :action\_typ… --- ## [Help with Logstash Multiline parsing for Stacktraces](https://discuss.elastic.co/t/help-with-logstash-multiline-parsing-for-stacktraces/360974)
**Author:** [@preguenga](https://discuss.elastic.co/u/preguenga)\ **Replies:** 2\ **Last updated:** [June 6, 2024, 2:27pm UTC](https://discuss.elastic.co/t/help-with-logstash-multiline-parsing-for-stacktraces/360974 "2024-06-06T14:27:28Z")
Hello Elastic Community, I am currently working on configuring Logstash to correctly process error stacktraces from Kubernetes logs. My goal is to ensure that stacktraces are combined into a single event before further … --- ## [Create array of objects in ruby](https://discuss.elastic.co/t/create-array-of-objects-in-ruby/360806)
**Author:** [@Armalyca](https://discuss.elastic.co/u/Armalyca)\ **Replies:** 2\ **Last updated:** [June 6, 2024, 12:36pm UTC](https://discuss.elastic.co/t/create-array-of-objects-in-ruby/360806 "2024-06-06T12:36:10Z")
Hello, I use logstash 7.17. I want to create an array of objects in ruby from my data, now I get a list of objects. I didn't success to implement a each loop, so there is a lot of duplicate ligns. Here is a snippet of … --- ## [Is there a option to limit how deep the JSON parsing would go?](https://discuss.elastic.co/t/is-there-a-option-to-limit-how-deep-the-json-parsing-would-go/360930)
**Author:** [@weily2](https://discuss.elastic.co/u/weily2)\ **Replies:** 0\ **Last updated:** [June 6, 2024, 7:50am UTC](https://discuss.elastic.co/t/is-there-a-option-to-limit-how-deep-the-json-parsing-would-go/360930 "2024-06-06T07:50:53Z")
For a case where there are multiple applications going through an ingest pipeline with many unique nested fields multiple levels deep, it would be helpful if there was an option just like max\_depth to limit how deep the … --- ## [S3 Multiple outputs using Logstash and Winlogbeat](https://discuss.elastic.co/t/s3-multiple-outputs-using-logstash-and-winlogbeat/360813)
**Author:** [@tcalvillo](https://discuss.elastic.co/u/tcalvillo)\ **Replies:** 3\ **Last updated:** [June 5, 2024, 5:26pm UTC](https://discuss.elastic.co/t/s3-multiple-outputs-using-logstash-and-winlogbeat/360813 "2024-06-05T17:26:47Z")
Hello everybody, I'm using AWS with a Windows EC2 instance and I collect logs by using Logstash and Winlogbeat. I'm able to collect all logs if I keep all together. However, I would like to separate logs according if a… --- ## [Logstash JDBC missing many documents](https://discuss.elastic.co/t/logstash-jdbc-missing-many-documents/360875)
**Author:** [@darrylds](https://discuss.elastic.co/u/darrylds)\ **Replies:** 0\ **Last updated:** [June 5, 2024, 5:17pm UTC](https://discuss.elastic.co/t/logstash-jdbc-missing-many-documents/360875 "2024-06-05T17:17:01Z")
Moving search from postgres full text index to Elasticsearch. Using Logstash JDBC to input documents into index. Index about 3 million documents a year roughly 3K to 10K documents a day. ElasticSeach has increase sear… --- ## [How could I export a set of logs from ELK to another source, ideally in json format?](https://discuss.elastic.co/t/how-could-i-export-a-set-of-logs-from-elk-to-another-source-ideally-in-json-format/360817)
**Author:** [@Bulevine](https://discuss.elastic.co/u/Bulevine)\ **Replies:** 0\ **Last updated:** [June 5, 2024, 3:10am UTC](https://discuss.elastic.co/t/how-could-i-export-a-set-of-logs-from-elk-to-another-source-ideally-in-json-format/360817 "2024-06-05T03:10:49Z")
To get a little more specific.. I am almost entirely new to ELK, so forgive my lack of knowledge. I am trying to export a set of logs for a given timeframe and matching tags over to DataDog. It would look something like… --- ## [ECK stack with Fluentd](https://discuss.elastic.co/t/eck-stack-with-fluentd/360793)
**Author:** [@daniela09](https://discuss.elastic.co/u/daniela09)\ **Replies:** 0\ **Last updated:** [June 4, 2024, 2:13pm UTC](https://discuss.elastic.co/t/eck-stack-with-fluentd/360793 "2024-06-04T14:13:40Z")
Hi, I have deployed ECK on Kubernetes and now I want to use fluentd to collect logs from other applications on Kubernetes and Kafka topic, it collected logs from the other application I want but not from Kafka topic. Thi… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=27) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=29)