# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=46 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 47 --- ## [Integration of elastic and logstash with other vizualization](https://discuss.elastic.co/t/integration-of-elastic-and-logstash-with-other-vizualization/350292)
**Author:** [@Karan\_Lobo](https://discuss.elastic.co/u/Karan_Lobo)\ **Replies:** 1\ **Last updated:** [January 3, 2024, 12:50pm UTC](https://discuss.elastic.co/t/integration-of-elastic-and-logstash-with-other-vizualization/350292 "2024-01-03T12:50:41Z")
Hey there i am working on a project that requires me to integrate ELK stack with other vizualiation tools fo free , however i am running into the issue of downloading the ODBC driver as it is showing that i would need a … --- ## [How to read base64 encoded logs from kafka through logstash pipeline](https://discuss.elastic.co/t/how-to-read-base64-encoded-logs-from-kafka-through-logstash-pipeline/349688)
**Author:** [@upreddy](https://discuss.elastic.co/u/upreddy)\ **Replies:** 9\ **Last updated:** [January 3, 2024, 9:28am UTC](https://discuss.elastic.co/t/how-to-read-base64-encoded-logs-from-kafka-through-logstash-pipeline/349688 "2024-01-03T09:28:30Z")
Hello all, I am working on something I have never worked on before and I really do not know where to go from here and I am hoping someone might have some direction for me to attempt trying to get this parsed to Elastics… --- ## [Date parsing logstash](https://discuss.elastic.co/t/date-parsing-logstash/350064)
**Author:** [@Haytham\_Shammout](https://discuss.elastic.co/u/Haytham_Shammout)\ **Replies:** 4\ **Last updated:** [January 2, 2024, 9:14pm UTC](https://discuss.elastic.co/t/date-parsing-logstash/350064 "2024-01-02T21:14:04Z")
Hello Dears, i am trying to use syslog timestamp as @timestamp in Elasticsearch, i tried to use date filter and it gives me \_dateparsefailure in the logs when i browse them on kibana. filter Plugin snippet. filter{ … --- ## [Logstash-plugin command for preparing offline pack is not working on 8.11.3](https://discuss.elastic.co/t/logstash-plugin-command-for-preparing-offline-pack-is-not-working-on-8-11-3/349302)
**Author:** [@ebiibe82](https://discuss.elastic.co/u/ebiibe82)\ **Replies:** 1\ **Last updated:** [January 2, 2024, 5:56pm UTC](https://discuss.elastic.co/t/logstash-plugin-command-for-preparing-offline-pack-is-not-working-on-8-11-3/349302 "2024-01-02T17:56:10Z")
Hello All, I am new to Elastic Stack. I have installed Logstash 8.11.3 using deb package on Ubuntu 22.04 Server. On top of it, I have installed logstash-output-syslog plugin. Till this point, it works fine. After this, … --- ## [Potential memory leak using tcp input?](https://discuss.elastic.co/t/potential-memory-leak-using-tcp-input/349968)
**Author:** [@udp\_issues\_are\_one](https://discuss.elastic.co/u/udp_issues_are_one)\ **Replies:** 14\ **Last updated:** [January 2, 2024, 3:15pm UTC](https://discuss.elastic.co/t/potential-memory-leak-using-tcp-input/349968 "2024-01-02T15:15:08Z")
Hello, we are using logstash to collect sflow. We use fluentd at the ingest point, which forwards flows to logstash to do some processing and push to the elastic cloud. These both reside on the same box and largely wor… --- ## [Two Logstash nodes. Same config. Persistent queue filling only in one of them](https://discuss.elastic.co/t/two-logstash-nodes-same-config-persistent-queue-filling-only-in-one-of-them/350229)
**Author:** [@nahiko](https://discuss.elastic.co/u/nahiko)\ **Replies:** 1\ **Last updated:** [January 2, 2024, 1:34pm UTC](https://discuss.elastic.co/t/two-logstash-nodes-same-config-persistent-queue-filling-only-in-one-of-them/350229 "2024-01-02T13:34:31Z")
Hello! I have a 3 node Elasticsearch cluster, 2 Logstash nodes and about 100 filebeats sending data to Logstash. Every piece is 7.17 Both Logstash nodes have the exact same configuration. There is a 16 GB persistent q… --- ## [Logstash inconsistency while reading csv data](https://discuss.elastic.co/t/logstash-inconsistency-while-reading-csv-data/348881)
**Author:** [@iko](https://discuss.elastic.co/u/iko)\ **Replies:** 8\ **Last updated:** [January 2, 2024, 1:15pm UTC](https://discuss.elastic.co/t/logstash-inconsistency-while-reading-csv-data/348881 "2024-01-02T13:15:05Z")
Hello, We are using Logstash for parsing csv data and load them into Postgresql and then after making proper transformation we move that data to Elasticsearch by using same Logstash . We don't have any problem about tra… --- ## [Logstash upgrade issue - 8.11.3 version](https://discuss.elastic.co/t/logstash-upgrade-issue-8-11-3-version/350132)
**Author:** [@siva0030](https://discuss.elastic.co/u/siva0030)\ **Replies:** 7\ **Last updated:** [January 2, 2024, 9:23am UTC](https://discuss.elastic.co/t/logstash-upgrade-issue-8-11-3-version/350132 "2024-01-02T09:23:40Z")
Hello Team, Good evening! Today I have upgraded the Logstash from version 8.10.4 to 8.11.3 version. After the upgrade the Logstash is keep restarting and throwing below errors. This type of FATAL error is coming for a… --- ## [Encountered a retryable error (will retry with exponential backoff) {:code=\>413}](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff-code-413/349802)
**Author:** [@sathishkumarD](https://discuss.elastic.co/u/sathishkumarD)\ **Replies:** 3\ **Last updated:** [January 2, 2024, 9:02am UTC](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff-code-413/349802 "2024-01-02T09:02:25Z")
Elastic search and Logstash version: 8.5.1 Getting below error from logstash when trying to transfer files to elasticsearch. Could someone help me to fix the issue. \[ERROR\]\[logstash.outputs.elasticsearch\]\[main\]\[532e27b… --- ## [Fail Setup Logstash](https://discuss.elastic.co/t/fail-setup-logstash/350203)
**Author:** [@Septianingrum.17](https://discuss.elastic.co/u/Septianingrum.17)\ **Replies:** 0\ **Last updated:** [January 2, 2024, 8:23am UTC](https://discuss.elastic.co/t/fail-setup-logstash/350203 "2024-01-02T08:23:05Z")
Hi, I tried setting up logstash in my environment, previously I had 3 elasticsearch nodes and 1 kibana. I followed the steps" based on the URL: https://www.elastic.co/blog/configuring-ssl-tls-and-https-to-secure-elasti… --- ## [Invalid version of beats protocol: 69](https://discuss.elastic.co/t/invalid-version-of-beats-protocol-69/349830)
**Author:** [@e-ferrari](https://discuss.elastic.co/u/e-ferrari)\ **Replies:** 10\ **Last updated:** [January 1, 2024, 10:28pm UTC](https://discuss.elastic.co/t/invalid-version-of-beats-protocol-69/349830 "2024-01-01T22:28:56Z")
Hello, I'm completely new to ELK. I'm reading the doc and try to execute this: But i got an error from logstash: \[2023-12-21T23:21:37,978\]\[WARN \]\[io.netty.channel.DefaultChannelPipeline\]\[main\]\[c6b88577022f3da3a78380… --- ## [Logstash configuration with multiple http\_poller did'nt ran for some indices](https://discuss.elastic.co/t/logstash-configuration-with-multiple-http-poller-didnt-ran-for-some-indices/350151)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 0\ **Last updated:** [December 31, 2023, 8:02am UTC](https://discuss.elastic.co/t/logstash-configuration-with-multiple-http-poller-didnt-ran-for-some-indices/350151 "2023-12-31T08:02:11Z")
Hello, I have a logstash configuration with multiple http\_poller input plugins. say input { http\_poller { id =\> "s1-input" urls =\> { sector\_api =\> { method =\> "POST" url =\> "url1" headers =\>… --- ## [How to integrate syslog input plugin](https://discuss.elastic.co/t/how-to-integrate-syslog-input-plugin/349025)
**Author:** [@Ravi\_Pattar](https://discuss.elastic.co/u/Ravi_Pattar)\ **Replies:** 41\ **Last updated:** [December 26, 2023, 12:24pm UTC](https://discuss.elastic.co/t/how-to-integrate-syslog-input-plugin/349025 "2023-12-26T12:24:13Z")
Hi, I have installed full stack ELK (version 7.17.13) and now I want to integrate syslog input plugin. Need some directions on the same on how to setup. Also when I tried with some changes in logstash.conf but I am fa… --- ## [Logstash pipeline to filter rss document](https://discuss.elastic.co/t/logstash-pipeline-to-filter-rss-document/349800)
**Author:** [@ramiwashere](https://discuss.elastic.co/u/ramiwashere)\ **Replies:** 3\ **Last updated:** [December 29, 2023, 2:52pm UTC](https://discuss.elastic.co/t/logstash-pipeline-to-filter-rss-document/349800 "2023-12-29T14:52:53Z")
I've just created a logstash that will retrieve documents from a feed. I receive the documents in return but the fields I want to add are all on the same document. Here's an example: \ \ \… --- ## [How to parse date field into @timestamp](https://discuss.elastic.co/t/how-to-parse-date-field-into-timestamp/349849)
**Author:** [@emoxam](https://discuss.elastic.co/u/emoxam)\ **Replies:** 11\ **Last updated:** [December 22, 2023, 2:26pm UTC](https://discuss.elastic.co/t/how-to-parse-date-field-into-timestamp/349849 "2023-12-22T14:26:55Z")
I want to move the fulltime from message field to @timestamp. That's what i created. filter { if \[message\] =~ /actions/ { json { source =\> "message" } date { match =\> \[ "message", "yyyy-MM-dd … --- ## [Elasticsearch not creating index](https://discuss.elastic.co/t/elasticsearch-not-creating-index/350016)
**Author:** [@bas\_kos](https://discuss.elastic.co/u/bas_kos)\ **Replies:** 0\ **Last updated:** [December 27, 2023, 11:06am UTC](https://discuss.elastic.co/t/elasticsearch-not-creating-index/350016 "2023-12-27T11:06:38Z")
I have elasticsearch, kibana and logstash installed on docker-compose. docker-compose.yml: version: "3.8" volumes: certs: driver: local esdata01: driver: local kibanadata: driver: local metricbeatd… --- ## [Logstash V fileBeat](https://discuss.elastic.co/t/logstash-v-filebeat/349979)
**Author:** [@pumiki](https://discuss.elastic.co/u/pumiki)\ **Replies:** 0\ **Last updated:** [December 26, 2023, 5:09pm UTC](https://discuss.elastic.co/t/logstash-v-filebeat/349979 "2023-12-26T17:09:27Z")
Hello, We have c# applications , running without docker. we want to write them to Elasticsearch. I have managed to run logstash (right now as exe) and make it write to csv files. next, i will change it to write to e… --- ## [Logstash not sending data to Elasticsearch](https://discuss.elastic.co/t/logstash-not-sending-data-to-elasticsearch/349736)
**Author:** [@gtartjr](https://discuss.elastic.co/u/gtartjr)\ **Replies:** 5\ **Last updated:** [December 26, 2023, 4:06pm UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-elasticsearch/349736 "2023-12-26T16:06:06Z")
I am unable to get Logstash to read data and send to Elasticsearch index. My Elastcistac is 8.11.2, under a Docker for Windows platform. I have 2 jsonl formatted files that I need to index into Elasticsearch by a unique … --- ## [MSK to Elasticksearch using logstash](https://discuss.elastic.co/t/msk-to-elasticksearch-using-logstash/349945)
**Author:** [@Gersi\_Tafili](https://discuss.elastic.co/u/Gersi_Tafili)\ **Replies:** 4\ **Last updated:** [December 26, 2023, 1:14pm UTC](https://discuss.elastic.co/t/msk-to-elasticksearch-using-logstash/349945 "2023-12-26T13:14:29Z")
I have create MSK in AWS also Elastic search cluster hostes in AWS. I am trying to read data from topic in MSK and send this data to elasticsearch index. input { kafka { bootstrap\_servers =\> "x:9096" topics =\>… --- ## [Duplicate messages with logstash and log4net RollingFileAppender](https://discuss.elastic.co/t/duplicate-messages-with-logstash-and-log4net-rollingfileappender/349932)
**Author:** [@pumiki](https://discuss.elastic.co/u/pumiki)\ **Replies:** 1\ **Last updated:** [December 26, 2023, 10:53am UTC](https://discuss.elastic.co/t/duplicate-messages-with-logstash-and-log4net-rollingfileappender/349932 "2023-12-26T10:53:46Z")
Hello, My app writes events using log4net with rolling file appender. I get messages duplicated in the file gerenated by logstash. I found the issue mentioned also here However, I am not sure about the solution. Cou… --- ## [Logstash with log4net](https://discuss.elastic.co/t/logstash-with-log4net/349919)
**Author:** [@pumiki](https://discuss.elastic.co/u/pumiki)\ **Replies:** 0\ **Last updated:** [December 25, 2023, 11:39pm UTC](https://discuss.elastic.co/t/logstash-with-log4net/349919 "2023-12-25T23:39:20Z")
Hello, we have c# app (many microservices), running without docker. for now, All the microservices use log4net to log to file. we want to write those logs to log4net. the question is how to do it ? Question 1: w… --- ## [Logstash terminating pipelines error "const\_missing, block in JDBC"](https://discuss.elastic.co/t/logstash-terminating-pipelines-error-const-missing-block-in-jdbc/349715)
**Author:** [@SamehSaeed](https://discuss.elastic.co/u/SamehSaeed)\ **Replies:** 3\ **Last updated:** [December 25, 2023, 1:10pm UTC](https://discuss.elastic.co/t/logstash-terminating-pipelines-error-const-missing-block-in-jdbc/349715 "2023-12-25T13:10:21Z")
Hello, I have a problem when running logstash with multiple pipelines (around 70). Logstash will always terminate some of them if i run more than 30 concurrently 1- First error : \[ERROR\]\[logstash.javapipeline \]\[bkge… --- ## [Error with http-plugin output Encountered non-2xx HTTP code 400](https://discuss.elastic.co/t/error-with-http-plugin-output-encountered-non-2xx-http-code-400/348215)
**Author:** [@bilal\_adoui](https://discuss.elastic.co/u/bilal_adoui)\ **Replies:** 3\ **Last updated:** [December 25, 2023, 10:27am UTC](https://discuss.elastic.co/t/error-with-http-plugin-output-encountered-non-2xx-http-code-400/348215 "2023-12-25T10:27:04Z")
Hi, I am trying to send a notification from Logstash to our Teams channel, using HTTP plugin however I am getting : \[HTTP Output Failure\] Encountered non-2xx HTTP code 400 {:response\_code=\>400 and this is my output c… --- ## [Creating a data view from logstash](https://discuss.elastic.co/t/creating-a-data-view-from-logstash/349899)
**Author:** [@Dor-Alter](https://discuss.elastic.co/u/Dor-Alter)\ **Replies:** 1\ **Last updated:** [December 24, 2023, 6:59pm UTC](https://discuss.elastic.co/t/creating-a-data-view-from-logstash/349899 "2023-12-24T18:59:16Z")
I have created a pipeline that loads my logs into elasticsearch. When I run the pipeline it works and in the developer console, in elasticsearch, I can see and run queries to the index. However, when I go to discover in … --- ## [Logstash helm chart with Elasticsearch input/output starts over after finishing](https://discuss.elastic.co/t/logstash-helm-chart-with-elasticsearch-input-output-starts-over-after-finishing/349900)
**Author:** [@shaigbdb](https://discuss.elastic.co/u/shaigbdb)\ **Replies:** 0\ **Last updated:** [December 24, 2023, 5:12pm UTC](https://discuss.elastic.co/t/logstash-helm-chart-with-elasticsearch-input-output-starts-over-after-finishing/349900 "2023-12-24T17:12:00Z")
Hi, I'm using the logstash helm chart with Logstash 8.9.0. The pipeline has an input and an output of Elasticsearch, basically importing an index from one cluster to another (using snapshots or reindex would've been be… --- ## [Logstash SNMP input plugin not seeing metadata](https://discuss.elastic.co/t/logstash-snmp-input-plugin-not-seeing-metadata/349524)
**Author:** [@bytelink](https://discuss.elastic.co/u/bytelink)\ **Replies:** 6\ **Last updated:** [December 22, 2023, 5:04pm UTC](https://discuss.elastic.co/t/logstash-snmp-input-plugin-not-seeing-metadata/349524 "2023-12-22T17:04:26Z")
I am trying to implement the SNMP input plugin to gther network data however when I try and access the metadata it returns the line of code not the data. I even tried copying the example from the documentation and get t… --- ## [Logstash xml input plugin - parsing log4net:event](https://discuss.elastic.co/t/logstash-xml-input-plugin-parsing-log4net-event/349817)
**Author:** [@pumiki](https://discuss.elastic.co/u/pumiki)\ **Replies:** 4\ **Last updated:** [December 21, 2023, 10:04pm UTC](https://discuss.elastic.co/t/logstash-xml-input-plugin-parsing-log4net-event/349817 "2023-12-21T22:04:50Z")
Hello, log4net generates xml file. every event is stored in xml element called log4net:event. The issue is that logstash cant parse the element with the ":" in it. any idea ? The xml \\… --- ## [Cant parse xml file generated with log4net using logstash](https://discuss.elastic.co/t/cant-parse-xml-file-generated-with-log4net-using-logstash/349728)
**Author:** [@pumiki](https://discuss.elastic.co/u/pumiki)\ **Replies:** 5\ **Last updated:** [December 21, 2023, 8:18pm UTC](https://discuss.elastic.co/t/cant-parse-xml-file-generated-with-log4net-using-logstash/349728 "2023-12-21T20:18:02Z")
Hello, I have c# app that logs xml file using log4net and log4net.Layout.XmlLayout (see configuration below). The generated log events contains message ang name-value collection: \\ **Author:** [@cass1ope1a](https://discuss.elastic.co/u/cass1ope1a)\ **Replies:** 0\ **Last updated:** [December 21, 2023, 7:29am UTC](https://discuss.elastic.co/t/extracting-nested-fileds-with-grok-or-kv/349766 "2023-12-21T07:29:38Z") I have logs like: Server response. Body={"valid":\[{"someId":"12345","someType":"somevalue123","isSome":true}\],"invalid":\[\]} current pipeline config: if \[syslog\_tag\] =~ "json" { json { source =\> root\_… --- ## [Persistent ECS warning](https://discuss.elastic.co/t/persistent-ecs-warning/349743)
**Author:** [@Chris\_Stone](https://discuss.elastic.co/u/Chris_Stone)\ **Replies:** 3\ **Last updated:** [December 20, 2023, 6:03pm UTC](https://discuss.elastic.co/t/persistent-ecs-warning/349743 "2023-12-20T18:03:45Z")
logstash 8.11.3 Can anyone tell me why with the following config, and everything else at the default install, why I continue to get the \[logstash.codecs.jsonlines\] ECS compatibility is enabled but \`target\` option was n… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=45) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=47)