# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=12 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 13 --- ## [Logstash-input-google\_pubsub performance is throttled after upgrading older versions to anything from 8.4.0 onwards](https://discuss.elastic.co/t/logstash-input-google-pubsub-performance-is-throttled-after-upgrading-older-versions-to-anything-from-8-4-0-onwards/374128)
**Author:** [@nobody](https://discuss.elastic.co/u/nobody)\ **Replies:** 0\ **Last updated:** [February 5, 2025, 12:50pm UTC](https://discuss.elastic.co/t/logstash-input-google-pubsub-performance-is-throttled-after-upgrading-older-versions-to-anything-from-8-4-0-onwards/374128 "2025-02-05T12:50:43Z")
Hello there, I did hit an issue with consuming pubsub messages with most of the 8.x versions of logstash. I tried to rule out every single point of the pipeline, it seems logstash simply does not ingests data fast enoug… --- ## [Converting date into date format in logstash](https://discuss.elastic.co/t/converting-date-into-date-format-in-logstash/374111)
**Author:** [@roopeshetty](https://discuss.elastic.co/u/roopeshetty)\ **Replies:** 0\ **Last updated:** [February 5, 2025, 9:09am UTC](https://discuss.elastic.co/t/converting-date-into-date-format-in-logstash/374111 "2025-02-05T09:09:50Z")
Hi We are trying to ship a csv file from Filebeat \> Logstash \> Elasticsearch. This csv files has column by name “CreateOn” which will have the dates as below; Thursday, March 10, 2016 3:00:23 PM UTC Thursday, May 4, … --- ## [Help sanitizing new lines](https://discuss.elastic.co/t/help-sanitizing-new-lines/373823)
**Author:** [@Jose\_E](https://discuss.elastic.co/u/Jose_E)\ **Replies:** 2\ **Last updated:** [February 3, 2025, 1:50pm UTC](https://discuss.elastic.co/t/help-sanitizing-new-lines/373823 "2025-02-03T13:50:08Z")
Hi there, I need help with what I thought was a very simple issue, but turns out it isn't. I have updated from logstash 7.x to 8.x, and even though all of my tests have passed correctly, I found several issues with one … --- ## [Scheduling mail from logstash](https://discuss.elastic.co/t/scheduling-mail-from-logstash/373919)
**Author:** [@Hitesh\_Bhanderi](https://discuss.elastic.co/u/Hitesh_Bhanderi)\ **Replies:** 2\ **Last updated:** [February 3, 2025, 9:16am UTC](https://discuss.elastic.co/t/scheduling-mail-from-logstash/373919 "2025-02-03T09:16:41Z")
Hi this might be silly but I want to know that can i schedule mail from logstash using email-plugin as i want to schedule mail which contains error message from the logs .i had successfully implemented to mail error … --- ## [Syslog Output only transferring the "message" field](https://discuss.elastic.co/t/syslog-output-only-transferring-the-message-field/353396)
**Author:** [@Grimlock](https://discuss.elastic.co/u/Grimlock)\ **Replies:** 15\ **Last updated:** [January 31, 2025, 8:31pm UTC](https://discuss.elastic.co/t/syslog-output-only-transferring-the-message-field/353396 "2025-01-31T20:31:37Z")
Hi, I am using the Syslog Output plugin to transfer data to another host: syslog { host =\> "IP" port =\> Port } However, the target host only receives the content of the message fields. My input looks like this: … --- ## [Logstash email image problem](https://discuss.elastic.co/t/logstash-email-image-problem/373713)
**Author:** [@Hannah\_J\_Swystun](https://discuss.elastic.co/u/Hannah_J_Swystun)\ **Replies:** 11\ **Last updated:** [January 31, 2025, 1:52pm UTC](https://discuss.elastic.co/t/logstash-email-image-problem/373713 "2025-01-31T13:52:28Z")
Hi everybody, i try to send email from logstash version 7.8.0 but, images are never shown! however attachments are added to email and can be opened! i used different kind of images, none of them work ? can someone he… --- ## [Remove all tags at one time?](https://discuss.elastic.co/t/remove-all-tags-at-one-time/373931)
**Author:** [@sbocquet](https://discuss.elastic.co/u/sbocquet)\ **Replies:** 5\ **Last updated:** [January 31, 2025, 1:33pm UTC](https://discuss.elastic.co/t/remove-all-tags-at-one-time/373931 "2025-01-31T13:33:03Z")
Hi, Is it possible to remove all tags in the \[tags\] field at one time ? I have this filter : # Transfert junk to the garbage datastream if \[Severity\] == "DEBUG" { mutate { update =\> { "\[data\_stre… --- ## [How to Run Logstash Immediately and Maintain a Scheduled Execution?](https://discuss.elastic.co/t/how-to-run-logstash-immediately-and-maintain-a-scheduled-execution/373889)
**Author:** [@Zikou](https://discuss.elastic.co/u/Zikou)\ **Replies:** 3\ **Last updated:** [January 30, 2025, 5:50pm UTC](https://discuss.elastic.co/t/how-to-run-logstash-immediately-and-maintain-a-scheduled-execution/373889 "2025-01-30T17:50:14Z")
I'm using Logstash to fetch data from a database and index it into Elasticsearch. My Logstash configuration includes a schedule to run every 50 minutes like this: input { jdbc { jdbc\_connection\_string =\> "jdbc:mys… --- ## [How to separate input with jmx input plugin?](https://discuss.elastic.co/t/how-to-separate-input-with-jmx-input-plugin/373803)
**Author:** [@Elgis\_Migle](https://discuss.elastic.co/u/Elgis_Migle)\ **Replies:** 1\ **Last updated:** [January 29, 2025, 6:23pm UTC](https://discuss.elastic.co/t/how-to-separate-input-with-jmx-input-plugin/373803 "2025-01-29T18:23:08Z")
Hello all, I'm trying to set up some monitoring of a confluentinc/cp-server-connect:7.6.0 docker container and the connectors I run within that. I am using the input-jmx plugin. Now I would like to separate the pipelin… --- ## [Logstash pushing event to older index](https://discuss.elastic.co/t/logstash-pushing-event-to-older-index/373730)
**Author:** [@devops\_training](https://discuss.elastic.co/u/devops_training)\ **Replies:** 10\ **Last updated:** [January 28, 2025, 8:37pm UTC](https://discuss.elastic.co/t/logstash-pushing-event-to-older-index/373730 "2025-01-28T20:37:29Z")
\[2025-01-27T11:17:59,721\]\[INFO \]\[logstash.outputs.elastic\]\[main\]\[160e296c5399e510649e9432dfb7810d3b2916f58b5ee2e872c7b33dd0983a64\] Retrying failed action {:status=\>429, :action=\>\["index", {:\_id=\>"a01a2730453bbd683e5db277… --- ## [Logstash benchmark](https://discuss.elastic.co/t/logstash-benchmark/373567)
**Author:** [@devops\_training](https://discuss.elastic.co/u/devops_training)\ **Replies:** 2\ **Last updated:** [January 27, 2025, 3:16pm UTC](https://discuss.elastic.co/t/logstash-benchmark/373567 "2025-01-27T15:16:49Z")
I see the below error in logstash \[2025-01-17T16:10:51,266\]\[ERROR\]\[logstash.outputs.elasticsearch\]\[main\]\[e65c1c48441d9162efee01ea49bd87b3c33b47189f25296954c014186f3f1bca\] Encountered a retryable error (will retry with e… --- ## [Possible error in documentaion about creating an API key for central management](https://discuss.elastic.co/t/possible-error-in-documentaion-about-creating-an-api-key-for-central-management/373355)
**Author:** [@jack\_a](https://discuss.elastic.co/u/jack_a)\ **Replies:** 3\ **Last updated:** [January 25, 2025, 5:08pm UTC](https://discuss.elastic.co/t/possible-error-in-documentaion-about-creating-an-api-key-for-central-management/373355 "2025-01-25T17:08:35Z")
So i was reading the document:Secure your connection to Elasticsearch section:Create an API key for central management and when i try to run the provided example: POST /\_security/api\_key { "name": "logstash\_host001", … --- ## [\[BIG-IP ASM\] Could not index event to Elasticsearch](https://discuss.elastic.co/t/big-ip-asm-could-not-index-event-to-elasticsearch/373465)
**Author:** [@Plauda](https://discuss.elastic.co/u/Plauda)\ **Replies:** 11\ **Last updated:** [January 24, 2025, 4:42pm UTC](https://discuss.elastic.co/t/big-ip-asm-could-not-index-event-to-elasticsearch/373465 "2025-01-24T16:42:45Z")
Hi There, I try to put some F5 BIG-IP ASM logs in ELK. I configured a logstash syslog conf file and tried to , but I get an error during indexation step: Jan 21 17:12:17 cnclelk12 logstash\[162998\]: \[2025-01-21T17:12:1… --- ## [DNS filter - unfrequent but regular error](https://discuss.elastic.co/t/dns-filter-unfrequent-but-regular-error/372975)
**Author:** [@Po-temkin](https://discuss.elastic.co/u/Po-temkin)\ **Replies:** 1\ **Last updated:** [January 24, 2025, 1:47pm UTC](https://discuss.elastic.co/t/dns-filter-unfrequent-but-regular-error/372975 "2025-01-24T13:47:46Z")
Hello to everyone! Not long ago I started to use Logstash to collect syslog from various devices in our infrastructure. To solve this problem, I created some amount of pipelines (now about 40) that help me route syslog… --- ## [IBMMQ Logstash](https://discuss.elastic.co/t/ibmmq-logstash/373624)
**Author:** [@adis3421](https://discuss.elastic.co/u/adis3421)\ **Replies:** 0\ **Last updated:** [January 24, 2025, 10:24am UTC](https://discuss.elastic.co/t/ibmmq-logstash/373624 "2025-01-24T10:24:30Z")
Hi, I have a problem with logstash performance when I download xml from IBMMQ queue. If I do not set logstash I cannot exceed 15k eps. It works best on default settings and heap 8gb my system under logstash 32vcp, 64gb … --- ## [Unable to get logstash listening on custom port](https://discuss.elastic.co/t/unable-to-get-logstash-listening-on-custom-port/373239)
**Author:** [@Plauda](https://discuss.elastic.co/u/Plauda)\ **Replies:** 18\ **Last updated:** [January 21, 2025, 9:47am UTC](https://discuss.elastic.co/t/unable-to-get-logstash-listening-on-custom-port/373239 "2025-01-21T09:47:02Z")
Hello, I need your help in order to get logstash listening on a custom port. Context : I'm trying to install an ELK stack in order to push syslog on it. For tests purpose, I installed elasticsearch and logstash on the… --- ## [Logstash issue with duplicates of logs](https://discuss.elastic.co/t/logstash-issue-with-duplicates-of-logs/373599)
**Author:** [@arcsons](https://discuss.elastic.co/u/arcsons)\ **Replies:** 0\ **Last updated:** [January 23, 2025, 7:46pm UTC](https://discuss.elastic.co/t/logstash-issue-with-duplicates-of-logs/373599 "2025-01-23T19:46:22Z")
Hello, I have a simple logstash.conf file that looks something like this. When using Filebeat, it remembered which logs were sent to Elasticsearch. How can I implement a similar solution for Logstash? Currently, it's se… --- ## [Logstash 7.6.2 cant connect to Elastic in Elastic Cloud](https://discuss.elastic.co/t/logstash-7-6-2-cant-connect-to-elastic-in-elastic-cloud/373584)
**Author:** [@Vinay6288](https://discuss.elastic.co/u/Vinay6288)\ **Replies:** 0\ **Last updated:** [January 23, 2025, 2:23pm UTC](https://discuss.elastic.co/t/logstash-7-6-2-cant-connect-to-elastic-in-elastic-cloud/373584 "2025-01-23T14:23:20Z")
I have Logstash 7.6.2 running in my local & using elasticsearch output plugin to write data to Elastic in Elastic Cloud. elasticsearch { hosts =\> \["https://ELASTIC-CLOUD-DNS:9243"\] ssl =\> true ssl\_certificat… --- ## [TCP output plugin blocking port on pipeline reload](https://discuss.elastic.co/t/tcp-output-plugin-blocking-port-on-pipeline-reload/373574)
**Author:** [@Sidd](https://discuss.elastic.co/u/Sidd)\ **Replies:** 0\ **Last updated:** [January 23, 2025, 10:30am UTC](https://discuss.elastic.co/t/tcp-output-plugin-blocking-port-on-pipeline-reload/373574 "2025-01-23T10:30:23Z")
On upgrading from logstash-oss-8.5.2-1 to 8.15.4 which started to use logstash-output-tcp Plugin version: v6.2.1 from Plugin version: v6.1.1. It seems the bug which was resolved as mentioned in changelog of 6.1.0 is intr… --- ## [How to retrieve fields content when quering runtime fields from Logstash](https://discuss.elastic.co/t/how-to-retrieve-fields-content-when-quering-runtime-fields-from-logstash/360466)
**Author:** [@Christian\_SANCHEZ](https://discuss.elastic.co/u/Christian_SANCHEZ)\ **Replies:** 1\ **Last updated:** [January 23, 2025, 9:57am UTC](https://discuss.elastic.co/t/how-to-retrieve-fields-content-when-quering-runtime-fields-from-logstash/360466 "2025-01-23T09:57:35Z")
Hello I am implementing Runtime fields. I added my runtime fields in my Index mapping. When searching the data from the Dev Console, i see by "fields" attribute, containing all the runtime fields I am expecting. But,… --- ## [LS constantly reloading failing pipeline & exhausting host connections](https://discuss.elastic.co/t/ls-constantly-reloading-failing-pipeline-exhausting-host-connections/373571)
**Author:** [@jdswifty](https://discuss.elastic.co/u/jdswifty)\ **Replies:** 0\ **Last updated:** [January 23, 2025, 9:56am UTC](https://discuss.elastic.co/t/ls-constantly-reloading-failing-pipeline-exhausting-host-connections/373571 "2025-01-23T09:56:11Z")
Noticed that when I have a pipeline, with an Elasticsearch output, that is failing, LS is then reloading the pipeline over & over but not clearing down the previously spawned connections to ES so I end up with connection… --- ## [How to Configure ILm policy in logstash and elastic search](https://discuss.elastic.co/t/how-to-configure-ilm-policy-in-logstash-and-elastic-search/373469)
**Author:** [@Ramisetti\_Vamsi\_Kris](https://discuss.elastic.co/u/Ramisetti_Vamsi_Kris)\ **Replies:** 2\ **Last updated:** [January 22, 2025, 11:22am UTC](https://discuss.elastic.co/t/how-to-configure-ilm-policy-in-logstash-and-elastic-search/373469 "2025-01-22T11:22:47Z")
Hey i am facing some issue with configuring ILM policy in Elasticsearch. While configuring the ilm policy i am facing rollover issues. Can you give me the implementation of ilm policy configuration in logstash and Elasti… --- ## [Elasticsearch java.lang.NullPointerException: null](https://discuss.elastic.co/t/elasticsearch-java-lang-nullpointerexception-null/373318)
**Author:** [@Mendiu](https://discuss.elastic.co/u/Mendiu)\ **Replies:** 3\ **Last updated:** [January 21, 2025, 9:06pm UTC](https://discuss.elastic.co/t/elasticsearch-java-lang-nullpointerexception-null/373318 "2025-01-21T21:06:16Z")
Hi after update from 7.17.13 -\> 7.17.18 Elastic v 7.17.18 Logstash 7.17.18 Java 11.0.18 I have err from logstash: logstash\[12445\]: \[2025-01-17T13:14:53,148\]\[ERROR\]\[logstash.outputs.elasticsearch\]\[main\]\[id\] Encounte… --- ## [Logstah doesn't split log into separate fields in elastic index](https://discuss.elastic.co/t/logstah-doesnt-split-log-into-separate-fields-in-elastic-index/373049)
**Author:** [@andp](https://discuss.elastic.co/u/andp)\ **Replies:** 4\ **Last updated:** [January 20, 2025, 1:38pm UTC](https://discuss.elastic.co/t/logstah-doesnt-split-log-into-separate-fields-in-elastic-index/373049 "2025-01-20T13:38:50Z")
Hi everyone! I am trying to import logs to elastic index using Filebeat and Logstah. The index is created but whole log line is present in one field message. For start I'm trying to separate datetime to a separate … --- ## [Segregating Data Between Spaces in a Single Deployment](https://discuss.elastic.co/t/segregating-data-between-spaces-in-a-single-deployment/373370)
**Author:** [@Core990](https://discuss.elastic.co/u/Core990)\ **Replies:** 2\ **Last updated:** [January 19, 2025, 11:47pm UTC](https://discuss.elastic.co/t/segregating-data-between-spaces-in-a-single-deployment/373370 "2025-01-19T23:47:47Z")
Hello, I have a use case where I want to create multiple spaces in a single Elastic deployment, and I need to ensure strict data segregation between these spaces. Specifically: I want the logs collected by agents in S… --- ## [Need to include a certain field from my Log into the Message part of my log without pulling the entire log in](https://discuss.elastic.co/t/need-to-include-a-certain-field-from-my-log-into-the-message-part-of-my-log-without-pulling-the-entire-log-in/373342)
**Author:** [@backlog](https://discuss.elastic.co/u/backlog)\ **Replies:** 5\ **Last updated:** [January 18, 2025, 12:41pm UTC](https://discuss.elastic.co/t/need-to-include-a-certain-field-from-my-log-into-the-message-part-of-my-log-without-pulling-the-entire-log-in/373342 "2025-01-18T12:41:16Z")
Hello, I am currently using security onion for pulling in zeek logs to our SIEM, and they are very particular about the JSON formatting of the data to have it show up properly parsed in the SIEM. For my logstash pipeli… --- ## [Centralized managed logstash pipeline issue](https://discuss.elastic.co/t/centralized-managed-logstash-pipeline-issue/373268)
**Author:** [@Joey\_Visbeen](https://discuss.elastic.co/u/Joey_Visbeen)\ **Replies:** 2\ **Last updated:** [January 17, 2025, 9:22am UTC](https://discuss.elastic.co/t/centralized-managed-logstash-pipeline-issue/373268 "2025-01-17T09:22:58Z")
We use the centralized pipeline manager in Kibana to update and upload Logstash pipeline(s) to our Logstash server. In general, everything works fine, but in the case an output filter is unable to process an event due to… --- ## [No index created in Elasticsearch. Can see logs in Docker](https://discuss.elastic.co/t/no-index-created-in-elasticsearch-can-see-logs-in-docker/372992)
**Author:** [@jcklam](https://discuss.elastic.co/u/jcklam)\ **Replies:** 4\ **Last updated:** [January 16, 2025, 2:05pm UTC](https://discuss.elastic.co/t/no-index-created-in-elasticsearch-can-see-logs-in-docker/372992 "2025-01-16T14:05:42Z")
Hi, I'm trying to get a simple ELK setup working with Docker. Issue is I don't see the index I've defined created docker-compose.yml I've verified that log files are copied over correctly onto the Docker container s… --- ## [How to Configure Logstash to Forward Data When New Data is Available in Elasticsearch?](https://discuss.elastic.co/t/how-to-configure-logstash-to-forward-data-when-new-data-is-available-in-elasticsearch/373151)
**Author:** [@Wei\_Li](https://discuss.elastic.co/u/Wei_Li)\ **Replies:** 1\ **Last updated:** [January 15, 2025, 4:54pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-forward-data-when-new-data-is-available-in-elasticsearch/373151 "2025-01-15T16:54:49Z")
Dear All. I am a beginner and I have the following questions to ask: Environment: I have a program that fetches data from a network device API every five minutes and stores it into an Elasticsearch index. The data in t… --- ## [Iterating over fields with multiple filter processes](https://discuss.elastic.co/t/iterating-over-fields-with-multiple-filter-processes/373187)
**Author:** [@philhagen](https://discuss.elastic.co/u/philhagen)\ **Replies:** 2\ **Last updated:** [January 14, 2025, 6:32pm UTC](https://discuss.elastic.co/t/iterating-over-fields-with-multiple-filter-processes/373187 "2025-01-14T18:32:35Z")
I run a series of consistent filters over a series of fields that may or may not exist. For consistency and to simplify maintenance, I'd like to do this with one code block instead of multiple repetitive ones. For a sp… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=11) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=13)