# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=8 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 9 --- ## [Logstash KV filter not working on Greedydata](https://discuss.elastic.co/t/logstash-kv-filter-not-working-on-greedydata/376764)
**Author:** [@Elk\_huh](https://discuss.elastic.co/u/Elk_huh)\ **Replies:** 4\ **Last updated:** [April 4, 2025, 12:40pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-not-working-on-greedydata/376764 "2025-04-04T12:40:44Z")
Raw logs \<190\>SCO-0000-CS01: 2025 Apr 3 17:53:36 UTC: %ACLLOG-6-ACLLOG: SGT: 0, Src IP: 1.1.1.1, Dst IP: 9.9.9.9, Src Port: 504, Dst Port: 443, Src Intf: port-channel1002, Protocol: "UDP"(17), ACL Name: alltraffic, ACE… --- ## [Logstash intermittently working](https://discuss.elastic.co/t/logstash-intermittently-working/375324)
**Author:** [@vaseemQA](https://discuss.elastic.co/u/vaseemQA)\ **Replies:** 3\ **Last updated:** [April 4, 2025, 9:14am UTC](https://discuss.elastic.co/t/logstash-intermittently-working/375324 "2025-04-04T09:14:07Z")
hi Team, I'm using Starting Logstash {"logstash.version"=\>"6.8.23"}, my es query is able to show the results in dev-tools, but when i run them through logstash from my local machine, its not working, let me give my samp… --- ## [Logstash Node stats API does not return anything for pipelines after upgrade to 8.17.3 (from 8.16.1)](https://discuss.elastic.co/t/logstash-node-stats-api-does-not-return-anything-for-pipelines-after-upgrade-to-8-17-3-from-8-16-1/376721)
**Author:** [@Dheeraj\_Gupta](https://discuss.elastic.co/u/Dheeraj_Gupta)\ **Replies:** 1\ **Last updated:** [April 4, 2025, 5:17am UTC](https://discuss.elastic.co/t/logstash-node-stats-api-does-not-return-anything-for-pipelines-after-upgrade-to-8-17-3-from-8-16-1/376721 "2025-04-04T05:17:22Z")
Hi, We have been using node stats API (pipelines end point) to monitor health of our multiple logstash pipelines. After we have upgraded from 8.16.1 to 8.17.3, the endpoint is returning pipelines: {} despite pipelines … --- ## [Unable to connect to Elastic Cloud using cloud\_id and cloud\_auth](https://discuss.elastic.co/t/unable-to-connect-to-elastic-cloud-using-cloud-id-and-cloud-auth/376745)
**Author:** [@Francesco\_Esposito](https://discuss.elastic.co/u/Francesco_Esposito)\ **Replies:** 1\ **Last updated:** [April 3, 2025, 1:52pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-elastic-cloud-using-cloud-id-and-cloud-auth/376745 "2025-04-03T13:52:40Z")
Good morning, I am trying to connect with Elasticsearch Output Plugin using cloud\_id and cloud\_auth. I receive an error message about permissions to access https://my\_cloud\_instance/\_license, looks like, for a grant pro… --- ## [LOGSTASH - wrong pipeline processor Pipeline](https://discuss.elastic.co/t/logstash-wrong-pipeline-processor-pipeline/376423)
**Author:** [@Echo\_01](https://discuss.elastic.co/u/Echo_01)\ **Replies:** 11\ **Last updated:** [April 2, 2025, 2:27pm UTC](https://discuss.elastic.co/t/logstash-wrong-pipeline-processor-pipeline/376423 "2025-04-02T14:27:32Z")
Having update a three node Elasticsearch cluster from 8.12.1 to 8.17.3 Logstash is generating error logs containing the following: , :error=\>{"type"=\>"illegal\_state\_exception", "reason"=\>"Pipeline processor configured f… --- ## [Logstash authenticating Error inAAD](https://discuss.elastic.co/t/logstash-authenticating-error-inaad/376627)
**Author:** [@LUMAL](https://discuss.elastic.co/u/LUMAL)\ **Replies:** 0\ **Last updated:** [April 1, 2025, 9:37am UTC](https://discuss.elastic.co/t/logstash-authenticating-error-inaad/376627 "2025-04-01T09:37:56Z")
I´m using Logstash to ingest information in Auxiliary logs and Sentinel the problem I have is that I´m receiving a \*Error while authenticating with AAD \[Excon::Error::BadRequest: '400', Response: '{"error":"unauthorized\_… --- ## [Logstash Grok isnt working](https://discuss.elastic.co/t/logstash-grok-isnt-working/376599)
**Author:** [@Elk\_huh](https://discuss.elastic.co/u/Elk_huh)\ **Replies:** 3\ **Last updated:** [March 31, 2025, 11:01pm UTC](https://discuss.elastic.co/t/logstash-grok-isnt-working/376599 "2025-03-31T23:01:41Z")
I am trying to parse this but it isnt working Raw Logs \<187\>SCO-N9504-CS01: 2025 Mar 31 18:34:03 UTC: %AUTHPRIV-3-SYSTEM\_MSG: pam\_aaa:Authentication failed from console - login (message repeated 1 time) Code filte… --- ## [Logstash 8.17.4 filter filter plugin name elasticsearch not found](https://discuss.elastic.co/t/logstash-8-17-4-filter-filter-plugin-name-elasticsearch-not-found/376533)
**Author:** [@dramis](https://discuss.elastic.co/u/dramis)\ **Replies:** 3\ **Last updated:** [March 31, 2025, 4:16pm UTC](https://discuss.elastic.co/t/logstash-8-17-4-filter-filter-plugin-name-elasticsearch-not-found/376533 "2025-03-31T16:16:13Z")
Hello, I just upgrade logstash to 8.17.4 and I got a error: \[2025-03-28T10:26:23,030\]\[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:filelogger, :except… --- ## [Logstash Kusto pipeline shows Uploading Failed for local file](https://discuss.elastic.co/t/logstash-kusto-pipeline-shows-uploading-failed-for-local-file/376593)
**Author:** [@Disha\_Bodade](https://discuss.elastic.co/u/Disha_Bodade)\ **Replies:** 0\ **Last updated:** [March 31, 2025, 3:32pm UTC](https://discuss.elastic.co/t/logstash-kusto-pipeline-shows-uploading-failed-for-local-file/376593 "2025-03-31T15:32:04Z")
Hi Team, Not sure it is right place to ask this. We have closed environment which is not exposed to internet. And I have configured to send logs to Azure Data Explorer via Logstash Kusto output plugin. We allowed traf… --- ## [How to deal with duplicate data](https://discuss.elastic.co/t/how-to-deal-with-duplicate-data/375229)
**Author:** [@ksobon](https://discuss.elastic.co/u/ksobon)\ **Replies:** 6\ **Last updated:** [March 31, 2025, 2:24pm UTC](https://discuss.elastic.co/t/how-to-deal-with-duplicate-data/375229 "2025-03-31T14:24:17Z")
I have a Filebeat pipeline that is ingesting data from an end-user machine that might be stored there for 30 days. My Logstash pipeline has the following settings: document\_id =\> "%{\[@metadata\]\[newId\]}" action =\> "creat… --- ## [NMAP Codec Plugin - Logstash Configuration and Index Template Issues](https://discuss.elastic.co/t/nmap-codec-plugin-logstash-configuration-and-index-template-issues/376547)
**Author:** [@ljonesa](https://discuss.elastic.co/u/ljonesa)\ **Replies:** 2\ **Last updated:** [March 29, 2025, 8:23pm UTC](https://discuss.elastic.co/t/nmap-codec-plugin-logstash-configuration-and-index-template-issues/376547 "2025-03-29T20:23:16Z")
Hello, I have been trying to take an Nmap scan's XML output into Elasticsearch and I have come across a few issues. Below is my Logstash configuration file. input { file { mode =\> "tail" path =\> "/usr/share/logs… --- ## [Cloudwatch Output -Log group Name](https://discuss.elastic.co/t/cloudwatch-output-log-group-name/376526)
**Author:** [@paw2025](https://discuss.elastic.co/u/paw2025)\ **Replies:** 2\ **Last updated:** [March 28, 2025, 6:26pm UTC](https://discuss.elastic.co/t/cloudwatch-output-log-group-name/376526 "2025-03-28T18:26:15Z")
Hi, I'm trying to send logs to Cloudwatch via logstash. How can I define Log group name in Cloudwatch Output? I don't see any option. Is there any default name for it? \[2025-03-27T21:34:33,162\]\[ERROR\]\[logstash.output… --- ## [Check event format against template](https://discuss.elastic.co/t/check-event-format-against-template/376451)
**Author:** [@ddoroshenko](https://discuss.elastic.co/u/ddoroshenko)\ **Replies:** 4\ **Last updated:** [March 28, 2025, 10:13am UTC](https://discuss.elastic.co/t/check-event-format-against-template/376451 "2025-03-28T10:13:03Z")
Hi, is it possible to check events format against certain template in Logstash? For example I want to make sure that event contains certain mandatory fields, i.e. "environment", "application" etc. For example I want t… --- ## [Logstash.inputs.tcp certificate\_expired and unknown\_ca](https://discuss.elastic.co/t/logstash-inputs-tcp-certificate-expired-and-unknown-ca/376484)
**Author:** [@mikewillis](https://discuss.elastic.co/u/mikewillis)\ **Replies:** 0\ **Last updated:** [March 27, 2025, 2:08pm UTC](https://discuss.elastic.co/t/logstash-inputs-tcp-certificate-expired-and-unknown-ca/376484 "2025-03-27T14:08:30Z")
Logstash 7.17 Can anyone offer any insight in to what would Logstash, acting as a server, to log this? \[2025-03-06T11:19:54,746\]\[ERROR\]\[logstash.inputs.tcp \] /10.70.12.45:38396: closing due: io.netty.handler.codec… --- ## [Filebeat or Logstash or Elastic Agent ? RPM or tarball?](https://discuss.elastic.co/t/filebeat-or-logstash-or-elastic-agent-rpm-or-tarball/376482)
**Author:** [@TheLotusMind](https://discuss.elastic.co/u/TheLotusMind)\ **Replies:** 0\ **Last updated:** [March 27, 2025, 2:03pm UTC](https://discuss.elastic.co/t/filebeat-or-logstash-or-elastic-agent-rpm-or-tarball/376482 "2025-03-27T14:03:08Z")
Hello, New to ELK trying to educate myself and find my way around... I "inherited" a project where I have to update ELK (and the OS of the VMs it's running on) to the latest version. Our setup consists of a few hundre… --- ## [Export logs from ELK stack to external destination](https://discuss.elastic.co/t/export-logs-from-elk-stack-to-external-destination/376472)
**Author:** [@IlleApprentice](https://discuss.elastic.co/u/IlleApprentice)\ **Replies:** 2\ **Last updated:** [March 27, 2025, 2:00pm UTC](https://discuss.elastic.co/t/export-logs-from-elk-stack-to-external-destination/376472 "2025-03-27T14:00:38Z")
Hi everyone, I am writing to you because I would need to export logs from inside elk to outside, like to blob in azure or any other destination point. Do you know any solution to date available. Thank you very much! --- ## [Logstash not picking/not processing all files with elasticsearch output plugin](https://discuss.elastic.co/t/logstash-not-picking-not-processing-all-files-with-elasticsearch-output-plugin/376396)
**Author:** [@Francesco\_Esposito](https://discuss.elastic.co/u/Francesco_Esposito)\ **Replies:** 4\ **Last updated:** [March 26, 2025, 2:02pm UTC](https://discuss.elastic.co/t/logstash-not-picking-not-processing-all-files-with-elasticsearch-output-plugin/376396 "2025-03-26T14:02:31Z")
Hello again. I am testing Logstash options and I am now creating this configuration: I am using 2 pipelines, one creates files with this configuration: input { http { port =\> 6043 } } output { file { pat… --- ## [Failed to publish events, Logstash and Agent](https://discuss.elastic.co/t/failed-to-publish-events-logstash-and-agent/376388)
**Author:** [@Viktor\_Movita](https://discuss.elastic.co/u/Viktor_Movita)\ **Replies:** 3\ **Last updated:** [March 26, 2025, 9:04am UTC](https://discuss.elastic.co/t/failed-to-publish-events-logstash-and-agent/376388 "2025-03-26T09:04:59Z")
Hello, I am trying to deploy a Logstash service on OCP using an Elastic-provided image: elastic/logstash:8.16.3 The service needs to receive input data from elastic-agents and forward it accordingly. My input pipeline… --- ## [Rollover daily at night 12 am](https://discuss.elastic.co/t/rollover-daily-at-night-12-am/376367)
**Author:** [@Vaishnavi\_Batgeri](https://discuss.elastic.co/u/Vaishnavi_Batgeri)\ **Replies:** 6\ **Last updated:** [March 26, 2025, 6:10am UTC](https://discuss.elastic.co/t/rollover-daily-at-night-12-am/376367 "2025-03-26T06:10:21Z")
I have created rollover indices for different applications, as I started the logstash service file at 8 PM, all the indices are getting created at that time only. However, there is a data mismatch, for instance: if the … --- ## [Logstash - Sending "\_id" in my document generates error](https://discuss.elastic.co/t/logstash-sending-id-in-my-document-generates-error/376343)
**Author:** [@Francesco\_Esposito](https://discuss.elastic.co/u/Francesco_Esposito)\ **Replies:** 3\ **Last updated:** [March 25, 2025, 8:18pm UTC](https://discuss.elastic.co/t/logstash-sending-id-in-my-document-generates-error/376343 "2025-03-25T20:18:35Z")
I have logstash using file input and elasticsearch output plugins and configured this way input { file { path =\> "D:/log\_streaming/my\_app/\*.log" start\_position =\> "beginning" sincedb\_path =\> "NUL" mode =\> "read… --- ## [Logstash exec timeout](https://discuss.elastic.co/t/logstash-exec-timeout/376389)
**Author:** [@O\_O\_O](https://discuss.elastic.co/u/O_O_O)\ **Replies:** 0\ **Last updated:** [March 25, 2025, 4:03pm UTC](https://discuss.elastic.co/t/logstash-exec-timeout/376389 "2025-03-25T16:03:55Z")
I have a long running task executed by exec input plugin, if the current task takes longer then the interval that I have set, does it simply cancel the current task and start a new one, or does it spawn a new child proce… --- ## [Logstash 8.12.2 - CSV Input to HTTP Output - Pipeline stuck looping](https://discuss.elastic.co/t/logstash-8-12-2-csv-input-to-http-output-pipeline-stuck-looping/376346)
**Author:** [@Steven\_LS](https://discuss.elastic.co/u/Steven_LS)\ **Replies:** 6\ **Last updated:** [March 25, 2025, 1:56pm UTC](https://discuss.elastic.co/t/logstash-8-12-2-csv-input-to-http-output-pipeline-stuck-looping/376346 "2025-03-25T13:56:50Z")
I have a logstash instance I am attempting to troubleshoot, where when I launch Logstash in debug mode, it looks like logstash is looping on no cgroup found. I've looked up and followed advice on swapping the sincedb pat… --- ## [Http\_poller and API Key ID and API KEY](https://discuss.elastic.co/t/http-poller-and-api-key-id-and-api-key/376381)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 0\ **Last updated:** [March 25, 2025, 1:55pm UTC](https://discuss.elastic.co/t/http-poller-and-api-key-id-and-api-key/376381 "2025-03-25T13:55:41Z")
Good Day, How do I send an API KEY ID and an API KEY with http\_poller. I am trying to replicate the following curl command which retrieves valid data but with http\_poller. -H 'Accept: application/json' \\ -H 'Content-T… --- ## [Unable to consume messages from kafka using logstash at a high throughput](https://discuss.elastic.co/t/unable-to-consume-messages-from-kafka-using-logstash-at-a-high-throughput/376325)
**Author:** [@optimus\_prime](https://discuss.elastic.co/u/optimus_prime)\ **Replies:** 3\ **Last updated:** [March 25, 2025, 4:24am UTC](https://discuss.elastic.co/t/unable-to-consume-messages-from-kafka-using-logstash-at-a-high-throughput/376325 "2025-03-25T04:24:59Z")
Hi I am currently load testing my logstash to consume events from kafka and index to opensearch. In our production environment we get around 300k events per seco nd so I was trying to replicate the same. I was able to p… --- ## [Saving just the "message" to file using http input and file output plugins](https://discuss.elastic.co/t/saving-just-the-message-to-file-using-http-input-and-file-output-plugins/376340)
**Author:** [@Francesco\_Esposito](https://discuss.elastic.co/u/Francesco_Esposito)\ **Replies:** 6\ **Last updated:** [March 24, 2025, 8:21pm UTC](https://discuss.elastic.co/t/saving-just-the-message-to-file-using-http-input-and-file-output-plugins/376340 "2025-03-24T20:21:12Z")
Hello, I am trying to store the http input "message" to file for an http call sending: { "index" : { "\_index" : "journaling\_insert","\_id":"A5CC1A05-09B9-4688-9796-14BB9E8A95FC"}} {"REMOTEIP":"1.111.1.11","CHAINCODE":"8… --- ## [Logstash reading events from March 14th on March 16th](https://discuss.elastic.co/t/logstash-reading-events-from-march-14th-on-march-16th/375982)
**Author:** [@devops\_training](https://discuss.elastic.co/u/devops_training)\ **Replies:** 10\ **Last updated:** [March 24, 2025, 12:48pm UTC](https://discuss.elastic.co/t/logstash-reading-events-from-march-14th-on-march-16th/375982 "2025-03-24T12:48:05Z")
\[2025-03-16T00:07:37,943\]\[INFO \]\[logstash.outputs.opensearch\]\[main\]\[f636cb73983bcc650332a7ed143a2c1655e1be6017f7947da7fc02b194adad2d\] Retrying failed action {:status=\>429, :action=\>\["index", {:\_id=\>nil, :\_index=\>"http\_se… --- ## [Idrac logs doesn't show SN or IP in logs. Can't figure out which idrac the log belongs to](https://discuss.elastic.co/t/idrac-logs-doesnt-show-sn-or-ip-in-logs-cant-figure-out-which-idrac-the-log-belongs-to/376311)
**Author:** [@Vitalii117](https://discuss.elastic.co/u/Vitalii117)\ **Replies:** 0\ **Last updated:** [March 24, 2025, 10:25am UTC](https://discuss.elastic.co/t/idrac-logs-doesnt-show-sn-or-ip-in-logs-cant-figure-out-which-idrac-the-log-belongs-to/376311 "2025-03-24T10:25:50Z")
Hi! I have moved logs from many IDRACs to ELK stack, did normal logstash setup but logs don't show any serial number or IP of Idrac so I can't figure out which idrac the log belongs to. Please help me to point out whic… --- ## [Integrating Avolution ABACUS (.odb files ) Data with Elasticsearch: Need Guidance](https://discuss.elastic.co/t/integrating-avolution-abacus-odb-files-data-with-elasticsearch-need-guidance/376292)
**Author:** [@rijo\_jose](https://discuss.elastic.co/u/rijo_jose)\ **Replies:** 1\ **Last updated:** [March 24, 2025, 5:23am UTC](https://discuss.elastic.co/t/integrating-avolution-abacus-odb-files-data-with-elasticsearch-need-guidance/376292 "2025-03-24T05:23:10Z")
I have an important project to integrate of data generated from Avolution ABACUS (Enterprise Architect tool) simulations with Elasticsearch. I understand that ABACUS primarily produces output in the .odb file format. We … --- ## [Logstash Kafka Output with Azure Managed Identity Failing with ClassNotFoundException](https://discuss.elastic.co/t/logstash-kafka-output-with-azure-managed-identity-failing-with-classnotfoundexception/376285)
**Author:** [@Sujay\_Babu\_Sekar](https://discuss.elastic.co/u/Sujay_Babu_Sekar)\ **Replies:** 0\ **Last updated:** [March 23, 2025, 7:29pm UTC](https://discuss.elastic.co/t/logstash-kafka-output-with-azure-managed-identity-failing-with-classnotfoundexception/376285 "2025-03-23T19:29:45Z")
Hello everyone, I'm trying to configure Logstash to send data to an Azure Event Hub using Kafka output with Azure Managed Identity authentication. I'm using a custom Kafka OAuthBearer class for Azure authentication, whi… --- ## [Multiple Labels for same field for SNMP](https://discuss.elastic.co/t/multiple-labels-for-same-field-for-snmp/375642)
**Author:** [@Sohaib\_Khan](https://discuss.elastic.co/u/Sohaib_Khan)\ **Replies:** 3\ **Last updated:** [March 23, 2025, 1:57pm UTC](https://discuss.elastic.co/t/multiple-labels-for-same-field-for-snmp/375642 "2025-03-23T13:57:22Z")
Hi, I'm using SNMP-Logstash plugin to get data from SNMP. It works fine to an extent, The issues comes in for a field that has multiple values, example "processorusage" as i have 2 fortigate devices but it is fetching a… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=7) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=9)