# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=4 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 5 --- ## [Logstash nested JSON Array, split and parse on nested JSON without using Ruby code](https://discuss.elastic.co/t/logstash-nested-json-array-split-and-parse-on-nested-json-without-using-ruby-code/380985)
**Author:** [@yelinaung](https://discuss.elastic.co/u/yelinaung)\ **Replies:** 2\ **Last updated:** [August 14, 2025, 2:36am UTC](https://discuss.elastic.co/t/logstash-nested-json-array-split-and-parse-on-nested-json-without-using-ruby-code/380985 "2025-08-14T02:36:14Z")
How to split separate field on nested JSON Array. Can someone kindly help with this? Thank you. Below is the nested JSON array Sample JSON Code "http": { "request": { "headers": \[ { "Name": "Host", … --- ## [Duplicate records in Elasticsearch](https://discuss.elastic.co/t/duplicate-records-in-elasticsearch/380625)
**Author:** [@SamehSaeed](https://discuss.elastic.co/u/SamehSaeed)\ **Replies:** 12\ **Last updated:** [August 10, 2025, 11:31am UTC](https://discuss.elastic.co/t/duplicate-records-in-elasticsearch/380625 "2025-08-10T11:31:09Z")
Hello I'm facing an issue with duplicate records in my index on rollover. I thought using rollover alias would let only one index to be "write" index which is supposed to not let any batch of records being indexed t… --- ## [Rename fields name with Logstash](https://discuss.elastic.co/t/rename-fields-name-with-logstash/380861)
**Author:** [@yelinaung](https://discuss.elastic.co/u/yelinaung)\ **Replies:** 11\ **Last updated:** [August 8, 2025, 9:30am UTC](https://discuss.elastic.co/t/rename-fields-name-with-logstash/380861 "2025-08-08T09:30:48Z")
How to rename for clarity Can someone kindly help with this? Thank you. filter { mutate { rename =\> { "\[test\]\[rex-platform"\\\\" \]" =\> "\[test\]\[rex-platform\]" } } } Output Di… --- ## [Unable to send logs from Filebeat to Logstash – EOF on TCP connection](https://discuss.elastic.co/t/unable-to-send-logs-from-filebeat-to-logstash-eof-on-tcp-connection/380791)
**Author:** [@anhtaw](https://discuss.elastic.co/u/anhtaw)\ **Replies:** 1\ **Last updated:** [August 7, 2025, 1:09pm UTC](https://discuss.elastic.co/t/unable-to-send-logs-from-filebeat-to-logstash-eof-on-tcp-connection/380791 "2025-08-07T13:09:44Z")
I'm deploying Logstash on EKS and trying to expose it on port 443 using HAProxy Ingress. I'm using Filebeat to send logs to Logstash via the Beats protocol (Lumberjack), but I’m getting the following errors: Failed to c… --- ## [How to send logs from Azure service bus to Elasticsearch through Logstash](https://discuss.elastic.co/t/how-to-send-logs-from-azure-service-bus-to-elasticsearch-through-logstash/380801)
**Author:** [@Farheen](https://discuss.elastic.co/u/Farheen)\ **Replies:** 1\ **Last updated:** [August 6, 2025, 10:30am UTC](https://discuss.elastic.co/t/how-to-send-logs-from-azure-service-bus-to-elasticsearch-through-logstash/380801 "2025-08-06T10:30:21Z")
How to send logs from Azure service bus to Elasticsearch through Logstash --- ## [How to create ingress TCP for logstash beats](https://discuss.elastic.co/t/how-to-create-ingress-tcp-for-logstash-beats/380790)
**Author:** [@anhtaw](https://discuss.elastic.co/u/anhtaw)\ **Replies:** 0\ **Last updated:** [August 6, 2025, 8:10am UTC](https://discuss.elastic.co/t/how-to-create-ingress-tcp-for-logstash-beats/380790 "2025-08-06T08:10:54Z")
I'm deploying Logstash on EKS and trying to expose it on port 443 using HAProxy Ingress. I'm using Filebeat to send logs to Logstash via the Beats protocol (Lumberjack), but I’m getting the following errors: Failed to c… --- ## [Store json object in a field in logstash](https://discuss.elastic.co/t/store-json-object-in-a-field-in-logstash/380748)
**Author:** [@alex\_petrov](https://discuss.elastic.co/u/alex_petrov)\ **Replies:** 1\ **Last updated:** [August 5, 2025, 10:03am UTC](https://discuss.elastic.co/t/store-json-object-in-a-field-in-logstash/380748 "2025-08-05T10:03:12Z")
consider this log : { "timestamp": "2025-08-04T12:32:50.9698972+03:30", "body": { "instanceId": "abc123", "variable": "testVar", "testInt": 42, "testBool": true, "data": { "ex… --- ## [Problems reading file by logstash input plugin (permissions)](https://discuss.elastic.co/t/problems-reading-file-by-logstash-input-plugin-permissions/380593)
**Author:** [@DetlefG](https://discuss.elastic.co/u/DetlefG)\ **Replies:** 4\ **Last updated:** [August 5, 2025, 8:53am UTC](https://discuss.elastic.co/t/problems-reading-file-by-logstash-input-plugin-permissions/380593 "2025-08-05T08:53:43Z")
Which permissions has to be set for a file read by input plugin of logstash ? Following situation: Logstash is running as service with user/group logstash/logstash. Input plugin of logstash should read the file /home/… --- ## [Logstash v8.14 compatibility with elastic search 8.19 , kibana 8.19 - RHEL 7](https://discuss.elastic.co/t/logstash-v8-14-compatibility-with-elastic-search-8-19-kibana-8-19-rhel-7/380650)
**Author:** [@Lakshmi1708](https://discuss.elastic.co/u/Lakshmi1708)\ **Replies:** 1\ **Last updated:** [August 1, 2025, 6:32am UTC](https://discuss.elastic.co/t/logstash-v8-14-compatibility-with-elastic-search-8-19-kibana-8-19-rhel-7/380650 "2025-08-01T06:32:42Z")
Does Logstash v8.14 works with Elastic search v8.19 and Kibana v8.19 all 3 hosted in RHEL 7 OS? --- ## [Filed names are not renaming in logstash pipeline](https://discuss.elastic.co/t/filed-names-are-not-renaming-in-logstash-pipeline/380583)
**Author:** [@upreddy253](https://discuss.elastic.co/u/upreddy253)\ **Replies:** 6\ **Last updated:** [July 31, 2025, 12:34pm UTC](https://discuss.elastic.co/t/filed-names-are-not-renaming-in-logstash-pipeline/380583 "2025-07-31T12:34:37Z")
Hi All, I am using mutate filter to rename the fields. But only one field is renamed remaining fields are not renamed. Below the logstash configuration. json { source =\> "jsonmessage" target =\> "\[x\]\[data\]" … --- ## [Certificate not accepted by logstash](https://discuss.elastic.co/t/certificate-not-accepted-by-logstash/380509)
**Author:** [@Tesla\_User](https://discuss.elastic.co/u/Tesla_User)\ **Replies:** 3\ **Last updated:** [July 29, 2025, 7:46pm UTC](https://discuss.elastic.co/t/certificate-not-accepted-by-logstash/380509 "2025-07-29T19:46:24Z")
Hi I am getting the below error while configuring a new certificate with Logstash. My old cert is working fine but the new one is not. Error: failed to perform request {:message=\>"certificate for \<10.10.x.x\> doesn't m… --- ## [Problem inserting Logstash process chain after setting up Filebeat-\>Elasticsearch-\>Kibana](https://discuss.elastic.co/t/problem-inserting-logstash-process-chain-after-setting-up-filebeat-elasticsearch-kibana/380542)
**Author:** [@dapig](https://discuss.elastic.co/u/dapig)\ **Replies:** 4\ **Last updated:** [July 29, 2025, 4:51pm UTC](https://discuss.elastic.co/t/problem-inserting-logstash-process-chain-after-setting-up-filebeat-elasticsearch-kibana/380542 "2025-07-29T16:51:48Z")
Hi all, This is my first time using Elasticsearch stack, I initially set up Filebeat, Elasticsearch, and Kibana to ingest the original free OSSEC alert.json file. However, the information was showing up in the message f… --- ## [Customized Logstash configuration for the existing LS service but process still showing the default path /etc/logstash, also, getting Warning: no jvm.options file found and WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/confi](https://discuss.elastic.co/t/customized-logstash-configuration-for-the-existing-ls-service-but-process-still-showing-the-default-path-etc-logstash-also-getting-warning-no-jvm-options-file-found-and-warning-could-not-find-logstash-yml-which-is-typically-located-in-ls-home-confi/380396)
**Author:** [@venkat\_tammi](https://discuss.elastic.co/u/venkat_tammi)\ **Replies:** 2\ **Last updated:** [July 24, 2025, 7:25pm UTC](https://discuss.elastic.co/t/customized-logstash-configuration-for-the-existing-ls-service-but-process-still-showing-the-default-path-etc-logstash-also-getting-warning-no-jvm-options-file-found-and-warning-could-not-find-logstash-yml-which-is-typically-located-in-ls-home-confi/380396 "2025-07-24T19:25:36Z")
Friends, I did modify logstash configuration and setting files to my desired path, then, I am able to run logstash service up and running. However, I can see default path /etc/logstash in the output of "ps -ef | grep log… --- ## [S3 input plugin sincedb not wprk](https://discuss.elastic.co/t/s3-input-plugin-sincedb-not-wprk/380406)
**Author:** [@Lich](https://discuss.elastic.co/u/Lich)\ **Replies:** 1\ **Last updated:** [July 24, 2025, 3:20am UTC](https://discuss.elastic.co/t/s3-input-plugin-sincedb-not-wprk/380406 "2025-07-24T03:20:23Z")
My conf: /etc/logstash/conf.d/test.conf input { s3 { access\_key\_id =\> "" secret\_access\_key =\> "" endpoint =\> "" bucket =\> "" prefix =\> "" region =\> "" codec … --- ## [Streaming logs with csv issue](https://discuss.elastic.co/t/streaming-logs-with-csv-issue/380360)
**Author:** [@Idan\_Klatza](https://discuss.elastic.co/u/Idan_Klatza)\ **Replies:** 3\ **Last updated:** [July 23, 2025, 9:27pm UTC](https://discuss.elastic.co/t/streaming-logs-with-csv-issue/380360 "2025-07-23T21:27:39Z")
Hi, I want to stream logs from Postgres to elk but it doesn't parse the log properly. Logstash.conf file: input { file { path =\> "/path/to/log.csv" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } }… --- ## [Adding a new field in logstash](https://discuss.elastic.co/t/adding-a-new-field-in-logstash/380333)
**Author:** [@DOkuwa](https://discuss.elastic.co/u/DOkuwa)\ **Replies:** 4\ **Last updated:** [July 23, 2025, 7:01am UTC](https://discuss.elastic.co/t/adding-a-new-field-in-logstash/380333 "2025-07-23T07:01:04Z")
Hi, I have this input file sending data to logstash and want to add a new field name as seen below I dont want to use filter { mutate { add\_field =\> { "new\_field\_name" =\> "new\_field\_value" } } } such as add\_fiel… --- ## [Generate UUID in input plugin](https://discuss.elastic.co/t/generate-uuid-in-input-plugin/380356)
**Author:** [@Francesco\_Esposito](https://discuss.elastic.co/u/Francesco_Esposito)\ **Replies:** 1\ **Last updated:** [July 22, 2025, 7:26pm UTC](https://discuss.elastic.co/t/generate-uuid-in-input-plugin/380356 "2025-07-22T19:26:48Z")
Hello everyone, I am trying to execute a query thru jdbc input plugin and I need to generate a unique ID to pass in my stored procedure to identify my "flight", my single execution. Is it possible? --- ## [Message field in logstash pipeline](https://discuss.elastic.co/t/message-field-in-logstash-pipeline/380304)
**Author:** [@TheJ](https://discuss.elastic.co/u/TheJ)\ **Replies:** 3\ **Last updated:** [July 22, 2025, 12:09pm UTC](https://discuss.elastic.co/t/message-field-in-logstash-pipeline/380304 "2025-07-22T12:09:35Z")
Hi, is this possible in logstash to change message match field to other name ? I mean I have input and filter section in logstash pipeline as follow: input { udp { host =\> "0.0.0.0" port =\> 895 } } filte… --- ## [Viewing progress( sample :1 out of 7, 2 out of 7)of a test execution using the logs pushed via logstash to Kibana](https://discuss.elastic.co/t/viewing-progress-sample-1-out-of-7-2-out-of-7-of-a-test-execution-using-the-logs-pushed-via-logstash-to-kibana/380273)
**Author:** [@MahaQA](https://discuss.elastic.co/u/MahaQA)\ **Replies:** 4\ **Last updated:** [July 22, 2025, 8:54am UTC](https://discuss.elastic.co/t/viewing-progress-sample-1-out-of-7-2-out-of-7-of-a-test-execution-using-the-logs-pushed-via-logstash-to-kibana/380273 "2025-07-22T08:54:07Z")
I have a requirement to view the status of my automation suite(eg in real time, I want to view 2 of 7 completed, 3 of 7 completed etc), and alerts if any failed test case failed. following logs I am pushing to Kibana. … --- ## [Process decoded data in more readable format](https://discuss.elastic.co/t/process-decoded-data-in-more-readable-format/380213)
**Author:** [@dashelastic](https://discuss.elastic.co/u/dashelastic)\ **Replies:** 7\ **Last updated:** [July 18, 2025, 11:51am UTC](https://discuss.elastic.co/t/process-decoded-data-in-more-readable-format/380213 "2025-07-18T11:51:32Z")
the decoded data from data streaming is coming into logstash. "decoded": \[ "{\\"headers\\":{\\"eventType\\":\\"com.data.globalIdentity.verb\\",\\"operation\\":\\"UPDATE\\",\\"messageType\\":\\"IDENTITY\\",\\"eventTime\\":\\"2025-07-… --- ## [Regarding pipeline.buffer.type](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 7\ **Last updated:** [July 18, 2025, 8:49am UTC](https://discuss.elastic.co/t/regarding-pipeline-buffer-type/379798 "2025-07-18T08:49:12Z")
Hi, Recently we've been enabling heap based processing of the beats input as you can see here: pipeline.workers: 2 pipeline.batch.size: 100 pipeline.buffer.type: heap However, we still see errors for some sites which … --- ## [Forward logs in CEF format to another host](https://discuss.elastic.co/t/forward-logs-in-cef-format-to-another-host/380226)
**Author:** [@juancamiloll](https://discuss.elastic.co/u/juancamiloll)\ **Replies:** 0\ **Last updated:** [July 17, 2025, 8:16pm UTC](https://discuss.elastic.co/t/forward-logs-in-cef-format-to-another-host/380226 "2025-07-17T20:16:06Z")
Hello, I am currently receiving the logs from fortinet through the integration “Fortinet FortiGate Firewall Logs” which works without problem. If I log into logstash and run a tcpdump I can see the logs being received. … --- ## [Grok and parsing of logstash data](https://discuss.elastic.co/t/grok-and-parsing-of-logstash-data/379408)
**Author:** [@DOkuwa](https://discuss.elastic.co/u/DOkuwa)\ **Replies:** 15\ **Last updated:** [July 17, 2025, 10:43am UTC](https://discuss.elastic.co/t/grok-and-parsing-of-logstash-data/379408 "2025-07-17T10:43:26Z")
Good morning I am integrating data from an application called telegraf into logstash here is my telegraf conf file # Global Agent Configuration \[agent\] hostname = "" flush\_interval = "15s" interval = "1500s" … --- ## [Read files from SharePoint using logstash](https://discuss.elastic.co/t/read-files-from-sharepoint-using-logstash/380196)
**Author:** [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)\ **Replies:** 0\ **Last updated:** [July 17, 2025, 4:43am UTC](https://discuss.elastic.co/t/read-files-from-sharepoint-using-logstash/380196 "2025-07-17T04:43:21Z")
Hi Team, Could you please let me know if there is a way to read the files from SharePoint using logstash. Thankyou in advance. --- ## [How to pull data from Elasticsearch to Splunk without data duplication?](https://discuss.elastic.co/t/how-to-pull-data-from-elasticsearch-to-splunk-without-data-duplication/380000)
**Author:** [@Ella\_conan](https://discuss.elastic.co/u/Ella_conan)\ **Replies:** 16\ **Last updated:** [July 16, 2025, 12:59pm UTC](https://discuss.elastic.co/t/how-to-pull-data-from-elasticsearch-to-splunk-without-data-duplication/380000 "2025-07-16T12:59:22Z")
Hello, I have data in Elasticsearch that I’ve stored across 3 indices. I want to copy the data to Splunk — I want only the data from the last minute, and I want to store the data from each Elasticsearch index into a se… --- ## [Run Logstash as service on RedHat](https://discuss.elastic.co/t/run-logstash-as-service-on-redhat/380134)
**Author:** [@andp](https://discuss.elastic.co/u/andp)\ **Replies:** 5\ **Last updated:** [July 15, 2025, 12:41pm UTC](https://discuss.elastic.co/t/run-logstash-as-service-on-redhat/380134 "2025-07-15T12:41:53Z")
Hi, I have unpack (as application user that will run it) logstash 7.16 to my applications folder on RedHat server. I made changes in logstash.conf file and to try I started logstash in command line like this: /applica… --- ## [Retrying http request, will sleep for X seconds](https://discuss.elastic.co/t/retrying-http-request-will-sleep-for-x-seconds/380085)
**Author:** [@Venality](https://discuss.elastic.co/u/Venality)\ **Replies:** 5\ **Last updated:** [July 15, 2025, 1:20am UTC](https://discuss.elastic.co/t/retrying-http-request-will-sleep-for-x-seconds/380085 "2025-07-15T01:20:34Z")
I configured my logstash.conf file properly in terms of URL and Authorization token however It's still not sending logs and giving me an http request failure like it's not able to communicate. I came to the conclusion th… --- ## [Logstash filter doesn't appear to be functioning](https://discuss.elastic.co/t/logstash-filter-doesnt-appear-to-be-functioning/380108)
**Author:** [@bmf614zzz](https://discuss.elastic.co/u/bmf614zzz)\ **Replies:** 2\ **Last updated:** [July 14, 2025, 4:59pm UTC](https://discuss.elastic.co/t/logstash-filter-doesnt-appear-to-be-functioning/380108 "2025-07-14T16:59:54Z")
Hello, Logstash version: 6.8.23 data message: cmd\_logger\_api.c(260) 10987641 %% INFO \[CLI:backupguy:10.0.1.254\] User has logged out type: rsyslog Logstash filter: filter { if "backupguy" in \[message\] { drop {} } … --- ## [Standard grok patterns consisting of only a group](https://discuss.elastic.co/t/standard-grok-patterns-consisting-of-only-a-group/380106)
**Author:** [@frans-wtax](https://discuss.elastic.co/u/frans-wtax)\ **Replies:** 1\ **Last updated:** [July 14, 2025, 4:28pm UTC](https://discuss.elastic.co/t/standard-grok-patterns-consisting-of-only-a-group/380106 "2025-07-14T16:28:24Z")
The documentation for the Grok filter states that to add a pattern to a custom patterns file, you: write the pattern you need as the pattern name, a space, then the regexp for that pattern. For example, doing the pos… --- ## [Your settings are invalid. Reason: Setting "" doesn't exist. Please check if you haven't made a typo](https://discuss.elastic.co/t/your-settings-are-invalid-reason-setting-doesnt-exist-please-check-if-you-havent-made-a-typo/380079)
**Author:** [@Venality](https://discuss.elastic.co/u/Venality)\ **Replies:** 2\ **Last updated:** [July 13, 2025, 10:54pm UTC](https://discuss.elastic.co/t/your-settings-are-invalid-reason-setting-doesnt-exist-please-check-if-you-havent-made-a-typo/380079 "2025-07-13T22:54:19Z")
Hi guys, I'm trying to write the config file for logstash to communicate with splunk and I keep getting this error. Not sure what I'm doing wrong and could use some input. This is my first time using it so here's the co… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=3) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=5)