# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=52 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 53 --- ## [Logstash and since db permission](https://discuss.elastic.co/t/logstash-and-since-db-permission/346934)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 2\ **Last updated:** [November 14, 2023, 7:15am UTC](https://discuss.elastic.co/t/logstash-and-since-db-permission/346934 "2023-11-14T07:15:56Z")
Logstash runs as a container.logstash version 8.11.0 Logstash input looks like the below input { file { path =\> "/common/logs/parser-server-tasks-application/app.log" start\_position =\> "beginning" sincedb… --- ## [Is Logstash Free use?](https://discuss.elastic.co/t/is-logstash-free-use/347078)
**Author:** [@inbeom\_cho](https://discuss.elastic.co/u/inbeom_cho)\ **Replies:** 1\ **Last updated:** [November 14, 2023, 6:10am UTC](https://discuss.elastic.co/t/is-logstash-free-use/347078 "2023-11-14T06:10:55Z")
Hi forum, when i use only logstash 8.8 version, that is free? Or If i want free, must use logstash oss? --- ## [Logstash Kafka input handling extended JSON format](https://discuss.elastic.co/t/logstash-kafka-input-handling-extended-json-format/347068)
**Author:** [@ys\_goh](https://discuss.elastic.co/u/ys_goh)\ **Replies:** 1\ **Last updated:** [November 14, 2023, 1:49am UTC](https://discuss.elastic.co/t/logstash-kafka-input-handling-extended-json-format/347068 "2023-11-14T01:49:54Z")
Hello all, I am trying to get data from MongoDB to OpenSearch, and this is our pipeline: MongoDB ==\> Kafka source connector ==\> Kafka topic ==\> Logstash ==\> OpenSearch Problem is, when MongoDB data get written into the… --- ## [Logstash: SNMP Poll Input - skipping "error: no such.."](https://discuss.elastic.co/t/logstash-snmp-poll-input-skipping-error-no-such/347051)
**Author:** [@erikg](https://discuss.elastic.co/u/erikg)\ **Replies:** 2\ **Last updated:** [November 13, 2023, 10:59pm UTC](https://discuss.elastic.co/t/logstash-snmp-poll-input-skipping-error-no-such/347051 "2023-11-13T22:59:49Z")
Hello, I am using SNMP poll input for logstash. Using SNMP V3, I have a wide range of network devices to monitor which means many oids that are specific by vendor. I want to know if there is a way to skip oids in whic… --- ## [Logstash does not execute certain queries correctly](https://discuss.elastic.co/t/logstash-does-not-execute-certain-queries-correctly/346800)
**Author:** [@Stefan\_Sabolowitsch](https://discuss.elastic.co/u/Stefan_Sabolowitsch)\ **Replies:** 1\ **Last updated:** [November 13, 2023, 4:20pm UTC](https://discuss.elastic.co/t/logstash-does-not-execute-certain-queries-correctly/346800 "2023-11-13T16:20:49Z")
Hi there i do not understand the behavior of logstash. Although the field is\_read exists, a successful query is still performed and an e-mail is sent. input { elasticsearch { hosts =\> "https://elasti… --- ## [Logstash error: \[FATAL\]\[org.logstash.Logstash \] Logstash stopped processing because of an error: (SystemExit) exit org.jruby.exceptions.SystemExit: (SystemExit) exit](https://discuss.elastic.co/t/logstash-error-fatal-org-logstash-logstash-logstash-stopped-processing-because-of-an-error-systemexit-exit-org-jruby-exceptions-systemexit-systemexit-exit/347001)
**Author:** [@Jann](https://discuss.elastic.co/u/Jann)\ **Replies:** 0\ **Last updated:** [November 13, 2023, 11:12am UTC](https://discuss.elastic.co/t/logstash-error-fatal-org-logstash-logstash-logstash-stopped-processing-because-of-an-error-systemexit-exit-org-jruby-exceptions-systemexit-systemexit-exit/347001 "2023-11-13T11:12:28Z")
Hello, I'm trying to send txt files from my server to my other server (where ELK is running). Otherwise when I try to send files, I receive this error: \[FATAL\]\[org.logstash.Logstash \] Logstash stopped processing bec… --- ## [Error: failed to publish events: write tcp XX.XX.XX.XX:50882-\>XX.XX.XX.XX:5044: write: broken pipe](https://discuss.elastic.co/t/error-failed-to-publish-events-write-tcp-xx-xx-xx-xx-50882-xx-xx-xx-xx-write-broken-pipe/346852)
**Author:** [@charown](https://discuss.elastic.co/u/charown)\ **Replies:** 12\ **Last updated:** [November 13, 2023, 9:15am UTC](https://discuss.elastic.co/t/error-failed-to-publish-events-write-tcp-xx-xx-xx-xx-50882-xx-xx-xx-xx-write-broken-pipe/346852 "2023-11-13T09:15:26Z")
I have docker-compose.yml version: "2.4" services: … --- ## [Logstash and Kafka Input](https://discuss.elastic.co/t/logstash-and-kafka-input/346638)
**Author:** [@rpd](https://discuss.elastic.co/u/rpd)\ **Replies:** 7\ **Last updated:** [November 10, 2023, 7:48pm UTC](https://discuss.elastic.co/t/logstash-and-kafka-input/346638 "2023-11-10T19:48:36Z")
Hello Folks, I have a query about an observed side-effect of my Logstash kafka-input configuration. It is not directly apparent to me what the problem is and hope people with deep expertise can help me out here. We hav… --- ## [Ruby filter to pack string value into object](https://discuss.elastic.co/t/ruby-filter-to-pack-string-value-into-object/346854)
**Author:** [@rcz](https://discuss.elastic.co/u/rcz)\ **Replies:** 7\ **Last updated:** [November 10, 2023, 2:17pm UTC](https://discuss.elastic.co/t/ruby-filter-to-pack-string-value-into-object/346854 "2023-11-10T14:17:09Z")
I have a client that sends HTTP request events with a nested structure, like: context.response.body context.response.code context.response.headers.Content-Length context.response.headers.Content-Type etc.. But sometime… --- ## [Nested json with multi field parsing through logstash](https://discuss.elastic.co/t/nested-json-with-multi-field-parsing-through-logstash/345549)
**Author:** [@sudhir\_singh](https://discuss.elastic.co/u/sudhir_singh)\ **Replies:** 15\ **Last updated:** [November 10, 2023, 1:31pm UTC](https://discuss.elastic.co/t/nested-json-with-multi-field-parsing-through-logstash/345549 "2023-11-10T13:31:21Z")
Please help me with below log how do I parse it ? {"@timestamp":"2023-10-11T07:38:56.607Z","log.level":"error","message":"API REQUEST TIME","ecs":{"version":"1.6.0"},"requestedAPI":\["https://cloudservices.indiatimes.com… --- ## [ Logstash stopped processing because of an error: (SystemExit) exit Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit-logstash-stopped-processing-because-of-an-error-systemexit-exit/346805)
**Author:** [@17\_Chinmay\_Shelke](https://discuss.elastic.co/u/17_Chinmay_Shelke)\ **Replies:** 3\ **Last updated:** [November 10, 2023, 10:34am UTC](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit-logstash-stopped-processing-because-of-an-error-systemexit-exit/346805 "2023-11-10T10:34:33Z")
Successfully started Logstash API endpoint {:port=\>9600, :ssl\_enabled=\>false} \[2023-11-09T11:25:11,753\]\[INFO \]\[logstash.runner \] Logstash shut down. \[2023-11-09T11:25:11,758\]\[FATAL\]\[org.logstash.Logstash \] … --- ## [Prune filter does not work with whitelist but it does with blacklist](https://discuss.elastic.co/t/prune-filter-does-not-work-with-whitelist-but-it-does-with-blacklist/346549)
**Author:** [@elk-user-0001](https://discuss.elastic.co/u/elk-user-0001)\ **Replies:** 1\ **Last updated:** [November 9, 2023, 8:11pm UTC](https://discuss.elastic.co/t/prune-filter-does-not-work-with-whitelist-but-it-does-with-blacklist/346549 "2023-11-09T20:11:15Z")
Hello colleagues! I am trying to use the prune filter with first level fields ( I know the problem with nested fields ) but I can't get it to work. I have a json of 900 fields and I am interested in keeping only a few,… --- ## [Using Key-value(KV) with multiple Value splits](https://discuss.elastic.co/t/using-key-value-kv-with-multiple-value-splits/346527)
**Author:** [@robnew](https://discuss.elastic.co/u/robnew)\ **Replies:** 6\ **Last updated:** [November 9, 2023, 7:53pm UTC](https://discuss.elastic.co/t/using-key-value-kv-with-multiple-value-splits/346527 "2023-11-09T19:53:20Z")
I have a wineventlog-application log which has (ie) 'EventCode=33210 EventRecordID=12345' then changes to session\_id:69,server\_principal\_id:226,etc etc so from = to : with , instead of spaces. Is there a way I can use th… --- ## [Fingerprint for json does not get resolved](https://discuss.elastic.co/t/fingerprint-for-json-does-not-get-resolved/346772)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 9\ **Last updated:** [November 9, 2023, 5:12pm UTC](https://discuss.elastic.co/t/fingerprint-for-json-does-not-get-resolved/346772 "2023-11-09T17:12:08Z")
fingerprint for json is not working input { file { path =\> "/shared/logs/logi2/stats.\*" start\_position =\> "beginning" sincedb\_path =\> "/shared/logs/.sincedb" type =\> "logi2-stats" … --- ## [How to know wich grok is failing?](https://discuss.elastic.co/t/how-to-know-wich-grok-is-failing/346535)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 2\ **Last updated:** [November 9, 2023, 3:46pm UTC](https://discuss.elastic.co/t/how-to-know-wich-grok-is-failing/346535 "2023-11-09T15:46:11Z")
Hi, Im reviewing the pipeline of an ex colleague, and there is almos 30 grok filters, wich will be the best way to identify wich grok is failing? Im using stdout in the output. Thanks! --- ## [Azure EventHub Plugin for Logstash Errors](https://discuss.elastic.co/t/azure-eventhub-plugin-for-logstash-errors/346811)
**Author:** [@Kris\_Felscher](https://discuss.elastic.co/u/Kris_Felscher)\ **Replies:** 0\ **Last updated:** [November 9, 2023, 2:42pm UTC](https://discuss.elastic.co/t/azure-eventhub-plugin-for-logstash-errors/346811 "2023-11-09T14:42:49Z")
We have Logstash installed on Kubernetes running on 2 pods. My main pipeline is configured to receive events from 2 separate EventHub instances. Here's my Pipeline Input: input { azure\_event\_hubs { config\_m… --- ## [Extract substring from the path](https://discuss.elastic.co/t/extract-substring-from-the-path/346787)
**Author:** [@Xhar](https://discuss.elastic.co/u/Xhar)\ **Replies:** 2\ **Last updated:** [November 9, 2023, 12:43pm UTC](https://discuss.elastic.co/t/extract-substring-from-the-path/346787 "2023-11-09T12:43:16Z")
in this config input { file { mode =\> "read" path =\> "/opt/stromReciever/parsed\_data/changedRights/csv/\*.json" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" codec =\> "json" type =\> … --- ## [Custom fields creation in jira using elasticsearch](https://discuss.elastic.co/t/custom-fields-creation-in-jira-using-elasticsearch/346759)
**Author:** [@Kumar\_6](https://discuss.elastic.co/u/Kumar_6)\ **Replies:** 0\ **Last updated:** [November 9, 2023, 6:27am UTC](https://discuss.elastic.co/t/custom-fields-creation-in-jira-using-elasticsearch/346759 "2023-11-09T06:27:51Z")
Hi, Can some one help to fix this issue. I want to create custom fields in jira by passing data from jira connecter in kibana. --- ## [Logstash is not up & running on MacOS](https://discuss.elastic.co/t/logstash-is-not-up-running-on-macos/346741)
**Author:** [@inandi](https://discuss.elastic.co/u/inandi)\ **Replies:** 1\ **Last updated:** [November 9, 2023, 1:11am UTC](https://discuss.elastic.co/t/logstash-is-not-up-running-on-macos/346741 "2023-11-09T01:11:32Z")
(in Docker) Logstash is not running on MAC, but the same thing is working on Windows getting below error 2023-11-09 01:12:57 runtime: failed to create new OS thread (have 2 already; errno=22) 2023-11-09 01:12:57 fatal … --- ## [Unexpected tCONSTANT in Ruby Script](https://discuss.elastic.co/t/unexpected-tconstant-in-ruby-script/346709)
**Author:** [@Kris\_Felscher](https://discuss.elastic.co/u/Kris_Felscher)\ **Replies:** 2\ **Last updated:** [November 8, 2023, 5:42pm UTC](https://discuss.elastic.co/t/unexpected-tconstant-in-ruby-script/346709 "2023-11-08T17:42:01Z")
We have a ton (200+) of applications that are all logging to the same index. Because of this, we are seeing a few field type collisions that cause messages to get bounced (to the tune of approximately 26 million bounced … --- ## [\[Logstash\] Use variables with ilm in Elasticsearch output](https://discuss.elastic.co/t/logstash-use-variables-with-ilm-in-elasticsearch-output/346697)
**Author:** [@quoctuan2311](https://discuss.elastic.co/u/quoctuan2311)\ **Replies:** 1\ **Last updated:** [November 8, 2023, 4:16pm UTC](https://discuss.elastic.co/t/logstash-use-variables-with-ilm-in-elasticsearch-output/346697 "2023-11-08T16:16:32Z")
Can you variables with ilm\_rollover\_alias and ilm\_policy. Current I use if else but if conditions increase with each log\_type by created. Logstash will be increase time start it. Pls support me with this case. elastic… --- ## [Logstash export not working correctly, only a part of data exported](https://discuss.elastic.co/t/logstash-export-not-working-correctly-only-a-part-of-data-exported/346657)
**Author:** [@andre22](https://discuss.elastic.co/u/andre22)\ **Replies:** 1\ **Last updated:** [November 7, 2023, 10:27pm UTC](https://discuss.elastic.co/t/logstash-export-not-working-correctly-only-a-part-of-data-exported/346657 "2023-11-07T22:27:33Z")
Hi, i want to export some data from old indexes and write them into a text file. When I restart logstash, it exports some data (a part of one day, the index has a complete month) and goes back to do nothing. I am using … --- ## [Logstash s3 output plugin and linux fs inode](https://discuss.elastic.co/t/logstash-s3-output-plugin-and-linux-fs-inode/346437)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 1\ **Last updated:** [November 7, 2023, 9:25pm UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-and-linux-fs-inode/346437 "2023-11-07T21:25:57Z")
Hello World! I'm using Logstash 7.17 and experiencing an issue with Logstash and S3 output plugin: $ logstash --version Using bundled JDK: /usr/share/logstash/jdk logstash 7.17.13 $ ./bin/logstash-plugin list logstash-… --- ## [How to solve \_geoip\_expired\_database](https://discuss.elastic.co/t/how-to-solve-geoip-expired-database/346583)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 6\ **Last updated:** [November 7, 2023, 5:53pm UTC](https://discuss.elastic.co/t/how-to-solve-geoip-expired-database/346583 "2023-11-07T17:53:15Z")
Hi, I've been experiencing an issue with the GeoIP filter here. So, at the beginning of my logstash deployment, the GeoIP filter was working well but recently I saw a tag on all my documents that said \_geoip\_expired\_dat… --- ## [Logstash log containing huge nested JSON-objects](https://discuss.elastic.co/t/logstash-log-containing-huge-nested-json-objects/346623)
**Author:** [@apt-get\_install\_skil](https://discuss.elastic.co/u/apt-get_install_skil)\ **Replies:** 0\ **Last updated:** [November 7, 2023, 2:08pm UTC](https://discuss.elastic.co/t/logstash-log-containing-huge-nested-json-objects/346623 "2023-11-07T14:08:46Z")
Hey guys, since we upgraded our stack components to version 8.10.2, Logstash's internal logging behaviour has changed. For example, after all pipelines were startet, Logstash logs the following message: { "level": "… --- ## [Creating JSON structure for sensor.community API](https://discuss.elastic.co/t/creating-json-structure-for-sensor-community-api/346470)
**Author:** [@CargoBikoMeter](https://discuss.elastic.co/u/CargoBikoMeter)\ **Replies:** 8\ **Last updated:** [November 7, 2023, 11:47am UTC](https://discuss.elastic.co/t/creating-json-structure-for-sensor-community-api/346470 "2023-11-07T11:47:54Z")
I will send data from my logstash pipeline to the sensor.community API. The API requires the following structure which works with my curl command: curl --location --request POST 'https://api.sensor.community/v1/push-sen… --- ## [Duplicates logs are available on running the query for same time](https://discuss.elastic.co/t/duplicates-logs-are-available-on-running-the-query-for-same-time/346586)
**Author:** [@Ayushi\_bhardwaj](https://discuss.elastic.co/u/Ayushi_bhardwaj)\ **Replies:** 1\ **Last updated:** [November 7, 2023, 5:32am UTC](https://discuss.elastic.co/t/duplicates-logs-are-available-on-running-the-query-for-same-time/346586 "2023-11-07T05:32:03Z")
Duplicates logs are available on running the query for same time (now-1m) We are running the query for last now-1m based upon our use case however we seeing duplicates getting generated in Output. Please help with the p… --- ## [SNMP with Logstash (Pipeline Error)](https://discuss.elastic.co/t/snmp-with-logstash-pipeline-error/346533)
**Author:** [@Funkster](https://discuss.elastic.co/u/Funkster)\ **Replies:** 2\ **Last updated:** [November 6, 2023, 4:03pm UTC](https://discuss.elastic.co/t/snmp-with-logstash-pipeline-error/346533 "2023-11-06T16:03:19Z")
Hello, I am trying to get SNMP-Loggin to work whithin ELK in Logstash and I get the following Error: root@vm-kibana:~# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash-snmp.conf --path.settings=/etc/lo… --- ## [Queries regarding logsatsh configuration file](https://discuss.elastic.co/t/queries-regarding-logsatsh-configuration-file/346491)
**Author:** [@Ajay\_Kumar.S](https://discuss.elastic.co/u/Ajay_Kumar.S)\ **Replies:** 0\ **Last updated:** [November 6, 2023, 6:16am UTC](https://discuss.elastic.co/t/queries-regarding-logsatsh-configuration-file/346491 "2023-11-06T06:16:45Z")
input { beats { port =\> "9006" } } filter { mutate { add\_field =\> { "beat\_version" =\> "%{\[beat\]\[version\]}" } } mutate { add\_field =\> { "log\_file" =\> "%{\[log\]\[file\]\[path\]}" } } mutate { add\_field =\> { "beat\_… --- ## [How can I get Gigabyte instead of number of records?](https://discuss.elastic.co/t/how-can-i-get-gigabyte-instead-of-number-of-records/346272)
**Author:** [@Indunil75](https://discuss.elastic.co/u/Indunil75)\ **Replies:** 1\ **Last updated:** [November 6, 2023, 4:34am UTC](https://discuss.elastic.co/t/how-can-i-get-gigabyte-instead-of-number-of-records/346272 "2023-11-06T04:34:02Z")
I have configured elasticsearch, kibana and logstash. fortigate firewall sends logs. While creating dashboard, It gives count of records. How can I get Gigabyte instead of count of records? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=51) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=53)