# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=42 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 43 --- ## [Removing empty fields from an event (2024 edition!)](https://discuss.elastic.co/t/removing-empty-fields-from-an-event-2024-edition/352571)
**Author:** [@Supermathie](https://discuss.elastic.co/u/Supermathie)\ **Replies:** 1\ **Last updated:** [February 5, 2024, 9:55pm UTC](https://discuss.elastic.co/t/removing-empty-fields-from-an-event-2024-edition/352571 "2024-02-05T21:55:15Z")
Continuing the discussion from Never ending story: how to check and remove empty fields, arrays etc: There's no simple way to (recursively) get all field names in an event, but one can do this: filter { ruby { … --- ## [Configuring Logstash to Accept Both SSL and Non-SSL Connections](https://discuss.elastic.co/t/configuring-logstash-to-accept-both-ssl-and-non-ssl-connections/352528)
**Author:** [@yago82](https://discuss.elastic.co/u/yago82)\ **Replies:** 1\ **Last updated:** [February 5, 2024, 4:28pm UTC](https://discuss.elastic.co/t/configuring-logstash-to-accept-both-ssl-and-non-ssl-connections/352528 "2024-02-05T16:28:57Z")
Hi, I'm currently working on setting up Logstash to accept both SSL and non-SSL connections from Beats. From my understanding, I would need to configure two separate Beats inputs on different ports. For example: input … --- ## [Logstash Proxy Credentials](https://discuss.elastic.co/t/logstash-proxy-credentials/352414)
**Author:** [@elk-user-0001](https://discuss.elastic.co/u/elk-user-0001)\ **Replies:** 2\ **Last updated:** [February 5, 2024, 9:22am UTC](https://discuss.elastic.co/t/logstash-proxy-credentials/352414 "2024-02-05T09:22:33Z")
Good afternoon colleagues! I have a logstash service in local and an elastic server in cloud. To reach it via curl I need to set --proxy https://proxy:8080 and --proxy-user 'user\\moreusername:password' . How can I do … --- ## [Logstash output email plugin failure](https://discuss.elastic.co/t/logstash-output-email-plugin-failure/351847)
**Author:** [@shailendra1](https://discuss.elastic.co/u/shailendra1)\ **Replies:** 9\ **Last updated:** [February 5, 2024, 3:38am UTC](https://discuss.elastic.co/t/logstash-output-email-plugin-failure/351847 "2024-02-05T03:38:51Z")
Hello Team, i am using elk 8.5.3 and in the logstash output plugin , its failing with SMTP syntax error as pasted below ERROR\]\[logstash.outputs.email \]\[main\]\[9ab43dc1824014b9cc39372f569aeded7925e535ec037015bed8af1439… --- ## [Unnest JSON](https://discuss.elastic.co/t/unnest-json/352447)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 6\ **Last updated:** [February 4, 2024, 2:28pm UTC](https://discuss.elastic.co/t/unnest-json/352447 "2024-02-04T14:28:21Z")
Good Day, I am trying to unnest a JSON log and I can't seem to get it to work. When I try to unnest I get the already unnested JSON showing up 3 times and I while I am able to unnest the JSON I can't write the actual f… --- ## [Logstash keystore create error](https://discuss.elastic.co/t/logstash-keystore-create-error/351854)
**Author:** [@jevonsnotes](https://discuss.elastic.co/u/jevonsnotes)\ **Replies:** 2\ **Last updated:** [February 4, 2024, 8:23am UTC](https://discuss.elastic.co/t/logstash-keystore-create-error/351854 "2024-02-04T08:23:32Z")
logstash version 7.11.1 when i try to create a keystore but got an error as \>uninitialized constant LogStash::Util::Password, how to resolve this? \[elk@centos70\_112 logstash-7.11.1\]$ ./bin/logstash-keystore --path.sett… --- ## [Logstash failed to differentiate log filtering based on data source IP address](https://discuss.elastic.co/t/logstash-failed-to-differentiate-log-filtering-based-on-data-source-ip-address/352471)
**Author:** [@pacoxpk](https://discuss.elastic.co/u/pacoxpk)\ **Replies:** 2\ **Last updated:** [February 4, 2024, 7:03am UTC](https://discuss.elastic.co/t/logstash-failed-to-differentiate-log-filtering-based-on-data-source-ip-address/352471 "2024-02-04T07:03:56Z")
Send logs from multiple systems through syslog to logstash, which can receive logs normally. In order to distinguish logs sent from different systems, it is necessary to distinguish them based on the IP address of the da… --- ## [Logstash for application.log + application.log.1.gz](https://discuss.elastic.co/t/logstash-for-application-log-application-log-1-gz/351577)
**Author:** [@RavaliJ](https://discuss.elastic.co/u/RavaliJ)\ **Replies:** 5\ **Last updated:** [February 2, 2024, 8:59pm UTC](https://discuss.elastic.co/t/logstash-for-application-log-application-log-1-gz/351577 "2024-02-02T20:59:08Z")
Hi, I have Rolling file appenders in my java code which creates log files like - application.log, application.log.1.gz,application.log.2.gz so on. I want to index all of these and make sure messages from log files are r… --- ## [Pagination on logstash http filter](https://discuss.elastic.co/t/pagination-on-logstash-http-filter/352320)
**Author:** [@Johnson\_will](https://discuss.elastic.co/u/Johnson_will)\ **Replies:** 7\ **Last updated:** [February 2, 2024, 6:40pm UTC](https://discuss.elastic.co/t/pagination-on-logstash-http-filter/352320 "2024-02-02T18:40:15Z")
I am generating a token by using http\_poller input plugin, using the generated token in http filter part and splitting the fields from results, Can anyone please suggest me on Pagination. As the api results are around 4… --- ## [Logstash stats API is showing wrong events values](https://discuss.elastic.co/t/logstash-stats-api-is-showing-wrong-events-values/352419)
**Author:** [@Freddy\_Laffita\_Almag](https://discuss.elastic.co/u/Freddy_Laffita_Almag)\ **Replies:** 0\ **Last updated:** [February 2, 2024, 3:17pm UTC](https://discuss.elastic.co/t/logstash-stats-api-is-showing-wrong-events-values/352419 "2024-02-02T15:17:53Z")
Hello everyone: I'm using logstash from a docker image, I'm consulting the stats API to show the logs processed by a pipeline to the user, but when I send 200 logs every 0.01 seconds the event information is showing wro… --- ## [Logstash netflow codec says "no template has been received" but it did receive one](https://discuss.elastic.co/t/logstash-netflow-codec-says-no-template-has-been-received-but-it-did-receive-one/352397)
**Author:** [@m0nkeyc0de](https://discuss.elastic.co/u/m0nkeyc0de)\ **Replies:** 1\ **Last updated:** [February 2, 2024, 1:30pm UTC](https://discuss.elastic.co/t/logstash-netflow-codec-says-no-template-has-been-received-but-it-did-receive-one/352397 "2024-02-02T13:30:06Z")
Hello, the Logstash codec logstash-codec-netflow says it can't decode a flowset because no template has been received. When looking in the packet capture, a template has been sent. \[2024-02-02T13:10:19,136\]\[WARN \]\[logst… --- ## [Logstash Fingerprint Plugin Target Unchanged](https://discuss.elastic.co/t/logstash-fingerprint-plugin-target-unchanged/352247)
**Author:** [@Cheese](https://discuss.elastic.co/u/Cheese)\ **Replies:** 3\ **Last updated:** [February 1, 2024, 9:44pm UTC](https://discuss.elastic.co/t/logstash-fingerprint-plugin-target-unchanged/352247 "2024-02-01T21:44:30Z")
HI all, Needing some help with using the Fingerprint plugin. I use the fingerprint plugin to generate a SHA-1 signature using a base string and a key. My logstash config file is like so: mutate { add\_field { "si… --- ## [Logstash creates page file size that reaches its limit causing other pages to not be created](https://discuss.elastic.co/t/logstash-creates-page-file-size-that-reaches-its-limit-causing-other-pages-to-not-be-created/352200)
**Author:** [@tcapp24](https://discuss.elastic.co/u/tcapp24)\ **Replies:** 11\ **Last updated:** [February 1, 2024, 8:46pm UTC](https://discuss.elastic.co/t/logstash-creates-page-file-size-that-reaches-its-limit-causing-other-pages-to-not-be-created/352200 "2024-02-01T20:46:29Z")
We 're currently running Logstash 7.17.8 using a docker container on a Linux VM. We have run into an issue where Logstash creates page.0 which quickly fills up and reaches its queue.page\_capacity of 256mb. Once it reach… --- ## [Logstash Tab / Space Delimiter](https://discuss.elastic.co/t/logstash-tab-space-delimiter/352231)
**Author:** [@dfir](https://discuss.elastic.co/u/dfir)\ **Replies:** 6\ **Last updated:** [February 1, 2024, 2:56pm UTC](https://discuss.elastic.co/t/logstash-tab-space-delimiter/352231 "2024-02-01T14:56:22Z")
Hi All. I am trying to ingest some IIS logs into Elastic via logtstash. They are CSVs but the separator is NOT a ,. How can I account for a space or a tab as the separator. I have multiple different kinds of files w… --- ## [I have created ubuntu server for receiving logs from fortigate30e using logstash](https://discuss.elastic.co/t/i-have-created-ubuntu-server-for-receiving-logs-from-fortigate30e-using-logstash/352278)
**Author:** [@Eepe123](https://discuss.elastic.co/u/Eepe123)\ **Replies:** 0\ **Last updated:** [February 1, 2024, 11:29am UTC](https://discuss.elastic.co/t/i-have-created-ubuntu-server-for-receiving-logs-from-fortigate30e-using-logstash/352278 "2024-02-01T11:29:43Z")
Its not working and everything points to a network error, fortigate30e cant ping our ubuntu server but ubuntu server can ping fortigate firewall. Does anyone know anything we could try to fix network issue or could this … --- ## [Logstash 8.11 reports error 403](https://discuss.elastic.co/t/logstash-8-11-reports-error-403/352224)
**Author:** [@andrew3](https://discuss.elastic.co/u/andrew3)\ **Replies:** 3\ **Last updated:** [January 31, 2024, 9:08pm UTC](https://discuss.elastic.co/t/logstash-8-11-reports-error-403/352224 "2024-01-31T21:08:42Z")
I am trying to connect logstash to elasticsearch. Both are on my local machine. I am using https. Logstash reports error 403. Viz: \` {:code=\>403, :url=\>"https://localhost:9200/\_bulk?filter\_path=errors,items.\*.error,i… --- ## [Logstash installation batch files v8.11.3 do not work](https://discuss.elastic.co/t/logstash-installation-batch-files-v8-11-3-do-not-work/351306)
**Author:** [@andrew3](https://discuss.elastic.co/u/andrew3)\ **Replies:** 3\ **Last updated:** [January 31, 2024, 8:41pm UTC](https://discuss.elastic.co/t/logstash-installation-batch-files-v8-11-3-do-not-work/351306 "2024-01-31T20:41:43Z")
JRUBY\_BIN: In v8.11.3 SETUP.BAT looks for it in a directory that does not exist in the Windows distribution, causing the "first stash" in the docs. to always fail. Anyone else run across this? Details: In SETUP.BAT lin… --- ## [Overwrite @timestamp field](https://discuss.elastic.co/t/overwrite-timestamp-field/352212)
**Author:** [@rmoss25](https://discuss.elastic.co/u/rmoss25)\ **Replies:** 1\ **Last updated:** [January 31, 2024, 5:03pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-field/352212 "2024-01-31T17:03:58Z")
Hi, I am trying to overwrite the @timestamp filed with the time from the log source but logstash fails to start when trying to run. I am guessing it has something to do with the "-04" in the time.....see below time fro… --- ## [Exclude version conflict, document already exists from logstash.log](https://discuss.elastic.co/t/exclude-version-conflict-document-already-exists-from-logstash-log/352186)
**Author:** [@lduvnjak](https://discuss.elastic.co/u/lduvnjak)\ **Replies:** 2\ **Last updated:** [January 31, 2024, 3:42pm UTC](https://discuss.elastic.co/t/exclude-version-conflict-document-already-exists-from-logstash-log/352186 "2024-01-31T15:42:08Z")
Hey Everyone, I'm having some issues with too much noise in my Logstash logs. What's happening is because I'm using the Threat Intel integrations it's spamming my logs with version conflict, document already exists WAR… --- ## [Logstash Gone Wrong After force shutdown](https://discuss.elastic.co/t/logstash-gone-wrong-after-force-shutdown/351867)
**Author:** [@akrog79](https://discuss.elastic.co/u/akrog79)\ **Replies:** 12\ **Last updated:** [January 31, 2024, 8:37am UTC](https://discuss.elastic.co/t/logstash-gone-wrong-after-force-shutdown/351867 "2024-01-31T08:37:15Z")
I have logstash installed as a service on a machine with Logstash+Kibana+Elastic. My logstash was updated, so its seems that doesn't shutdown properly and now show an error with a pipeline: Jan 26 10:19:56 esearch logs… --- ## [Not able to run ALTER query through logstash JDBC plugin](https://discuss.elastic.co/t/not-able-to-run-alter-query-through-logstash-jdbc-plugin/352072)
**Author:** [@rpraveenr](https://discuss.elastic.co/u/rpraveenr)\ **Replies:** 4\ **Last updated:** [January 31, 2024, 6:27am UTC](https://discuss.elastic.co/t/not-able-to-run-alter-query-through-logstash-jdbc-plugin/352072 "2024-01-31T06:27:24Z")
I am not able to run ALTER queries on my Oracle database through Logstash. Below is the snippet from the config file: input { jdbc { jdbc\_validate\_connection =\> true jdbc\_driver\_library =\> "ojdbc7.ja… --- ## ["\_dateparsefailure" When trying to overwrite @timestamp field](https://discuss.elastic.co/t/dateparsefailure-when-trying-to-overwrite-timestamp-field/352129)
**Author:** [@rmoss25](https://discuss.elastic.co/u/rmoss25)\ **Replies:** 7\ **Last updated:** [January 30, 2024, 10:10pm UTC](https://discuss.elastic.co/t/dateparsefailure-when-trying-to-overwrite-timestamp-field/352129 "2024-01-30T22:10:18Z")
Hi, I am trying to solve this issue but don't seem to be having much luck. My log is as follows: 01-JAN-24 00:00:50|1.1.1.1|1|CN=test\_user,OU=test Users,OU=test,OU=Business,DC=test,DC=corp,DC=abc,DC=ca|Z/dtEse7dwP2VEV… --- ## [Logstash Filter If loop](https://discuss.elastic.co/t/logstash-filter-if-loop/352119)
**Author:** [@adityak248](https://discuss.elastic.co/u/adityak248)\ **Replies:** 1\ **Last updated:** [January 30, 2024, 7:50pm UTC](https://discuss.elastic.co/t/logstash-filter-if-loop/352119 "2024-01-30T19:50:53Z")
Which one is correct: filter { if \[kubernetes\_labels\]\[app\] == "app-name" and \[kubernetes\_container\_name\] == "nginx" { grok { match =\> {"message" =\> 'XXX'}}} else if \[kubernetes\_labels\]\[app\] =… --- ## [Multiple kafka topics using logstash and make index inside output with \`.conf\` file](https://discuss.elastic.co/t/multiple-kafka-topics-using-logstash-and-make-index-inside-output-with-conf-file/352112)
**Author:** [@Tony\_Haf.H](https://discuss.elastic.co/u/Tony_Haf.H)\ **Replies:** 2\ **Last updated:** [January 30, 2024, 6:58pm UTC](https://discuss.elastic.co/t/multiple-kafka-topics-using-logstash-and-make-index-inside-output-with-conf-file/352112 "2024-01-30T18:58:09Z")
I am using Logstash 8.11, and I have problem like subject title I have consulted a few solutions in other topics, and I still have not solved the problem. Example old topic: How to pull data data from 2 kafka topics us… --- ## [How to define a proper grok pattern for php error logs](https://discuss.elastic.co/t/how-to-define-a-proper-grok-pattern-for-php-error-logs/351978)
**Author:** [@Ritikapawar](https://discuss.elastic.co/u/Ritikapawar)\ **Replies:** 8\ **Last updated:** [January 30, 2024, 11:35am UTC](https://discuss.elastic.co/t/how-to-define-a-proper-grok-pattern-for-php-error-logs/351978 "2024-01-30T11:35:14Z")
Hello, I'm creating a connection for elasticsearch and want to upload PHP error logs by creating a script using logstash PHP error logs format-- \[16-Jan-2024 13:39:08 Asia/Calcutta\] PHP Warning: Module 'mcrypt' alrea… --- ## [Logstash XML file not parsing](https://discuss.elastic.co/t/logstash-xml-file-not-parsing/352048)
**Author:** [@Shawn\_Lim](https://discuss.elastic.co/u/Shawn_Lim)\ **Replies:** 2\ **Last updated:** [January 30, 2024, 9:31am UTC](https://discuss.elastic.co/t/logstash-xml-file-not-parsing/352048 "2024-01-30T09:31:03Z")
Hi guys, I'm very new to Elasticsearch stack, need some help over here... Currently I'm trying to parse XML file, output to Elasticsearch and use it on Grafana for visualization. Now I facing a problem is my XML files … --- ## [Logstash cpu usage is very high](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high/351619)
**Author:** [@Manoj\_Sangwan](https://discuss.elastic.co/u/Manoj_Sangwan)\ **Replies:** 3\ **Last updated:** [January 30, 2024, 4:56am UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high/351619 "2024-01-30T04:56:32Z")
Logstash CPU usage is up to 90% and assuming this would increase with data growth. From the Application side seems everything fine. How to solve this issue in my project? --- ## [Trying to decrypt data but it is not working as expected](https://discuss.elastic.co/t/trying-to-decrypt-data-but-it-is-not-working-as-expected/351977)
**Author:** [@Pallavibhushan](https://discuss.elastic.co/u/Pallavibhushan)\ **Replies:** 4\ **Last updated:** [January 29, 2024, 7:22pm UTC](https://discuss.elastic.co/t/trying-to-decrypt-data-but-it-is-not-working-as-expected/351977 "2024-01-29T19:22:47Z")
Below is my config input { file { path =\> "C:/logstash-7.16.2/data/input/test\*.json" start\_position =\> "beginning" sincedb\_path =\> "null" } } filter { json { source =\> "message" target =\> "document" } mutate… --- ## [How to add new config file to logstash which is created by docker compose?](https://discuss.elastic.co/t/how-to-add-new-config-file-to-logstash-which-is-created-by-docker-compose/351655)
**Author:** [@shrm](https://discuss.elastic.co/u/shrm)\ **Replies:** 3\ **Last updated:** [January 29, 2024, 1:43pm UTC](https://discuss.elastic.co/t/how-to-add-new-config-file-to-logstash-which-is-created-by-docker-compose/351655 "2024-01-29T13:43:48Z")
How to add a new config file to logstash which is created by docker-compose? I created my service with docker-compose and log stash is connected to kibana and elasticsearch. Now I created a new config file for logstash… --- ## [Delete dictionary from array if one key is empty](https://discuss.elastic.co/t/delete-dictionary-from-array-if-one-key-is-empty/351863)
**Author:** [@ITIC](https://discuss.elastic.co/u/ITIC)\ **Replies:** 2\ **Last updated:** [January 29, 2024, 1:08pm UTC](https://discuss.elastic.co/t/delete-dictionary-from-array-if-one-key-is-empty/351863 "2024-01-29T13:08:00Z")
Hi! It's been a while since I last wrestled with logstash, and I can feel the rust! I'm trying to delete an array element with delete\_if, and I have trouble with it. The element itself is a dictionary, and the conditi… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=41) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=43)