# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=37 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 38 --- ## [Sincedb minor device number changes in Linux VM | alternative sincedb configurations options?](https://discuss.elastic.co/t/sincedb-minor-device-number-changes-in-linux-vm-alternative-sincedb-configurations-options/355506)
**Author:** [@flomickl](https://discuss.elastic.co/u/flomickl)\ **Replies:** 3\ **Last updated:** [March 15, 2024, 4:56pm UTC](https://discuss.elastic.co/t/sincedb-minor-device-number-changes-in-linux-vm-alternative-sincedb-configurations-options/355506 "2024-03-15T16:56:48Z")
Hi, I have a Logstash -\> Elasticsearch set p. Everything is working as expected, but I have a problem with the sincedb or better with my Linux VM underneath. If I reboot the system, the device number of the partition i… --- ## [No logs for current timestamp shown in Disover tab](https://discuss.elastic.co/t/no-logs-for-current-timestamp-shown-in-disover-tab/355018)
**Author:** [@Ravi\_Pattar](https://discuss.elastic.co/u/Ravi_Pattar)\ **Replies:** 23\ **Last updated:** [March 15, 2024, 3:06pm UTC](https://discuss.elastic.co/t/no-logs-for-current-timestamp-shown-in-disover-tab/355018 "2024-03-15T15:06:04Z")
Hello, Most recently the security feature (xpack.security) was enabled and the changes were only in the elasticsearch.yml and kibana.yml. Because of this change it was possible to have the authentication while accessing… --- ## [Order of processing: Input from Kafka, output to file](https://discuss.elastic.co/t/order-of-processing-input-from-kafka-output-to-file/355442)
**Author:** [@ankh](https://discuss.elastic.co/u/ankh)\ **Replies:** 1\ **Last updated:** [March 15, 2024, 1:07am UTC](https://discuss.elastic.co/t/order-of-processing-input-from-kafka-output-to-file/355442 "2024-03-15T01:07:51Z")
I am debugging a pipeline that has input from Kafka and output to Elasticsearch. As part of the debugging I'm writing to a file using the file output plugin. The Kafka topic is a single partition. This simplified confi… --- ## [Logstash config for windows events](https://discuss.elastic.co/t/logstash-config-for-windows-events/355421)
**Author:** [@deanj](https://discuss.elastic.co/u/deanj)\ **Replies:** 0\ **Last updated:** [March 14, 2024, 3:36pm UTC](https://discuss.elastic.co/t/logstash-config-for-windows-events/355421 "2024-03-14T15:36:22Z")
Hi All, I'm very new to logstash, so i'm new to manually configuring its config for event filtering. What i am planning: 1: Collect windows events. Sent from a remote location by windows event forwarding service on … --- ## [Key-value split failing on comma in field](https://discuss.elastic.co/t/key-value-split-failing-on-comma-in-field/355408)
**Author:** [@e.vedelaar](https://discuss.elastic.co/u/e.vedelaar)\ **Replies:** 1\ **Last updated:** [March 14, 2024, 2:36pm UTC](https://discuss.elastic.co/t/key-value-split-failing-on-comma-in-field/355408 "2024-03-14T14:36:12Z")
When ingesting fortigate logs i come across the following error message: field \[syslog5424\_sd\] does not contain value\_split \[=\] this occurs because i do KV splits on ",". While this works with most logs, there are a fe… --- ## [Multiple questions about http\_poller input plugin](https://discuss.elastic.co/t/multiple-questions-about-http-poller-input-plugin/355329)
**Author:** [@nilsen](https://discuss.elastic.co/u/nilsen)\ **Replies:** 3\ **Last updated:** [March 14, 2024, 10:24am UTC](https://discuss.elastic.co/t/multiple-questions-about-http-poller-input-plugin/355329 "2024-03-14T10:24:06Z")
Hello, I decided to merge all of my questions into one thread, instead of making multiple threads, considering they are all http\_poller related. Goal: I'm trying to get Logstash to do an http request to our local Active… --- ## [Logstash HTTP Filter sending once per node](https://discuss.elastic.co/t/logstash-http-filter-sending-once-per-node/355360)
**Author:** [@JoelAU](https://discuss.elastic.co/u/JoelAU)\ **Replies:** 2\ **Last updated:** [March 14, 2024, 5:25am UTC](https://discuss.elastic.co/t/logstash-http-filter-sending-once-per-node/355360 "2024-03-14T05:25:48Z")
I currently have a Logstash pipeline with a JDBC input, a HTTP filter (to send externally), and an output to Elasticsearch. There is also two Logstash nodes running. Everything is working as expected, except that a HTTP… --- ## [KV pairs automatically parsed by logstash?](https://discuss.elastic.co/t/kv-pairs-automatically-parsed-by-logstash/355331)
**Author:** [@Joe\_Martin](https://discuss.elastic.co/u/Joe_Martin)\ **Replies:** 3\ **Last updated:** [March 13, 2024, 6:17pm UTC](https://discuss.elastic.co/t/kv-pairs-automatically-parsed-by-logstash/355331 "2024-03-13T18:17:56Z")
I have a device sending syslogs in standard kv pair format; with a comma (,) separating fields and equal (=) separating key from value. sample: key1="value1",key2="value2",key3="value3",... In order to ensure ECS comp… --- ## [How to add a second logstash instance for clustering](https://discuss.elastic.co/t/how-to-add-a-second-logstash-instance-for-clustering/355334)
**Author:** [@Patrick.kirk](https://discuss.elastic.co/u/Patrick.kirk)\ **Replies:** 0\ **Last updated:** [March 13, 2024, 5:37pm UTC](https://discuss.elastic.co/t/how-to-add-a-second-logstash-instance-for-clustering/355334 "2024-03-13T17:37:37Z")
I'm trying to add a second logstash instance and cluster logstash to help with in coming windows logs. Within the outputs section of kibana, I need to add a second cert and key. How do I do this? Thanks in advance --- ## [How to install netflow on linux and use it for flow collector](https://discuss.elastic.co/t/how-to-install-netflow-on-linux-and-use-it-for-flow-collector/355332)
**Author:** [@jatindersharma1](https://discuss.elastic.co/u/jatindersharma1)\ **Replies:** 0\ **Last updated:** [March 13, 2024, 5:08pm UTC](https://discuss.elastic.co/t/how-to-install-netflow-on-linux-and-use-it-for-flow-collector/355332 "2024-03-13T17:08:21Z")
HI We are currently in the process of implementing a NetFlow-based flow collector for our network infrastructure, and we are seeking assistance with the installation procedure and configuration details for deploying the… --- ## [The tags field in the message conflicts with the built-in tags field in Logstash](https://discuss.elastic.co/t/the-tags-field-in-the-message-conflicts-with-the-built-in-tags-field-in-logstash/355290)
**Author:** [@uniquecats](https://discuss.elastic.co/u/uniquecats)\ **Replies:** 0\ **Last updated:** [March 13, 2024, 7:48am UTC](https://discuss.elastic.co/t/the-tags-field-in-the-message-conflicts-with-the-built-in-tags-field-in-logstash/355290 "2024-03-13T07:48:34Z")
The version: logstash:8.12.2 I have the following data structure in kafka. { "contents": { "content": "2024-03-13 12:17:18,614 \[aecd9c48e39fb9fe,aecd9c48e39fb9fe,,false\] INFO com.td.mc.gate.util...." }, "ta… --- ## [Logstash variable default value crashes logstash](https://discuss.elastic.co/t/logstash-variable-default-value-crashes-logstash/355265)
**Author:** [@learningelastic](https://discuss.elastic.co/u/learningelastic)\ **Replies:** 0\ **Last updated:** [March 12, 2024, 11:03pm UTC](https://discuss.elastic.co/t/logstash-variable-default-value-crashes-logstash/355265 "2024-03-12T23:03:52Z")
I have the following files which work perfectly and does exactly what I expect. My .env file: COMPOSE\_PROJECT\_NAME=wp LOGSTASH\_DEBUG\_SCHEDULE="\* \* \* \* \*" My docker-compose.yml: version: "3.8" networks: default: … --- ## [Array search from Logstash elasticsearch plugin](https://discuss.elastic.co/t/array-search-from-logstash-elasticsearch-plugin/355258)
**Author:** [@RRGTHWAR1](https://discuss.elastic.co/u/RRGTHWAR1)\ **Replies:** 0\ **Last updated:** [March 12, 2024, 7:27pm UTC](https://discuss.elastic.co/t/array-search-from-logstash-elasticsearch-plugin/355258 "2024-03-12T19:27:42Z")
I'm bringing records from a SQL Server database into Elasticsearch using Logstash, and I need to convert a numerical field that could apply several times to each SQL Server record into one array field in the Elasticsearc… --- ## [How to fetch first value from Json Array](https://discuss.elastic.co/t/how-to-fetch-first-value-from-json-array/355214)
**Author:** [@upreddy](https://discuss.elastic.co/u/upreddy)\ **Replies:** 1\ **Last updated:** [March 12, 2024, 6:38pm UTC](https://discuss.elastic.co/t/how-to-fetch-first-value-from-json-array/355214 "2024-03-12T18:38:14Z")
Hi Team, I am sharing logline below. "independentFailureReport": { "failureReportEntries": \[ { "errorCode": abc, "additionalInfo": { "length": xyz, "result": "XXXXXXXXXXXXXXXXXXX" }, "execScheduleInfo": { "ena… --- ## [Logstash parsing data incorrectly](https://discuss.elastic.co/t/logstash-parsing-data-incorrectly/355221)
**Author:** [@gyrao\_72](https://discuss.elastic.co/u/gyrao_72)\ **Replies:** 2\ **Last updated:** [March 12, 2024, 11:23am UTC](https://discuss.elastic.co/t/logstash-parsing-data-incorrectly/355221 "2024-03-12T11:23:41Z")
0 I am using Logstash version 6.5.4 Data is read and parsed using Filebeat -\> Logstash -\> Elasticsearch But as I can see on Kibana for a few cases, I am getting duplicate field data i.e data is converted into an array … --- ## [JDBC streaming filter with SQL IN () condition](https://discuss.elastic.co/t/jdbc-streaming-filter-with-sql-in-condition/354429)
**Author:** [@Joss](https://discuss.elastic.co/u/Joss)\ **Replies:** 1\ **Last updated:** [March 12, 2024, 11:19am UTC](https://discuss.elastic.co/t/jdbc-streaming-filter-with-sql-in-condition/354429 "2024-03-12T11:19:03Z")
Greetings community, I stuck with development of data processing pipeline where is one step is fetch data from DB by array of ids. Prerequisites: logstash 7.17 logstash-filter-jdbc\_streaming (OOTB) jdbc client for ma… --- ## [JDBC connection issue after upgrade to v8](https://discuss.elastic.co/t/jdbc-connection-issue-after-upgrade-to-v8/355197)
**Author:** [@snakeapit](https://discuss.elastic.co/u/snakeapit)\ **Replies:** 0\ **Last updated:** [March 12, 2024, 12:14am UTC](https://discuss.elastic.co/t/jdbc-connection-issue-after-upgrade-to-v8/355197 "2024-03-12T00:14:21Z")
Our elastics-logstash solution unable to index due to JDBC connection timeout. From the logs, I could see a "Caused by: java.net.SocketTimeoutException: Connect timed out" error in there Confirm there is no issue with … --- ## [Failed to parse field \[host\] of type \[text\] after upgrade from 7.17 to 8.12](https://discuss.elastic.co/t/failed-to-parse-field-host-of-type-text-after-upgrade-from-7-17-to-8-12/353734)
**Author:** [@snakeapit](https://discuss.elastic.co/u/snakeapit)\ **Replies:** 2\ **Last updated:** [March 11, 2024, 10:37pm UTC](https://discuss.elastic.co/t/failed-to-parse-field-host-of-type-text-after-upgrade-from-7-17-to-8-12/353734 "2024-03-11T22:37:14Z")
Seeing this error below in logstash after upgrade from 7.17 to 8.12, data unable to index to Elasticsearch. I'm new in Elastics, hopefully someone can assist here. Suspect the change in version 8.12 causing different ty… --- ## [Logstash http input plugin acknowledge, when message is persisted](https://discuss.elastic.co/t/logstash-http-input-plugin-acknowledge-when-message-is-persisted/355167)
**Author:** [@Casper\_Thrane](https://discuss.elastic.co/u/Casper_Thrane)\ **Replies:** 3\ **Last updated:** [March 11, 2024, 9:50pm UTC](https://discuss.elastic.co/t/logstash-http-input-plugin-acknowledge-when-message-is-persisted/355167 "2024-03-11T21:50:18Z")
Hi Is it correct understood, to make logstash return 200 after message is persisted, a persistent queue is needed. The input plugin will return it's status code, before the output pipeline is done. Br Casper --- ## [Load balancing using logstash](https://discuss.elastic.co/t/load-balancing-using-logstash/353439)
**Author:** [@kriti\_dabas](https://discuss.elastic.co/u/kriti_dabas)\ **Replies:** 2\ **Last updated:** [March 11, 2024, 6:16am UTC](https://discuss.elastic.co/t/load-balancing-using-logstash/353439 "2024-03-11T06:16:01Z")
I want to use two logstash so that if one goes down I can use the other one. Is there a chance of duplicacy if i am using two logstash with same pipelines? I will configure elasticsearch in the output of logstash. --- ## [Logstash dockerfile issue while syncing elasticsearch with mysql using python](https://discuss.elastic.co/t/logstash-dockerfile-issue-while-syncing-elasticsearch-with-mysql-using-python/355083)
**Author:** [@abhinavtyagi](https://discuss.elastic.co/u/abhinavtyagi)\ **Replies:** 9\ **Last updated:** [March 10, 2024, 4:30pm UTC](https://discuss.elastic.co/t/logstash-dockerfile-issue-while-syncing-elasticsearch-with-mysql-using-python/355083 "2024-03-10T16:30:39Z")
Below is the logstash dockerfile taken from GitHub - r13i/sync-elasticsearch-mysql: Using Logstash to synchronize an Elasticsearch index with MySQL data but note that I am using Python and JDBC works with Java. How can … --- ## [Encountering \_geoip\_lookup\_failure within the IP address range 104.16.16.0/20](https://discuss.elastic.co/t/encountering-geoip-lookup-failure-within-the-ip-address-range-104-16-16-0-20/354958)
**Author:** [@yogeshk04](https://discuss.elastic.co/u/yogeshk04)\ **Replies:** 12\ **Last updated:** [March 10, 2024, 9:55am UTC](https://discuss.elastic.co/t/encountering-geoip-lookup-failure-within-the-ip-address-range-104-16-16-0-20/354958 "2024-03-10T09:55:55Z")
Could someone clarify why I consistently encounter the tags in my logstash pipeline "\_geoip\_lookup\_failure" issue with the IP address range 104.16.16.0/20, despite it working fine for other public IP addresses? --- ## [How to fix to keep run the logstash till SQL server query complete](https://discuss.elastic.co/t/how-to-fix-to-keep-run-the-logstash-till-sql-server-query-complete/355085)
**Author:** [@J\_S](https://discuss.elastic.co/u/J_S)\ **Replies:** 0\ **Last updated:** [March 9, 2024, 6:06pm UTC](https://discuss.elastic.co/t/how-to-fix-to-keep-run-the-logstash-till-sql-server-query-complete/355085 "2024-03-09T18:06:22Z")
I have a JDBC Logstash config file, which works perfectly but after few mins, the command prompt window closed automatically and there are still some data left over in the SQL database need to be pushed to Elastic Index. … --- ## [Memcached in logstash](https://discuss.elastic.co/t/memcached-in-logstash/355047)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 10\ **Last updated:** [March 8, 2024, 11:11pm UTC](https://discuss.elastic.co/t/memcached-in-logstash/355047 "2024-03-08T23:11:08Z")
I have huge file that I am using in logstash with translate filter. But I think it is causing some issue as it misses some match for some event. By looking around I discover that memcached may be answer to that. but I … --- ## [Few logs are missing in logstash from filebeat](https://discuss.elastic.co/t/few-logs-are-missing-in-logstash-from-filebeat/355046)
**Author:** [@Mrudula](https://discuss.elastic.co/u/Mrudula)\ **Replies:** 0\ **Last updated:** [March 8, 2024, 4:34pm UTC](https://discuss.elastic.co/t/few-logs-are-missing-in-logstash-from-filebeat/355046 "2024-03-08T16:34:47Z")
Hi Team, We are seeing very few logs are missing from pods of OCP to logstash, We are trying to pull the logs by using the filebeat agent. We are seeing very few (1%) logs are missing from some pods and we are seeing ot… --- ## [Change timestamp in logstash](https://discuss.elastic.co/t/change-timestamp-in-logstash/354909)
**Author:** [@sudharsanam132](https://discuss.elastic.co/u/sudharsanam132)\ **Replies:** 10\ **Last updated:** [March 8, 2024, 9:04am UTC](https://discuss.elastic.co/t/change-timestamp-in-logstash/354909 "2024-03-08T09:04:53Z")
Im having logs like below from logstash 8.9.1 2024-03-06T23:43:37.317829530Z 192.168.1.132 \<14\> 2024-03-06T23:43:37Z forwarder.tmes.trendmicro.com tmes\[1\]: CEF:0|Trend Micro|TMES|1.0.0.0|100101|DETECTION|6|rt=2024-03-06… --- ## [A plugin had an unrecoverable error. Will restart this plugin](https://discuss.elastic.co/t/a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/354973)
**Author:** [@Guilherme\_Carvalho](https://discuss.elastic.co/u/Guilherme_Carvalho)\ **Replies:** 1\ **Last updated:** [March 8, 2024, 3:42am UTC](https://discuss.elastic.co/t/a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/354973 "2024-03-08T03:42:14Z")
I'm having some problems with my logsatsh pipeline, i get an error saying that the plugin had an unrecoverable error and it will restart, bellow is the log, do you guys have any ideia about this problem? \[2024-03-06T09:… --- ## [Logstash failed to execute action](https://discuss.elastic.co/t/logstash-failed-to-execute-action/354852)
**Author:** [@shrm](https://discuss.elastic.co/u/shrm)\ **Replies:** 5\ **Last updated:** [March 7, 2024, 1:16pm UTC](https://discuss.elastic.co/t/logstash-failed-to-execute-action/354852 "2024-03-07T13:16:24Z")
I have a docker-compose controlled elk, with the following configs. logstash/logstash.conf : input { jdbc { jdbc\_driver\_library =\> "/home/user/extvol/elk/mysql-connector-j-8.2.0.jar" jdbc\_driver\_class =\> "com… --- ## [ELK ESXi Dashboard](https://discuss.elastic.co/t/elk-esxi-dashboard/354942)
**Author:** [@Leo2](https://discuss.elastic.co/u/Leo2)\ **Replies:** 0\ **Last updated:** [March 7, 2024, 10:47am UTC](https://discuss.elastic.co/t/elk-esxi-dashboard/354942 "2024-03-07T10:47:58Z")
Hello, I use Elastic Search, Kibana and Logstash I would like to create a dashboard that includes a pannel with: Successful connections on an Esxi with root from an IP that is not mine . Failed connections on an E… --- ## [CSV files slow input/fitering comparing to python script](https://discuss.elastic.co/t/csv-files-slow-input-fitering-comparing-to-python-script/354886)
**Author:** [@Benjamin\_Lin](https://discuss.elastic.co/u/Benjamin_Lin)\ **Replies:** 0\ **Last updated:** [March 6, 2024, 10:06pm UTC](https://discuss.elastic.co/t/csv-files-slow-input-fitering-comparing-to-python-script/354886 "2024-03-06T22:06:03Z")
Hi I have a 4 columns csv data to be proceed by Logstash and aggregating before sending to Elasticsearch. However, from my testing, I tune the JVM heap size up along with batch size to 50k, batch delay 1m. It stil took 1… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=36) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=38)