# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=87 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 88 --- ## [Logstash Config File not running getting error: contains non-ascii characters but are not UTF-8 encoded](https://discuss.elastic.co/t/logstash-config-file-not-running-getting-error-contains-non-ascii-characters-but-are-not-utf-8-encoded/327080)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 2\ **Last updated:** [March 6, 2023, 2:24pm UTC](https://discuss.elastic.co/t/logstash-config-file-not-running-getting-error-contains-non-ascii-characters-but-are-not-utf-8-encoded/327080 "2023-03-06T14:24:27Z")
Hello All, I'm getting below error while running the logstash config,Unable to understand how it can be resolved.Eearlier it worked by now giving error. input { jdbc { jdbc\_connection\_string =\> "jdbc:oracle:thin… --- ## [Make a grok pattern for a field that might be missing](https://discuss.elastic.co/t/make-a-grok-pattern-for-a-field-that-might-be-missing/327014)
**Author:** [@ira-zaya](https://discuss.elastic.co/u/ira-zaya)\ **Replies:** 3\ **Last updated:** [March 6, 2023, 11:12am UTC](https://discuss.elastic.co/t/make-a-grok-pattern-for-a-field-that-might-be-missing/327014 "2023-03-06T11:12:38Z")
Hi! There are logs in the following format: 2023-03-05 17:07:01.586+0000 \[L: WARN\] \[O: A.b.c.d.e.FGScript\] \[I: \] \[U: email@example.com\] \[S: \] \[P: \] \[T: ABCProcessor-23 \] @@@ aboba=5 beboba=1 ceboba=4 So I have a correc… --- ## [Logstash: HTTP Poller Formatting Issue](https://discuss.elastic.co/t/logstash-http-poller-formatting-issue/326844)
**Author:** [@alaine](https://discuss.elastic.co/u/alaine)\ **Replies:** 4\ **Last updated:** [March 5, 2023, 10:53pm UTC](https://discuss.elastic.co/t/logstash-http-poller-formatting-issue/326844 "2023-03-05T22:53:08Z")
I am trying to use the HTTP poller to automate a curl command that I am able to run successfully in my environment. I am trying to run a query, put the results through a pipeline and then send the output to elasticsearch… --- ## [TLS Error in Logstash](https://discuss.elastic.co/t/tls-error-in-logstash/326962)
**Author:** [@sta02](https://discuss.elastic.co/u/sta02)\ **Replies:** 0\ **Last updated:** [March 3, 2023, 5:25pm UTC](https://discuss.elastic.co/t/tls-error-in-logstash/326962 "2023-03-03T17:25:53Z")
Hello, We are trying to send logs from an application hosted in kubernetes cluster to logstash via fluentd. The logs are sent in syslog over TCP on an encrypted channel with TLS configuration. At the logstash end we ar… --- ## [Date/time field formatting from csv input](https://discuss.elastic.co/t/date-time-field-formatting-from-csv-input/326984)
**Author:** [@karlkras](https://discuss.elastic.co/u/karlkras)\ **Replies:** 2\ **Last updated:** [March 4, 2023, 9:19pm UTC](https://discuss.elastic.co/t/date-time-field-formatting-from-csv-input/326984 "2023-03-04T21:19:27Z")
Please excuse the ignorance of my question, I'm still trying get my arms around working with elk, not my forte. I'm generating a csv for a report that contains a few columns that refer to date/time stamps. During gener… --- ## [Log4j configuration to not display particular error](https://discuss.elastic.co/t/log4j-configuration-to-not-display-particular-error/326964)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 2\ **Last updated:** [March 3, 2023, 6:14pm UTC](https://discuss.elastic.co/t/log4j-configuration-to-not-display-particular-error/326964 "2023-03-03T18:14:04Z")
Hi Im using kafka input, when I get errors it shows me to much data, how to silence this error: \[ERROR\]\[logstash.javapipeline \] in log4j2.properties? thanks --- ## [Grok - Extracting words between two phrases that remain constant](https://discuss.elastic.co/t/grok-extracting-words-between-two-phrases-that-remain-constant/326901)
**Author:** [@demonsquatch](https://discuss.elastic.co/u/demonsquatch)\ **Replies:** 2\ **Last updated:** [March 3, 2023, 5:47pm UTC](https://discuss.elastic.co/t/grok-extracting-words-between-two-phrases-that-remain-constant/326901 "2023-03-03T17:47:46Z")
Hi All, I'm currently trying to extract a VM name from vSphere logs and am having some issues as the VM names can be of variable length and contain an array of characters. So far the only delimiting factor for separatin… --- ## [Logstash - A plugin had an unrecoverable error. Will restart this plugin. java.util.concurrent.ScheduledThreadPoolExecutor](https://discuss.elastic.co/t/logstash-a-plugin-had-an-unrecoverable-error-will-restart-this-plugin-java-util-concurrent-scheduledthreadpoolexecutor/326007)
**Author:** [@Grant\_Hope](https://discuss.elastic.co/u/Grant_Hope)\ **Replies:** 10\ **Last updated:** [March 3, 2023, 3:01pm UTC](https://discuss.elastic.co/t/logstash-a-plugin-had-an-unrecoverable-error-will-restart-this-plugin-java-util-concurrent-scheduledthreadpoolexecutor/326007 "2023-03-03T15:01:03Z")
I'm running into a problem with java.util.concurrent.ScheduledThreadPoolExecutor. Setup details: logstash 8.5.1 Logstash was installed via RPM Linux server1 4.18.0-372.26.1.el8\_6.x86\_64 #1 SMP Sat Aug 27 02:44:20 EDT… --- ## [Use logstash to collect NextCloud Audit logs](https://discuss.elastic.co/t/use-logstash-to-collect-nextcloud-audit-logs/326928)
**Author:** [@GEHsu](https://discuss.elastic.co/u/GEHsu)\ **Replies:** 0\ **Last updated:** [March 3, 2023, 10:44am UTC](https://discuss.elastic.co/t/use-logstash-to-collect-nextcloud-audit-logs/326928 "2023-03-03T10:44:42Z")
Use logstash to collect NextCloud Audit logs, Chinese word will become a word starting with % or \\u, how to make it display normally I have added filter {urldecode {all\_fields =\> true}}, Chinese starting with % are disp… --- ## [Logstash close\_older in tail mode](https://discuss.elastic.co/t/logstash-close-older-in-tail-mode/326896)
**Author:** [@akassabi](https://discuss.elastic.co/u/akassabi)\ **Replies:** 7\ **Last updated:** [March 3, 2023, 4:26am UTC](https://discuss.elastic.co/t/logstash-close-older-in-tail-mode/326896 "2023-03-03T04:26:01Z")
Suppose we have an input file input.dat and we are reading it in Logstash in tail mode. We set close\_older to 10 minutes. My questions are: Is the close\_older setting deprecated? The docs say it is "retained for back… --- ## [Logstash connection error](https://discuss.elastic.co/t/logstash-connection-error/326133)
**Author:** [@peinmercado](https://discuss.elastic.co/u/peinmercado)\ **Replies:** 1\ **Last updated:** [March 3, 2023, 1:27am UTC](https://discuss.elastic.co/t/logstash-connection-error/326133 "2023-03-03T01:27:32Z")
Hi all, I'm trying to setup log stash for my elasticsearch master region to backup region for our BCP, I will be using the in and out information of logstash however, I got an error \[2023-02-22T07:14:21,226\]\[ERROR\]\[l… --- ## [Shift value in CSV condition](https://discuss.elastic.co/t/shift-value-in-csv-condition/326801)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 6\ **Last updated:** [March 2, 2023, 5:05pm UTC](https://discuss.elastic.co/t/shift-value-in-csv-condition/326801 "2023-03-02T17:05:29Z")
Hi How I can shift some fields with value in "if" condition for example: CLLI, SWREL for the output in one event? expected output: { "NDCFLXDA" =\> "0", "@version" =\> "1", "NDCFLXDC" =\> "0", "STATUS" =\> "K", "NM… --- ## [Pipeline on Logstash](https://discuss.elastic.co/t/pipeline-on-logstash/326249)
**Author:** [@psanggabuana](https://discuss.elastic.co/u/psanggabuana)\ **Replies:** 2\ **Last updated:** [March 2, 2023, 3:54pm UTC](https://discuss.elastic.co/t/pipeline-on-logstash/326249 "2023-03-02T15:54:33Z")
Hi all, I need some suggestions about the pipeline on Logstash. I have running the pipeline on Logstash, and suddenly I have an issue. The pipeline is configured that is automatically changed without restarting the se… --- ## [Help receiving logs in Logstash deployed in Heroku](https://discuss.elastic.co/t/help-receiving-logs-in-logstash-deployed-in-heroku/326795)
**Author:** [@lglt](https://discuss.elastic.co/u/lglt)\ **Replies:** 6\ **Last updated:** [March 2, 2023, 2:43pm UTC](https://discuss.elastic.co/t/help-receiving-logs-in-logstash-deployed-in-heroku/326795 "2023-03-02T14:43:22Z")
Hi! I just deployed my Logstash project in Heroku. My plan is use it to process and send the logs to my Elastic Cloud deploy (Kibana and Elasticsearch). Logstash is running successfully. These are the Logstash logs that… --- ## [Logstash is processing old events](https://discuss.elastic.co/t/logstash-is-processing-old-events/326336)
**Author:** [@Nikhitha\_Karennagari](https://discuss.elastic.co/u/Nikhitha_Karennagari)\ **Replies:** 7\ **Last updated:** [March 2, 2023, 1:11pm UTC](https://discuss.elastic.co/t/logstash-is-processing-old-events/326336 "2023-03-02T13:11:57Z")
Getting continuous errors like below in logstash { "timestamp": "2023-02-10T14:04:33.661-08:00", "severity": "warning", "message": "Could not index event to Elasticsearch. {:status=\>404, :action=\>\['index', {:\_id=\>nil, :… --- ## [Ruby filter counting error Workers](https://discuss.elastic.co/t/ruby-filter-counting-error-workers/326330)
**Author:** [@puched](https://discuss.elastic.co/u/puched)\ **Replies:** 6\ **Last updated:** [March 2, 2023, 12:51pm UTC](https://discuss.elastic.co/t/ruby-filter-counting-error-workers/326330 "2023-03-02T12:51:42Z")
I want to store the line number as the document\_id, but i saw that sometimes its not storing in the right way the numbers in order, i guess its because of the multiple workers, the question is Is there a way to store num… --- ## [How does Logstash convert a integer value to another integer value](https://discuss.elastic.co/t/how-does-logstash-convert-a-integer-value-to-another-integer-value/326830)
**Author:** [@wensi](https://discuss.elastic.co/u/wensi)\ **Replies:** 1\ **Last updated:** [March 2, 2023, 11:02am UTC](https://discuss.elastic.co/t/how-does-logstash-convert-a-integer-value-to-another-integer-value/326830 "2023-03-02T11:02:10Z")
In the following pipeline, I want to add a shift (1000000) to id column value, id was 1, and I want it to be 1000001. However, with mutate update generates a string "1 + 1000000" instead of making integer shifted. I al… --- ## [Logstash isn't sending data to elastic](https://discuss.elastic.co/t/logstash-isnt-sending-data-to-elastic/326686)
**Author:** [@Uzzi](https://discuss.elastic.co/u/Uzzi)\ **Replies:** 8\ **Last updated:** [March 1, 2023, 6:34pm UTC](https://discuss.elastic.co/t/logstash-isnt-sending-data-to-elastic/326686 "2023-03-01T18:34:18Z")
Hi, I've 1 vmq for Kibana+elasticsearch and 1 vm for Logstash. Logstash isn't sending data to elastic, this is log: \[DEBUG\]\[logstash.instrument.periodicpoller.cgroup\] One or more required cgroup files or directories no… --- ## [Groke parse failure on Haproxy logs while debugger is OK](https://discuss.elastic.co/t/groke-parse-failure-on-haproxy-logs-while-debugger-is-ok/326779)
**Author:** [@Bastien\_Bsc](https://discuss.elastic.co/u/Bastien_Bsc)\ **Replies:** 1\ **Last updated:** [March 1, 2023, 4:29pm UTC](https://discuss.elastic.co/t/groke-parse-failure-on-haproxy-logs-while-debugger-is-ok/326779 "2023-03-01T16:29:42Z")
Hello, I have a weird situation where the data is correctly parsed, grok debugger doesn't return errors. But logstash still adds a grokeparse\_failure tag. Here is an exemple log (in /var/log/haproxy.log) : Mar 1 15:4… --- ## [Logstash how to mutate string of float array to denseVector](https://discuss.elastic.co/t/logstash-how-to-mutate-string-of-float-array-to-densevector/326750)
**Author:** [@wensi](https://discuss.elastic.co/u/wensi)\ **Replies:** 1\ **Last updated:** [March 1, 2023, 11:03am UTC](https://discuss.elastic.co/t/logstash-how-to-mutate-string-of-float-array-to-densevector/326750 "2023-03-01T11:03:50Z")
I am using Logstash to transfer data from MySQL to ES, one column in MySQL is \`vec\` text NOT NULL vec has value like \[0.1, 0.2, 0.3\] and is of string type. How to convert string of float array to ES dense\_vector? I tr… --- ## [Autodetect\_column\_names in condition](https://discuss.elastic.co/t/autodetect-column-names-in-condition/326439)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 6\ **Last updated:** [March 1, 2023, 10:36am UTC](https://discuss.elastic.co/t/autodetect-column-names-in-condition/326439 "2023-03-01T10:36:45Z")
Hi Maybe You have idea why in this case autodetect\_column\_names doesn't work as independent. input: "CLLI","SWREL","RPTDATE","RPTIME","TZ","RPTTYPE","RPTPD","IVALDATE","IVALSTART","IVALEND","NUMENTIDS" "wifi06","EAGL… --- ## [Output stdout does not print to shell when given info log level](https://discuss.elastic.co/t/output-stdout-does-not-print-to-shell-when-given-info-log-level/326392)
**Author:** [@wensi](https://discuss.elastic.co/u/wensi)\ **Replies:** 1\ **Last updated:** [March 1, 2023, 9:47am UTC](https://discuss.elastic.co/t/output-stdout-does-not-print-to-shell-when-given-info-log-level/326392 "2023-03-01T09:47:19Z")
The following script reads from mysql through jdbc plugin, and output every item through stdout. However, by running logstash -f mysql2es.conf， it does not print anything. Then I added --debug, I can see entities jdbc re… --- ## [Run Logstash manually without interrupting logstash service and logstash Scheduler](https://discuss.elastic.co/t/run-logstash-manually-without-interrupting-logstash-service-and-logstash-scheduler/326036)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 8\ **Last updated:** [March 1, 2023, 6:25am UTC](https://discuss.elastic.co/t/run-logstash-manually-without-interrupting-logstash-service-and-logstash-scheduler/326036 "2023-03-01T06:25:43Z")
Hello, I have a configuration file with http\_poller plugins, where I have some scheduler which on the given schedule pull the data and enter it into my elastic db. But to manually run lagstash instantly, What I have to… --- ## [Logstash Netflow Codec Plugin - "unsupported enterprise" error with IPFIX template](https://discuss.elastic.co/t/logstash-netflow-codec-plugin-unsupported-enterprise-error-with-ipfix-template/326701)
**Author:** [@nosql\_injection](https://discuss.elastic.co/u/nosql_injection)\ **Replies:** 0\ **Last updated:** [February 28, 2023, 6:29pm UTC](https://discuss.elastic.co/t/logstash-netflow-codec-plugin-unsupported-enterprise-error-with-ipfix-template/326701 "2023-02-28T18:29:29Z")
Hi there, when trying to ingest IPFIX, we get the Can't (yet) decode flowset id 317 from observation domain id 6422528, because no template to decode it with has been received. This message will usually go away after … --- ## [Using aggregate to add modsecurity data to previous event](https://discuss.elastic.co/t/using-aggregate-to-add-modsecurity-data-to-previous-event/326423)
**Author:** [@admin\_berlin](https://discuss.elastic.co/u/admin_berlin)\ **Replies:** 3\ **Last updated:** [February 28, 2023, 7:33pm UTC](https://discuss.elastic.co/t/using-aggregate-to-add-modsecurity-data-to-previous-event/326423 "2023-02-28T19:33:13Z")
Hi, I have a modsec logfile that looks like this: --8afa774c-A-- \[24/Feb/2023:04:34:53 +0100\] Y-WoWiTsAtEsT123 12.139.152.111 14327 10.29.14.193 8080 --8afa774c-B-- POST /its/a/test/dude HTTP/1.1 Host: www.my-site.de Co… --- ## [Query not executing in Elasticsearch input plugin for logstash](https://discuss.elastic.co/t/query-not-executing-in-elasticsearch-input-plugin-for-logstash/326362)
**Author:** [@aelam](https://discuss.elastic.co/u/aelam)\ **Replies:** 6\ **Last updated:** [February 28, 2023, 3:55pm UTC](https://discuss.elastic.co/t/query-not-executing-in-elasticsearch-input-plugin-for-logstash/326362 "2023-02-28T15:55:15Z")
I'm new to the stack, and am trying to execute simple queries in logstash via the elasticsearch input plugin. I have worked through some initial errors and now have only a couple of notable warnings, but am not getting a… --- ## [Dynamic naming of elasticsearch data-streams](https://discuss.elastic.co/t/dynamic-naming-of-elasticsearch-data-streams/325278)
**Author:** [@sbocquet](https://discuss.elastic.co/u/sbocquet)\ **Replies:** 12\ **Last updated:** [February 28, 2023, 3:22pm UTC](https://discuss.elastic.co/t/dynamic-naming-of-elasticsearch-data-streams/325278 "2023-02-28T15:22:30Z")
Hi, I'm trying to have some dynamic naming for my data streams based on some syslog fields. Here is my rsyslog conf file for sending datas in JSON format to logstash. # cat logstash-json.conf template(name="json-templ… --- ## [Capture Log for Logstash For every successfull pipeline execution](https://discuss.elastic.co/t/capture-log-for-logstash-for-every-successfull-pipeline-execution/326298)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 6\ **Last updated:** [February 28, 2023, 11:23am UTC](https://discuss.elastic.co/t/capture-log-for-logstash-for-every-successfull-pipeline-execution/326298 "2023-02-28T11:23:00Z")
Hello, I want to capture log for logstash sucessfully fetching the api data from http\_poller plugin and entering it to my elasticDB. My configuration is: input { http\_poller { id =\> "test-plugin" urls =\> { t… --- ## [Unable to split the data in message field](https://discuss.elastic.co/t/unable-to-split-the-data-in-message-field/325987)
**Author:** [@anik-27](https://discuss.elastic.co/u/anik-27)\ **Replies:** 10\ **Last updated:** [February 28, 2023, 7:45am UTC](https://discuss.elastic.co/t/unable-to-split-the-data-in-message-field/325987 "2023-02-28T07:45:19Z")
Hello I am running a python file using exec input plugin, the python file is making multiple api calls and collecting the data in a list(array). I am printing the list and getting the data in message field, but I am un… --- ## [Logstash log separation per pipeline doesn't work!](https://discuss.elastic.co/t/logstash-log-separation-per-pipeline-doesnt-work/324975)
**Author:** [@Siavash\_Fazli](https://discuss.elastic.co/u/Siavash_Fazli)\ **Replies:** 8\ **Last updated:** [February 28, 2023, 6:57am UTC](https://discuss.elastic.co/t/logstash-log-separation-per-pipeline-doesnt-work/324975 "2023-02-28T06:57:58Z")
Hi guys. We use ELK version 8.4.2. on logstash, we have 15 pipelines. Now we need to separate the logs. There is a solution in the elastic document: This document says to insert 2 directives in logstash.yml that aut… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=86) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=88)