# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=102 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 103 --- ## [Silent error on date parsing](https://discuss.elastic.co/t/silent-error-on-date-parsing/320187)
**Author:** [@rfirpo](https://discuss.elastic.co/u/rfirpo)\ **Replies:** 3\ **Last updated:** [November 30, 2022, 9:19pm UTC](https://discuss.elastic.co/t/silent-error-on-date-parsing/320187 "2022-11-30T21:19:16Z")
Logstash is silently dropping my logs after implementing a small modification in my grok filter. The original filter looks like: filter { if \[internal\]\[logtype\] == "mycustomtype" { grok { match =\> { … --- ## [How to load balance data coming from various applications to logstash](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989)
**Author:** [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Replies:** 6\ **Last updated:** [November 30, 2022, 7:32pm UTC](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989 "2022-11-30T19:32:23Z")
Hi All, I need some suggestions on how/what can be used to balance the data that is coming from 1000 of applications to logstash. Architecture looks like this: We have various sources who's logs needs to be integrated… --- ## [How to convert latitude and longitude in logstash and store in elasticserach in a proper format](https://discuss.elastic.co/t/how-to-convert-latitude-and-longitude-in-logstash-and-store-in-elasticserach-in-a-proper-format/320116)
**Author:** [@Dhanushka\_Samarasing](https://discuss.elastic.co/u/Dhanushka_Samarasing)\ **Replies:** 1\ **Last updated:** [November 30, 2022, 5:45pm UTC](https://discuss.elastic.co/t/how-to-convert-latitude-and-longitude-in-logstash-and-store-in-elasticserach-in-a-proper-format/320116 "2022-11-30T17:45:04Z")
I have latitude and longitude data in a Postgresql DB, now I need to move those data to Elasticsearch through Logstash pipeline. I have tried different methods but non of them did not work so far. This is the sample rec… --- ## [Number of open sockets](https://discuss.elastic.co/t/number-of-open-sockets/320087)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 3\ **Last updated:** [November 30, 2022, 2:19pm UTC](https://discuss.elastic.co/t/number-of-open-sockets/320087 "2022-11-30T14:19:36Z")
Hi I'm getting regularly such output \>worker0\] elasticsearch - Failed to perform request {:message=\>"Connection reset", :exception=\>Manticore::SocketException, :cause=\>java.ne \[WARN \] 2022-11-29 18:33:45.720 \[\[my\_pipel… --- ## [Logstash Scheduled Task](https://discuss.elastic.co/t/logstash-scheduled-task/319822)
**Author:** [@smam](https://discuss.elastic.co/u/smam)\ **Replies:** 2\ **Last updated:** [November 30, 2022, 11:04am UTC](https://discuss.elastic.co/t/logstash-scheduled-task/319822 "2022-11-30T11:04:33Z")
I read this post: Running Logstash on Windows | Logstash Reference \[8.5\] | Elastic And configured my Server to run Logstash every day. First question: I know that you can run logstash once and then just wait for new d… --- ## [Unable to connect to logstash from filebeat](https://discuss.elastic.co/t/unable-to-connect-to-logstash-from-filebeat/320131)
**Author:** [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\ **Replies:** 0\ **Last updated:** [November 30, 2022, 9:21am UTC](https://discuss.elastic.co/t/unable-to-connect-to-logstash-from-filebeat/320131 "2022-11-30T09:21:47Z")
Hi I've setup filebeat on the machine A elasticsearch and kibana on machine B and logstash on machine C. When i try to send logs from filebeat to logstash i am getting a connection refused error in status. all the servi… --- ## [KafkaException javax.security.auth.login.LoginException on latest logstash version](https://discuss.elastic.co/t/kafkaexception-javax-security-auth-login-loginexception-on-latest-logstash-version/320126)
**Author:** [@Ganesh4](https://discuss.elastic.co/u/Ganesh4)\ **Replies:** 0\ **Last updated:** [November 30, 2022, 8:42am UTC](https://discuss.elastic.co/t/kafkaexception-javax-security-auth-login-loginexception-on-latest-logstash-version/320126 "2022-11-30T08:42:13Z")
ssl.truststore.password = null ssl.truststore.type = JKS transaction.timeout.ms = 60000 transactional.id = null value.serializer = class org.apache.kafka.common.serialization.StringSerializer \[2022-1… --- ## [Unable to start logstash service Windows server 64 bit](https://discuss.elastic.co/t/unable-to-start-logstash-service-windows-server-64-bit/319363)
**Author:** [@jisha](https://discuss.elastic.co/u/jisha)\ **Replies:** 8\ **Last updated:** [November 30, 2022, 5:26am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-service-windows-server-64-bit/319363 "2022-11-30T05:26:05Z")
Hi, I have downloaded and unzipped logstash and tried to install it as service using nssm. Logstash gets successfully installed but when i try to start the service it is showing as below screenshot. This is the erro… --- ## [Prevent collision of field types of structured logs](https://discuss.elastic.co/t/prevent-collision-of-field-types-of-structured-logs/319806)
**Author:** [@katexochen](https://discuss.elastic.co/u/katexochen)\ **Replies:** 3\ **Last updated:** [November 29, 2022, 5:01pm UTC](https://discuss.elastic.co/t/prevent-collision-of-field-types-of-structured-logs/319806 "2022-11-29T17:01:09Z")
I'm collecting logs to filebeat, sending them to logstash and from there to elastic. As the logs are structured, I'm using the json filter in logstash to parse the message. Let's say I'm having two messages/logs in json… --- ## [Change timestamp in logstash input](https://discuss.elastic.co/t/change-timestamp-in-logstash-input/319506)
**Author:** [@zerratriani](https://discuss.elastic.co/u/zerratriani)\ **Replies:** 8\ **Last updated:** [November 29, 2022, 8:21am UTC](https://discuss.elastic.co/t/change-timestamp-in-logstash-input/319506 "2022-11-29T08:21:49Z")
Hi, I have a log like this and want to change the timestamp to the time in the log. Example Log event1: aaaaaaa | event2: xxxxxxx | event3: ccccccc | date : 2022-11-07T21:03:48.9110; The logstash input { file { … --- ## [Centralized pipeline managent or not for new setup](https://discuss.elastic.co/t/centralized-pipeline-managent-or-not-for-new-setup/320003)
**Author:** [@Soren\_vdc](https://discuss.elastic.co/u/Soren_vdc)\ **Replies:** 0\ **Last updated:** [November 29, 2022, 7:30am UTC](https://discuss.elastic.co/t/centralized-pipeline-managent-or-not-for-new-setup/320003 "2022-11-29T07:30:41Z")
Hi, I have some sources that will sent his data by TCP and not the regular filebeat setup. I will setup logstash to collect the TCP data and send to ES. Now I'm checking to use "Centralized pipeline management" setup in… --- ## [Enabling SSL with Elasticsearch cause logstash pipeline error](https://discuss.elastic.co/t/enabling-ssl-with-elasticsearch-cause-logstash-pipeline-error/319131)
**Author:** [@Indigo\_Star](https://discuss.elastic.co/u/Indigo_Star)\ **Replies:** 9\ **Last updated:** [November 29, 2022, 2:05am UTC](https://discuss.elastic.co/t/enabling-ssl-with-elasticsearch-cause-logstash-pipeline-error/319131 "2022-11-29T02:05:32Z")
hi, I am using 8.4.3, for both elastic and logstash. We have enabled ssl with Elasticsearch. with the ssl our logstash fails to output data there, without ssl it works fine. the output settings: output{ elasticsearch… --- ## [Logstash Pipeline Tester 2.0 - Now more user friendly](https://discuss.elastic.co/t/logstash-pipeline-tester-2-0-now-more-user-friendly/319984)
**Author:** [@epacke](https://discuss.elastic.co/u/epacke)\ **Replies:** 0\ **Last updated:** [November 28, 2022, 10:05pm UTC](https://discuss.elastic.co/t/logstash-pipeline-tester-2-0-now-more-user-friendly/319984 "2022-11-28T22:05:57Z")
Hi there! A few years ago I wrote a a tool for testing logstash pipelines. This year I did a complete redesign of the web interface using Material UI making it more user friendly. Basically it's a tool that: Gives yo… --- ## [Messsage throws \[beats\_input\_codec\_plain\_applied, \_grokparsefailure\] although the Grok pattern is accepted in Dev Tools Debugger](https://discuss.elastic.co/t/messsage-throws-beats-input-codec-plain-applied-grokparsefailure-although-the-grok-pattern-is-accepted-in-dev-tools-debugger/319942)
**Author:** [@franknord](https://discuss.elastic.co/u/franknord)\ **Replies:** 1\ **Last updated:** [November 28, 2022, 1:15pm UTC](https://discuss.elastic.co/t/messsage-throws-beats-input-codec-plain-applied-grokparsefailure-although-the-grok-pattern-is-accepted-in-dev-tools-debugger/319942 "2022-11-28T13:15:49Z")
In my Logstash configuration I have a block saying: else if \[fields\]\[source\] == "Scan4SoftwareAndLicenses" { grok { match =\> { "message" =\> "\\"MachineData\\",\\"%{WORD:host\_… --- ## [Base64 decode issue](https://discuss.elastic.co/t/base64-decode-issue/319874)
**Author:** [@Rajesh\_R](https://discuss.elastic.co/u/Rajesh_R)\ **Replies:** 2\ **Last updated:** [November 28, 2022, 2:55am UTC](https://discuss.elastic.co/t/base64-decode-issue/319874 "2022-11-28T02:55:18Z")
Hi Team, Does Anyone know how to decode base64 in logstash? Logfile : {"ID":"11166946081959","Type":"LOG","pID":"rajesh.r@gmail.com","Interface":"Offboarding","payload": "PD94bWzCoHZlcnNpb249IjEuMCLCoGVuY29kaW5nPSJJU08… --- ## [Logging information is displayed in the wrong index](https://discuss.elastic.co/t/logging-information-is-displayed-in-the-wrong-index/319828)
**Author:** [@sharbich](https://discuss.elastic.co/u/sharbich)\ **Replies:** 8\ **Last updated:** [November 26, 2022, 5:43pm UTC](https://discuss.elastic.co/t/logging-information-is-displayed-in-the-wrong-index/319828 "2022-11-26T17:43:54Z")
Hello, I have two conf files with different indexes in the "/etc/logstash/conf.d/" folder. These are as follows: root@dsme01:~# cat /etc/logstash/conf.d/02-snmp.conf input { snmp { tables =\> \[{ "name" =\> "i… --- ## [How to parse \_jsonparsefailure from Laravel with backslashes](https://discuss.elastic.co/t/how-to-parse-jsonparsefailure-from-laravel-with-backslashes/319724)
**Author:** [@strazhnyk](https://discuss.elastic.co/u/strazhnyk)\ **Replies:** 8\ **Last updated:** [November 25, 2022, 4:36pm UTC](https://discuss.elastic.co/t/how-to-parse-jsonparsefailure-from-laravel-with-backslashes/319724 "2022-11-25T16:36:28Z")
Could anyone help me with the reason why logstash doesn't parse Laraver logs, please? Where is the core problem? Interesting that access logs and some errors log are parsed well. I can't understand why Laravel access lo… --- ## [Logstash sometimes ignoring datastream configuration in elasticsearch output](https://discuss.elastic.co/t/logstash-sometimes-ignoring-datastream-configuration-in-elasticsearch-output/319743)
**Author:** [@Alexander\_A](https://discuss.elastic.co/u/Alexander_A)\ **Replies:** 1\ **Last updated:** [November 25, 2022, 6:25am UTC](https://discuss.elastic.co/t/logstash-sometimes-ignoring-datastream-configuration-in-elasticsearch-output/319743 "2022-11-25T06:25:26Z")
Hello, we are running logstash 8.5.0 with multiple pipelines outputting to elasticsearch. Most of the time this works fine, but sometimes logstash will ignore the datastream configuration on startup and tries to write … --- ## [Logstash retries 404 instead of dropping - output stuck](https://discuss.elastic.co/t/logstash-retries-404-instead-of-dropping-output-stuck/319741)
**Author:** [@Harm](https://discuss.elastic.co/u/Harm)\ **Replies:** 3\ **Last updated:** [November 24, 2022, 7:08pm UTC](https://discuss.elastic.co/t/logstash-retries-404-instead-of-dropping-output-stuck/319741 "2022-11-24T19:08:08Z")
Hi! Today I had a situation where one of our Elasticsearch outputs in logstash got unavailable. It replied with a 404. Based upon the documentation logstash should give a warning and drop the event. We have no DLQ confi… --- ## [How to do calculations in Logstash](https://discuss.elastic.co/t/how-to-do-calculations-in-logstash/319759)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 0\ **Last updated:** [November 24, 2022, 1:23pm UTC](https://discuss.elastic.co/t/how-to-do-calculations-in-logstash/319759 "2022-11-24T13:23:30Z")
Hello, Please find my configuration file structure input{ http\_poller{ // here I have mentioned API url and authorisation tags =\> \["tag1"\] } http\_poller{ // here I have mentioned API url and authorisation tags =\> \["tag… --- ## [Logstash Shutiing Down](https://discuss.elastic.co/t/logstash-shutiing-down/319744)
**Author:** [@ChinigamiHunter](https://discuss.elastic.co/u/ChinigamiHunter)\ **Replies:** 0\ **Last updated:** [November 24, 2022, 11:19am UTC](https://discuss.elastic.co/t/logstash-shutiing-down/319744 "2022-11-24T11:19:38Z")
i'm new to ELK i have a folder contient json files and i'm using logstash for indexing sometimes i get this error \[2022-11-24T11:08:58,748\]\[WARN \]\[logstash.runner \] SIGINT received. Shutting down. \[2022-11-24… --- ## [I get error Unable to configure plugins: (ArgumentError) Cannot determine timezone from nil](https://discuss.elastic.co/t/i-get-error-unable-to-configure-plugins-argumenterror-cannot-determine-timezone-from-nil/319531)
**Author:** [@mdinalova](https://discuss.elastic.co/u/mdinalova)\ **Replies:** 5\ **Last updated:** [November 24, 2022, 9:08am UTC](https://discuss.elastic.co/t/i-get-error-unable-to-configure-plugins-argumenterror-cannot-determine-timezone-from-nil/319531 "2022-11-24T09:08:18Z")
Hello guys! I want to run my logtstash connect to elastic using input jdbc. Then i create conf file to run logstash but i always get this error, i try any changes in conf file but still not working. Anyone can help me t… --- ## [Logstash stopped processing](https://discuss.elastic.co/t/logstash-stopped-processing/319641)
**Author:** [@shubham.s](https://discuss.elastic.co/u/shubham.s)\ **Replies:** 10\ **Last updated:** [November 24, 2022, 7:27am UTC](https://discuss.elastic.co/t/logstash-stopped-processing/319641 "2022-11-24T07:27:29Z")
Hi, The elasticsearch and kibana are working as intentional but after installing and configuring logstash in centos , the service stops \[2022-11-23T15:20:38,842\]\[INFO \]\[logstash.runner \] Log4j configuration pa… --- ## [How will create a global variable in logstash?](https://discuss.elastic.co/t/how-will-create-a-global-variable-in-logstash/319705)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 0\ **Last updated:** [November 24, 2022, 7:08am UTC](https://discuss.elastic.co/t/how-will-create-a-global-variable-in-logstash/319705 "2022-11-24T07:08:57Z")
Hello, My config is like this - input{ http\_poller{ // here I have mentioned API url and authorisation tags =\> \["tag1"\] } http\_poller{ // here I have mentioned API url and authorisation tags =\> \["tag2"\] } http\_poller{ … --- ## [Ingest only ERROR and warning logs to elastic search](https://discuss.elastic.co/t/ingest-only-error-and-warning-logs-to-elastic-search/319634)
**Author:** [@rahul\_sirugudi](https://discuss.elastic.co/u/rahul_sirugudi)\ **Replies:** 2\ **Last updated:** [November 24, 2022, 4:55am UTC](https://discuss.elastic.co/t/ingest-only-error-and-warning-logs-to-elastic-search/319634 "2022-11-24T04:55:28Z")
Hi I am trying to ingest only ERROR and warning logs to Elasticsearch. Currently filebeat is shipping logs to log stash, i am using grok pattern. grok { match =\> { "message" =\> "%{IPV4:ip} - \\\[%{TIMESTAMP\_ISO8601:timest… --- ## [Logstash tcp plugin](https://discuss.elastic.co/t/logstash-tcp-plugin/319584)
**Author:** [@SilasMuniz1](https://discuss.elastic.co/u/SilasMuniz1)\ **Replies:** 3\ **Last updated:** [November 24, 2022, 12:08am UTC](https://discuss.elastic.co/t/logstash-tcp-plugin/319584 "2022-11-24T00:08:35Z")
Hello, I need convert a data field to date field. I am working in restore processed, where I get an old file and insert into elasticsearch by logstash. I received a file with this date format \[06/Jun/2022:13:20:01 -03… --- ## [How to iterate through json array and print a formatted string?](https://discuss.elastic.co/t/how-to-iterate-through-json-array-and-print-a-formatted-string/319676)
**Author:** [@learningelastic](https://discuss.elastic.co/u/learningelastic)\ **Replies:** 1\ **Last updated:** [November 23, 2022, 6:27pm UTC](https://discuss.elastic.co/t/how-to-iterate-through-json-array-and-print-a-formatted-string/319676 "2022-11-23T18:27:47Z")
I'm learning how to use logstash and I got things to mostly work. But I want to learn how to iterate through a json array and print a formatted result set. For example, I created a file called /root/test.log with the fo… --- ## [Logstash x handling duplicate](https://discuss.elastic.co/t/logstash-x-handling-duplicate/319604)
**Author:** [@lchan](https://discuss.elastic.co/u/lchan)\ **Replies:** 2\ **Last updated:** [November 23, 2022, 5:27pm UTC](https://discuss.elastic.co/t/logstash-x-handling-duplicate/319604 "2022-11-23T17:27:26Z")
Hi all, I am trying to increase number of logstash servers for redundancy and want to know if using fingerprint would achieve it. Does this basically sending the same stream of logs/messages via multiple logstash serve… --- ## [Include session variables in the connection string for jdbc\_static fails](https://discuss.elastic.co/t/include-session-variables-in-the-connection-string-for-jdbc-static-fails/319631)
**Author:** [@Muecker](https://discuss.elastic.co/u/Muecker)\ **Replies:** 1\ **Last updated:** [November 23, 2022, 4:24pm UTC](https://discuss.elastic.co/t/include-session-variables-in-the-connection-string-for-jdbc-static-fails/319631 "2022-11-23T16:24:05Z")
Hello, i want to use the jdbc\_static plugin from logstash, but it throws an error: \[2022-11-22T13:36:19,153\]\[WARN \]\[org.mariadb.jdbc.message.server.ErrorPacket\]\[pipeline\_name\] Error: 1064-42000: You have an error in you… --- ## [Split filepath to a new field](https://discuss.elastic.co/t/split-filepath-to-a-new-field/319657)
**Author:** [@nestro](https://discuss.elastic.co/u/nestro)\ **Replies:** 2\ **Last updated:** [November 23, 2022, 2:51pm UTC](https://discuss.elastic.co/t/split-filepath-to-a-new-field/319657 "2022-11-23T14:51:56Z")
Hi! I use Filebeat on a central Syslog server which collects logs from all network devices. Filebeat is configured to collect the logs (which are arrenged by days in the month) from this server and sends them to Logstash… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=101) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=103)