# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=62 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 63 --- ## [How do parse log format apache tomcat](https://discuss.elastic.co/t/how-do-parse-log-format-apache-tomcat/341529)
**Author:** [@vanhaiit90](https://discuss.elastic.co/u/vanhaiit90)\ **Replies:** 2\ **Last updated:** [August 24, 2023, 7:36am UTC](https://discuss.elastic.co/t/how-do-parse-log-format-apache-tomcat/341529 "2023-08-24T07:36:39Z")
Good moring everyone! I have a log with the format of the tomcat access log service (localaccesslog.txt) 10.0.xx.xx \[22/Aug/2023:00:00:30 +0700\] "GET /zkau?dtid=z\_qe0&cmd\_0=rmDesktop&opt\_0=i HTTP/1.0" 200 17 0 Now I… --- ## [Logstash-7.17.12 file input not working](https://discuss.elastic.co/t/logstash-7-17-12-file-input-not-working/341527)
**Author:** [@Jongwook\_Seong](https://discuss.elastic.co/u/Jongwook_Seong)\ **Replies:** 2\ **Last updated:** [August 24, 2023, 4:44am UTC](https://discuss.elastic.co/t/logstash-7-17-12-file-input-not-working/341527 "2023-08-24T04:44:44Z")
I am using logstash-7.17.12 to input the contents of logstash-test.conf file. The contents of logstash-test.conf are as follows. input { file { path =\> "C:/Users/user/logstash-7.17.12/config/filter-example.log" … --- ## [Looking for data in Kibana](https://discuss.elastic.co/t/looking-for-data-in-kibana/341383)
**Author:** [@Naveen.Bhonagiri](https://discuss.elastic.co/u/Naveen.Bhonagiri)\ **Replies:** 5\ **Last updated:** [August 23, 2023, 8:41pm UTC](https://discuss.elastic.co/t/looking-for-data-in-kibana/341383 "2023-08-23T20:41:46Z")
HI Team, I am looking for a help, i am having some devices list (approx 900 devices) which are injecting logs to Elastic, i want help in finding the devices that never sent logs to elastic from my actual devices. If an… --- ## [Data parse from multiple rsyslog to logstash to elasticsearch](https://discuss.elastic.co/t/data-parse-from-multiple-rsyslog-to-logstash-to-elasticsearch/341506)
**Author:** [@ermilan2309](https://discuss.elastic.co/u/ermilan2309)\ **Replies:** 0\ **Last updated:** [August 23, 2023, 5:38pm UTC](https://discuss.elastic.co/t/data-parse-from-multiple-rsyslog-to-logstash-to-elasticsearch/341506 "2023-08-23T17:38:28Z")
Hello, I am new to ELK. I have deployed my ELK with this article. https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elastic-stack-on-ubuntu-22-04 I skipped the nginx par… --- ## [Logstash stops processing AWS WAF logs when fields exceed 1000 (or any number)](https://discuss.elastic.co/t/logstash-stops-processing-aws-waf-logs-when-fields-exceed-1000-or-any-number/341497)
**Author:** [@feo13](https://discuss.elastic.co/u/feo13)\ **Replies:** 4\ **Last updated:** [August 23, 2023, 4:41pm UTC](https://discuss.elastic.co/t/logstash-stops-processing-aws-waf-logs-when-fields-exceed-1000-or-any-number/341497 "2023-08-23T16:41:41Z")
Hi there, I'm ingesting AWS WAF logs and it works fine for a few minutes but then stops with the following error: response=\>{"index"=\>{"\_index"=\>"waf-logs-2023.08.01", "\_id"=\>"rjCKGYoBsxYs-jwL007l", "status"=\>400, "err… --- ## [Indices are not generating through logstash to see the logs](https://discuss.elastic.co/t/indices-are-not-generating-through-logstash-to-see-the-logs/341394)
**Author:** [@vikascateina](https://discuss.elastic.co/u/vikascateina)\ **Replies:** 0\ **Last updated:** [August 22, 2023, 4:26pm UTC](https://discuss.elastic.co/t/indices-are-not-generating-through-logstash-to-see-the-logs/341394 "2023-08-22T16:26:34Z")
Hi, I have implemented logstash but in my index management I am not able to see Indices of logstash but Ingest pipeline is created as shown in image. I want to push my mule application logs to logstash.How can I do … --- ## [Logstash runs forever but no index got created](https://discuss.elastic.co/t/logstash-runs-forever-but-no-index-got-created/341428)
**Author:** [@uma\_parvathy](https://discuss.elastic.co/u/uma_parvathy)\ **Replies:** 0\ **Last updated:** [August 23, 2023, 6:13am UTC](https://discuss.elastic.co/t/logstash-runs-forever-but-no-index-got-created/341428 "2023-08-23T06:13:30Z")
Hi All, I'm using the below configuration in logstash to populate the data on Elasticsearch 7.17.11 from logstash 7.17.12. It shows pipeline started but no index got created . input { jdbc { jdbc\_driver\_li… --- ## [Unable to start logstash - Tried to load a plugin's code, but failed. {:exception=\>#\ **Author:** [@pavank](https://discuss.elastic.co/u/pavank)\ **Replies:** 2\ **Last updated:** [August 22, 2023, 7:15pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-tried-to-load-a-plugins-code-but-failed-exception-loaderror-no-such-file-to-load-logstash-outputs-microsoft-logstash-output-azure-loganalytics/341403 "2023-08-22T19:15:37Z") Hi We are trying to install the microsoft-logstash-output-azure-loganalytics output plugin to send logs to Azure Log analytics, but getting the following error in Logstash start-up and the Logstash service keeps restart… --- ## [Microsoft-sentinel-log-analytics-logstash-output-plugin functionality](https://discuss.elastic.co/t/microsoft-sentinel-log-analytics-logstash-output-plugin-functionality/341374)
**Author:** [@shadu88](https://discuss.elastic.co/u/shadu88)\ **Replies:** 0\ **Last updated:** [August 22, 2023, 1:40pm UTC](https://discuss.elastic.co/t/microsoft-sentinel-log-analytics-logstash-output-plugin-functionality/341374 "2023-08-22T13:40:10Z")
Hello Dear ELKs i was using "microsoft-sentinel-log-analytics-logstash-output-plugin" to forward the logs to azure sentinel but we switched to AMA( azure native) recently but post this switch the amount of logs doubled/… --- ## [How to Optimize time start Logstash with than 100 condition in output](https://discuss.elastic.co/t/how-to-optimize-time-start-logstash-with-than-100-condition-in-output/341335)
**Author:** [@quoctuan2311](https://discuss.elastic.co/u/quoctuan2311)\ **Replies:** 0\ **Last updated:** [August 22, 2023, 7:37am UTC](https://discuss.elastic.co/t/how-to-optimize-time-start-logstash-with-than-100-condition-in-output/341335 "2023-08-22T07:37:58Z")
Hi, I have built an ES with architect such as picture. And deploy it on AWS EKS. My expected is filebeat will collect logs all pods on EKS. And send it to Logstash. And Logstash will send this to Elasticsearch. At… --- ## [Coerce seems not working](https://discuss.elastic.co/t/coerce-seems-not-working/341019)
**Author:** [@Jan\_Vavra](https://discuss.elastic.co/u/Jan_Vavra)\ **Replies:** 2\ **Last updated:** [August 22, 2023, 7:33am UTC](https://discuss.elastic.co/t/coerce-seems-not-working/341019 "2023-08-22T07:33:54Z")
I am constructing datetime from directory structure, eg. 2023\\08\\17\\15\\08 represent files stored at 2023-08-17 15:08. I have this logstash.conf that parses each directory name into variables and hours and minutes are opt… --- ## [UDP-input Receiving an encoding value �](https://discuss.elastic.co/t/udp-input-receiving-an-encoding-value/341199)
**Author:** [@aurangzeb99](https://discuss.elastic.co/u/aurangzeb99)\ **Replies:** 7\ **Last updated:** [August 22, 2023, 6:36am UTC](https://discuss.elastic.co/t/udp-input-receiving-an-encoding-value/341199 "2023-08-22T06:36:03Z")
Hi I am using logstash udp input and in elasticsearch field event.original have true values. but in a document field.DeviceCapabilities value is "�" and for field.PoleCapabilities is empty. fieldname: event.original Va… --- ## [Cannot change log format with pipeline config file, pipeline config file is not getting read](https://discuss.elastic.co/t/cannot-change-log-format-with-pipeline-config-file-pipeline-config-file-is-not-getting-read/340081)
**Author:** [@Jenkins-Jobs](https://discuss.elastic.co/u/Jenkins-Jobs)\ **Replies:** 7\ **Last updated:** [August 21, 2023, 4:04pm UTC](https://discuss.elastic.co/t/cannot-change-log-format-with-pipeline-config-file-pipeline-config-file-is-not-getting-read/340081 "2023-08-21T16:04:30Z")
Greetings, First time posting here, elasticsearch 8.9 rhel 7 I am getting logs from jenkins jobs using logstash plugin with no issues the only mime type that seems to work is "application/json" If i try any other t… --- ## [Is it possible to disable or remove log4j-core-2.17.1.jar from Logstash?](https://discuss.elastic.co/t/is-it-possible-to-disable-or-remove-log4j-core-2-17-1-jar-from-logstash/341221)
**Author:** [@kam89](https://discuss.elastic.co/u/kam89)\ **Replies:** 1\ **Last updated:** [August 21, 2023, 1:20pm UTC](https://discuss.elastic.co/t/is-it-possible-to-disable-or-remove-log4j-core-2-17-1-jar-from-logstash/341221 "2023-08-21T13:20:59Z")
Hi, We are running on Logstash 8.8.0 and our IT security team has concern about the log4j-core-2.17.1.jar in the logstash-core\\lib\\jars. Can we disable log4j totally in Logstash and remove the log4j-core-2.17.1.jar fro… --- ## [How to configure Logstash pipeline to not OOM-Kill ElasticSearch](https://discuss.elastic.co/t/how-to-configure-logstash-pipeline-to-not-oom-kill-elasticsearch/340949)
**Author:** [@amattice](https://discuss.elastic.co/u/amattice)\ **Replies:** 7\ **Last updated:** [August 21, 2023, 5:18am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-pipeline-to-not-oom-kill-elasticsearch/340949 "2023-08-21T05:18:59Z")
I'm very new here and to the ELK stack in general. I setup an Ubuntu VM in my Azure resource group and installed the latest Elasticsearch,LogStash, and Kibana. I have basic user authentication setup for kibana, and no SS… --- ## [Using jdbc\_static to build connectionstring that are used in later step](https://discuss.elastic.co/t/using-jdbc-static-to-build-connectionstring-that-are-used-in-later-step/341189)
**Author:** [@johanwallenborg](https://discuss.elastic.co/u/johanwallenborg)\ **Replies:** 2\ **Last updated:** [August 20, 2023, 7:17pm UTC](https://discuss.elastic.co/t/using-jdbc-static-to-build-connectionstring-that-are-used-in-later-step/341189 "2023-08-20T19:17:02Z")
I found and read about jdbc\_static and trying to get my head around. But I have some questions before i dive deeper into this one. Say that I have a database with a table with rows that contains i.e a identifier, datab… --- ## [Logstash doesn't reads JSON file on Windows](https://discuss.elastic.co/t/logstash-doesnt-reads-json-file-on-windows/341156)
**Author:** [@VSKMurali](https://discuss.elastic.co/u/VSKMurali)\ **Replies:** 10\ **Last updated:** [August 19, 2023, 11:35pm UTC](https://discuss.elastic.co/t/logstash-doesnt-reads-json-file-on-windows/341156 "2023-08-19T23:35:17Z")
Hello, I am trying to configure a following setup on Windows machine. JSON file (creates every 5 mins with the same file name) and Elasticsearch should read the file and push to Index and this is my Logstash config fil… --- ## [How to Apply Custom Template to Logstash (8.x)](https://discuss.elastic.co/t/how-to-apply-custom-template-to-logstash-8-x/341110)
**Author:** [@inkweon7269](https://discuss.elastic.co/u/inkweon7269)\ **Replies:** 2\ **Last updated:** [August 18, 2023, 11:25pm UTC](https://discuss.elastic.co/t/how-to-apply-custom-template-to-logstash-8-x/341110 "2023-08-18T23:25:31Z")
I am studying Elasticsearch using Docker Compose and ELK. I am having an issue where the custom template is not being applied in logstash.conf. How can I fixed it? csv-template.json { "template": "csv", "order": "1"… --- ## [General advice on sucking in whole databases into Elastic?](https://discuss.elastic.co/t/general-advice-on-sucking-in-whole-databases-into-elastic/341096)
**Author:** [@McJava1967](https://discuss.elastic.co/u/McJava1967)\ **Replies:** 2\ **Last updated:** [August 18, 2023, 1:44pm UTC](https://discuss.elastic.co/t/general-advice-on-sucking-in-whole-databases-into-elastic/341096 "2023-08-18T13:44:23Z")
Hi all. I'm working on pulling my company's relational data into Elastic using the Logstash JDBC. It's working, but I have a general question. Currently, I've just written a JOIN over two tables containing a few field… --- ## [Filter elasticsearch data with logstash](https://discuss.elastic.co/t/filter-elasticsearch-data-with-logstash/341077)
**Author:** [@john.hoogeveen](https://discuss.elastic.co/u/john.hoogeveen)\ **Replies:** 2\ **Last updated:** [August 18, 2023, 10:01am UTC](https://discuss.elastic.co/t/filter-elasticsearch-data-with-logstash/341077 "2023-08-18T10:01:33Z")
Hello, I am trying to export some data from an elastic stack using logstash but it doesn't work. For this I connected it to a test stack with this config file input { elasticsearch { hosts =\> "localhost:9200" … --- ## [Parsing log in logstash with format xml and json embebed](https://discuss.elastic.co/t/parsing-log-in-logstash-with-format-xml-and-json-embebed/341031)
**Author:** [@Oscar\_Lopez](https://discuss.elastic.co/u/Oscar_Lopez)\ **Replies:** 1\ **Last updated:** [August 17, 2023, 4:21pm UTC](https://discuss.elastic.co/t/parsing-log-in-logstash-with-format-xml-and-json-embebed/341031 "2023-08-17T16:21:53Z")
hello everyone Hello everyone, at this moment I am trying to ingest some logs in elasticsearch with logstash with the following structure: \ID:XXX… --- ## [Logstash kafka input plugin not working](https://discuss.elastic.co/t/logstash-kafka-input-plugin-not-working/340956)
**Author:** [@MheniMerz](https://discuss.elastic.co/u/MheniMerz)\ **Replies:** 2\ **Last updated:** [August 17, 2023, 12:14pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-plugin-not-working/340956 "2023-08-17T12:14:49Z")
Hi, i'm trying to use logstash kafka input plugin to read messages then send them to elasticsearch. my logstash version is 8.6.2 and my kafka version is 3.5.1 root@logstash-02:~# /usr/share/logstash/bin/logstash --ver… --- ## [I am using EVENTHUB input Plugin of Logstash for Capturing Azure event hub Audit logs. But there is Data loss](https://discuss.elastic.co/t/i-am-using-eventhub-input-plugin-of-logstash-for-capturing-azure-event-hub-audit-logs-but-there-is-data-loss/340994)
**Author:** [@Subrato1](https://discuss.elastic.co/u/Subrato1)\ **Replies:** 0\ **Last updated:** [August 17, 2023, 10:21am UTC](https://discuss.elastic.co/t/i-am-using-eventhub-input-plugin-of-logstash-for-capturing-azure-event-hub-audit-logs-but-there-is-data-loss/340994 "2023-08-17T10:21:17Z")
Below Configuration We are using: input { azure\_event\_hubs { config\_mode =\> "basic" #Insert primary connection string from shared access policies in event hub namespace from azure portal event\_hub… --- ## [I'm trying to get xewriter to deliver MSSQL logfiles to my elasticsearch](https://discuss.elastic.co/t/im-trying-to-get-xewriter-to-deliver-mssql-logfiles-to-my-elasticsearch/340826)
**Author:** [@fribse](https://discuss.elastic.co/u/fribse)\ **Replies:** 4\ **Last updated:** [August 17, 2023, 9:21am UTC](https://discuss.elastic.co/t/im-trying-to-get-xewriter-to-deliver-mssql-logfiles-to-my-elasticsearch/340826 "2023-08-17T09:21:18Z")
In my logstash I have this to handle different logfiles with different index names: output { elasticsearch { hosts =\> "https://elasticsearch:9200" index =\> "%{\[fields\]\[logtype\]}-%{\[@metadata\]\[version\]}-%{+YY… --- ## [Can I access data in my 2nd lookup using 1st lookup result using jdbc\_static filter in Logstash for my mariaDB data](https://discuss.elastic.co/t/can-i-access-data-in-my-2nd-lookup-using-1st-lookup-result-using-jdbc-static-filter-in-logstash-for-my-mariadb-data/340977)
**Author:** [@aurangzeb99](https://discuss.elastic.co/u/aurangzeb99)\ **Replies:** 0\ **Last updated:** [August 17, 2023, 7:32am UTC](https://discuss.elastic.co/t/can-i-access-data-in-my-2nd-lookup-using-1st-lookup-result-using-jdbc-static-filter-in-logstash-for-my-mariadb-data/340977 "2023-08-17T07:32:20Z")
Hello I am using jdbc\_static filter in Logstash for my mariaDB data . where I have define 2 local\_lookups. Can I access data in my 2nd lookup using 1st lookup result fields ???????????????? local\_lookups =\> \[ { … --- ## [I am trying to capture audit logs from 2 event hubs but there is data loss or some time not getting the audit logs. I am using below input configuration](https://discuss.elastic.co/t/i-am-trying-to-capture-audit-logs-from-2-event-hubs-but-there-is-data-loss-or-some-time-not-getting-the-audit-logs-i-am-using-below-input-configuration/340924)
**Author:** [@Subrato1](https://discuss.elastic.co/u/Subrato1)\ **Replies:** 0\ **Last updated:** [August 16, 2023, 1:21pm UTC](https://discuss.elastic.co/t/i-am-trying-to-capture-audit-logs-from-2-event-hubs-but-there-is-data-loss-or-some-time-not-getting-the-audit-logs-i-am-using-below-input-configuration/340924 "2023-08-16T13:21:40Z")
input { azure\_event\_hubs { config\_mode =\> "basic" #Insert primary connection string from shared access policies in event hub namespace from azure portal event\_hub\_connections =\> \["\ **Author:** [@Tony\_Stark](https://discuss.elastic.co/u/Tony_Stark)\ **Replies:** 9\ **Last updated:** [August 16, 2023, 12:48pm UTC](https://discuss.elastic.co/t/executing-multiple-conf-files-in-one-instance/340749 "2023-08-16T12:48:52Z") I am trying to execute multiple .conf files in logstash with logstash -f "path\*.conf" but logstash processes the .conf file the same number of times as the number of .conf files I have , if I have 5 .conf files , I get o… --- ## [Excessive RAM usage, gets OOM killed in logstash](https://discuss.elastic.co/t/excessive-ram-usage-gets-oom-killed-in-logstash/340716)
**Author:** [@tanveer14](https://discuss.elastic.co/u/tanveer14)\ **Replies:** 4\ **Last updated:** [August 16, 2023, 9:05am UTC](https://discuss.elastic.co/t/excessive-ram-usage-gets-oom-killed-in-logstash/340716 "2023-08-16T09:05:30Z")
Hi! I'm running pipelines on logstash. Everything ran smoothly, but the POD went down for a certain amount of time. In values.yml ll set JVM option as "logstashJavaOpts: "-Xmx5g -Xms3g" still end up getting killed by… --- ## [Logstash ruby code plugin validate JSON](https://discuss.elastic.co/t/logstash-ruby-code-plugin-validate-json/340889)
**Author:** [@z\_z](https://discuss.elastic.co/u/z_z)\ **Replies:** 0\ **Last updated:** [August 16, 2023, 9:01am UTC](https://discuss.elastic.co/t/logstash-ruby-code-plugin-validate-json/340889 "2023-08-16T09:01:42Z")
My logstash.conf is as follows, it is used to read data from events.txt and use ruby code plugin to add tag to non json event. input { file { path =\> \["/home/events.txt"\] start\_position =\> "beginning… --- ## [Log Filtration Issue with Filebeat and Logstash Configuration](https://discuss.elastic.co/t/log-filtration-issue-with-filebeat-and-logstash-configuration/340609)
**Author:** [@Priyaansh\_Dwivedi](https://discuss.elastic.co/u/Priyaansh_Dwivedi)\ **Replies:** 18\ **Last updated:** [August 16, 2023, 6:02am UTC](https://discuss.elastic.co/t/log-filtration-issue-with-filebeat-and-logstash-configuration/340609 "2023-08-16T06:02:59Z")
Hello everyone, I'm using Filebeat to send three different logs to Logstash, where I'm applying parsing through filters. In the parsing process, I've taken into consideration the logs that are visible in the observation… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=61) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=63)