# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=117 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 118 --- ## [Grok pattern - sometimes nesting bracket](https://discuss.elastic.co/t/grok-pattern-sometimes-nesting-bracket/313797)
**Author:** [@elkuser1234](https://discuss.elastic.co/u/elkuser1234)\ **Replies:** 2\ **Last updated:** [September 7, 2022, 6:34am UTC](https://discuss.elastic.co/t/grok-pattern-sometimes-nesting-bracket/313797 "2022-09-07T06:34:08Z")
Hi I have a difficult log for me. Because sometimes I have nested bracket and sometimes i don't How to parse it in a grok. \[2022-09-05 17:27:24,537\] \[apps-thread | test-policy\] WARN \[2022-09-06 14:19:25,708\] \[App (ap… --- ## [Change local ip address to "any text" in elasticsearch logs](https://discuss.elastic.co/t/change-local-ip-address-to-any-text-in-elasticsearch-logs/313771)
**Author:** [@kurdit](https://discuss.elastic.co/u/kurdit)\ **Replies:** 6\ **Last updated:** [September 6, 2022, 10:36pm UTC](https://discuss.elastic.co/t/change-local-ip-address-to-any-text-in-elasticsearch-logs/313771 "2022-09-06T22:36:54Z")
hi all! we collect nginx access logs in elasticsearch. here is an example of incoming nginx access logs: kibana-linux.elk.com 10.211.45.45 - \[05/Sep/2022:15:26:54 +0300\] UNIX-TIME-1662380814.467 "GET /internal/security… --- ## [Apache Access Logs with Filebeats-\>Logstash-\>EL\<-Kibana: Throws errors at default dashboard](https://discuss.elastic.co/t/apache-access-logs-with-filebeats-logstash-el-kibana-throws-errors-at-default-dashboard/313585)
**Author:** [@SirStephanikus](https://discuss.elastic.co/u/SirStephanikus)\ **Replies:** 10\ **Last updated:** [September 6, 2022, 8:39pm UTC](https://discuss.elastic.co/t/apache-access-logs-with-filebeats-logstash-el-kibana-throws-errors-at-default-dashboard/313585 "2022-09-06T20:39:12Z")
Hello Everyone While I'm trying to deep dive into ELK, I followed a video course that is unfortunately outdated... My ELK version is 8.3.3 However, the lab goal is this: Use filebeat to parse some apache access logs,… --- ## [Logstash plugin install error](https://discuss.elastic.co/t/logstash-plugin-install-error/313611)
**Author:** [@zapatannico](https://discuss.elastic.co/u/zapatannico)\ **Replies:** 1\ **Last updated:** [September 3, 2022, 10:20pm UTC](https://discuss.elastic.co/t/logstash-plugin-install-error/313611 "2022-09-03T22:20:36Z")
Hello im new with logstash, im trying to install syslog plugin but i always get this error, someone could help me? \\logstash\\bin\>logstash-plugin install "Using bundled JDK: C:\\logstash\\jdk\\bin\\java.exe" jruby: warning:… --- ## [Ruby filter to check fields dynamically](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709)
**Author:** [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Replies:** 9\ **Last updated:** [September 6, 2022, 6:42pm UTC](https://discuss.elastic.co/t/ruby-filter-to-check-fields-dynamically/313709 "2022-09-06T18:42:30Z")
Hie All, From the following logs I want to check on condition HasErrorState = true and drop the entire event. Data field will be single entry and in that we have multiple entries for Scales. "Data": \[ { … --- ## [Logstash not starting](https://discuss.elastic.co/t/logstash-not-starting/313807)
**Author:** [@mancharagopan](https://discuss.elastic.co/u/mancharagopan)\ **Replies:** 6\ **Last updated:** [September 6, 2022, 4:41pm UTC](https://discuss.elastic.co/t/logstash-not-starting/313807 "2022-09-06T16:41:58Z")
I just installed logstash 8.4.1 on an Ubuntu VM. When i start the service It doesn't start and keep throwing this error. Log4j configuration path used is: /etc/logstash/log4j2.properties \[2022-09-06T14:56:04,394\]\[INFO \]… --- ## [My exec input plugin does not compatible with ecs](https://discuss.elastic.co/t/my-exec-input-plugin-does-not-compatible-with-ecs/313718)
**Author:** [@MKH](https://discuss.elastic.co/u/MKH)\ **Replies:** 2\ **Last updated:** [September 6, 2022, 4:03pm UTC](https://discuss.elastic.co/t/my-exec-input-plugin-does-not-compatible-with-ecs/313718 "2022-09-06T16:03:15Z")
Hi, I am trying to learn ow to use ecs schema within logstash. I use logstash 7.10.0 and input plugin I want to use in my logstash config file is "exec input". I am trying to just follow the documentation guideline htt… --- ## [Error with paging large result set using SQL Server](https://discuss.elastic.co/t/error-with-paging-large-result-set-using-sql-server/313483)
**Author:** [@manuel.urbano](https://discuss.elastic.co/u/manuel.urbano)\ **Replies:** 7\ **Last updated:** [September 6, 2022, 1:27pm UTC](https://discuss.elastic.co/t/error-with-paging-large-result-set-using-sql-server/313483 "2022-09-06T13:27:59Z")
Hello everyone! I'm trying to import a data set with more than 4 million rows from my database in SQL Server. I followed the advice here Jdbc input plugin | Logstash Reference \[8.4\] | Elastic and set jdbc\_fetch\_size wi… --- ## [Parsing advise](https://discuss.elastic.co/t/parsing-advise/313712)
**Author:** [@sta02](https://discuss.elastic.co/u/sta02)\ **Replies:** 4\ **Last updated:** [September 6, 2022, 12:21pm UTC](https://discuss.elastic.co/t/parsing-advise/313712 "2022-09-06T12:21:38Z")
Hello All, How can I parse the below message? The below events are sent over syslog, and the output is sent to Sentinel. I tried to use the KV filter but its failing, as part of parsing, I should also extract the usern… --- ## [Issue with provision large file to logstash](https://discuss.elastic.co/t/issue-with-provision-large-file-to-logstash/313719)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 1\ **Last updated:** [September 6, 2022, 7:32am UTC](https://discuss.elastic.co/t/issue-with-provision-large-file-to-logstash/313719 "2022-09-06T07:32:03Z")
Hi I'm facing with case with large file (~800MB) during transfer to logstash Indeed this is a case where data doesn't match in order This case has begun on (Csv plugin cooperating with multiplying pattern - #11 by Bad… --- ## [\`block in start\_input\` appears when trying to run logstash input using jdbc plugin](https://discuss.elastic.co/t/block-in-start-input-appears-when-trying-to-run-logstash-input-using-jdbc-plugin/313697)
**Author:** [@Khammassi\_HoussemEdd](https://discuss.elastic.co/u/Khammassi_HoussemEdd)\ **Replies:** 2\ **Last updated:** [September 6, 2022, 7:49am UTC](https://discuss.elastic.co/t/block-in-start-input-appears-when-trying-to-run-logstash-input-using-jdbc-plugin/313697 "2022-09-06T07:49:09Z")
Hello i have a mysql database and i want to ingest it's data into elasticsearch using logstash after configuring the input and output i am getting the following error: org/jruby/RubyArray.java:1865:in \`each' /usr/share… --- ## [Urgent Help : How to Parse nested XML (Key value patterns) with multiple Parent children](https://discuss.elastic.co/t/urgent-help-how-to-parse-nested-xml-key-value-patterns-with-multiple-parent-children/312546)
**Author:** [@sonirajil](https://discuss.elastic.co/u/sonirajil)\ **Replies:** 4\ **Last updated:** [September 6, 2022, 7:30am UTC](https://discuss.elastic.co/t/urgent-help-how-to-parse-nested-xml-key-value-patterns-with-multiple-parent-children/312546 "2022-09-06T07:30:14Z")
Hello Community, I have minimal exp. in xml logstash parsing stuff. I have explored various discussions , one such which matches a bit to my case is : \[SOLVED\] Split filter question a.k.a flatten json sub array - #10 b… --- ## [The logs aren't read by Logstash deployed with Docker after rotating](https://discuss.elastic.co/t/the-logs-arent-read-by-logstash-deployed-with-docker-after-rotating/311685)
**Author:** [@kent010341](https://discuss.elastic.co/u/kent010341)\ **Replies:** 4\ **Last updated:** [September 6, 2022, 2:54am UTC](https://discuss.elastic.co/t/the-logs-arent-read-by-logstash-deployed-with-docker-after-rotating/311685 "2022-09-06T02:54:00Z")
I'm using log4j to write logs to files, and whenever the log rotates (compressed to a .gz file), the new log file is never read by Logstash (using version 7.14.4, on MacOS 12.5). I think it's important to track the inod… --- ## [\[Logstash\] windows logstash plugins](https://discuss.elastic.co/t/logstash-windows-logstash-plugins/313702)
**Author:** [@zapatannico](https://discuss.elastic.co/u/zapatannico)\ **Replies:** 1\ **Last updated:** [September 5, 2022, 5:48pm UTC](https://discuss.elastic.co/t/logstash-windows-logstash-plugins/313702 "2022-09-05T17:48:11Z")
Hello, I have some doubts regarding the latest versions of logstash. I am trying to receive syslog events and have them be logged to a .log file My current configuration is the following: input { syslog { por… --- ## [Logs don't show up when trying to use logstash](https://discuss.elastic.co/t/logs-dont-show-up-when-trying-to-use-logstash/312937)
**Author:** [@mammodde](https://discuss.elastic.co/u/mammodde)\ **Replies:** 15\ **Last updated:** [September 5, 2022, 12:33pm UTC](https://discuss.elastic.co/t/logs-dont-show-up-when-trying-to-use-logstash/312937 "2022-09-05T12:33:34Z")
when trying to execute winlogbeat.exe from powershell i receive this kind of errors in logs {"log.level":"error","@timestamp":"xxx","log.logger":"publisher\_pipeline\_output","log.origin":{"file.name":"pipeline/client\_wor… --- ## [Working with messages with string format and split the fields of them in logstash](https://discuss.elastic.co/t/working-with-messages-with-string-format-and-split-the-fields-of-them-in-logstash/313645)
**Author:** [@f.haeri](https://discuss.elastic.co/u/f.haeri)\ **Replies:** 3\ **Last updated:** [September 5, 2022, 10:33am UTC](https://discuss.elastic.co/t/working-with-messages-with-string-format-and-split-the-fields-of-them-in-logstash/313645 "2022-09-05T10:33:05Z")
I have this message: \[ReadDockerQueue\] \[ReadMessageQueue\] message: {"requestType":"PortfolioResponse","parametersJson":"{"algorithmAccessSpecifications":{"dockerName":"D1","xxx":"xxx","userName":"xxx","instanceGuid":"95… --- ## [Logstash centralized pipeline](https://discuss.elastic.co/t/logstash-centralized-pipeline/313625)
**Author:** [@vpolizki](https://discuss.elastic.co/u/vpolizki)\ **Replies:** 2\ **Last updated:** [September 5, 2022, 7:32am UTC](https://discuss.elastic.co/t/logstash-centralized-pipeline/313625 "2022-09-05T07:32:44Z")
Hi, I use Logstash centralized pipeline in my new cluster. I create pipeline for filebeat with: input/filter/output and it is work excellent. Now I want to add another pipeline from different filebeat that use same po… --- ## [\[LOGSTASH\] - Plugin input-udp and message charset](https://discuss.elastic.co/t/logstash-plugin-input-udp-and-message-charset/313617)
**Author:** [@manunc](https://discuss.elastic.co/u/manunc)\ **Replies:** 1\ **Last updated:** [September 4, 2022, 10:47am UTC](https://discuss.elastic.co/t/logstash-plugin-input-udp-and-message-charset/313617 "2022-09-04T10:47:18Z")
Hello, I have a an issue with the message charset received from the input-udp plugin. A raw trace taken using tcpdmp command on linux show the message of the UDP packat like this: Data: 8689630446410580020000001f46e00… --- ## [How to calculate and present times between logs](https://discuss.elastic.co/t/how-to-calculate-and-present-times-between-logs/313324)
**Author:** [@amir\_Bialek](https://discuss.elastic.co/u/amir_Bialek)\ **Replies:** 4\ **Last updated:** [September 3, 2022, 10:06am UTC](https://discuss.elastic.co/t/how-to-calculate-and-present-times-between-logs/313324 "2022-09-03T10:06:59Z")
Hey, new user of elk with Logstash. I am using this as a confid: input { file { path =\> "/myapp/Logs/\*.slog" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { j… --- ## [Filebeat/Logstash Apache Access Logs timestamp issue](https://discuss.elastic.co/t/filebeat-logstash-apache-access-logs-timestamp-issue/313278)
**Author:** [@SirStephanikus](https://discuss.elastic.co/u/SirStephanikus)\ **Replies:** 8\ **Last updated:** [September 2, 2022, 8:24pm UTC](https://discuss.elastic.co/t/filebeat-logstash-apache-access-logs-timestamp-issue/313278 "2022-09-02T20:24:38Z")
Hello everyone I'm new to the ELK world and need some help with a basic Apache HTTP filebeat -\> logstash -\> Elasticsearch \<- Kibana filter/pipeline for learning purposes. Version: 8.3.3 Problem: A dummy apache acce… --- ## [Error starting Logstash](https://discuss.elastic.co/t/error-starting-logstash/313470)
**Author:** [@steevhise](https://discuss.elastic.co/u/steevhise)\ **Replies:** 6\ **Last updated:** [September 2, 2022, 4:06pm UTC](https://discuss.elastic.co/t/error-starting-logstash/313470 "2022-09-02T16:06:58Z")
Logstash is exiting with an error when starting up: \[ERROR\] 2022-09-01 17:35:32.473 \[Converge PipelineAction::Create\\] agent - Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :… --- ## [Logstash multiline codec do not read all lines](https://discuss.elastic.co/t/logstash-multiline-codec-do-not-read-all-lines/313542)
**Author:** [@alex\_96](https://discuss.elastic.co/u/alex_96)\ **Replies:** 2\ **Last updated:** [September 2, 2022, 1:33pm UTC](https://discuss.elastic.co/t/logstash-multiline-codec-do-not-read-all-lines/313542 "2022-09-02T13:33:33Z")
Hi I'm using multiline codec to concatenate lines into one event, here's an example of it: # Time: 2022-08-29T07:43:30.314166Z # User@Host: root\[root\] @ 1.1.1.1. \[\] Id: 111111 # Query\_time: 0.019870 Lock\_time: 0.00000… --- ## [How to Increase checkpoint interval in azure event Hub logstash](https://discuss.elastic.co/t/how-to-increase-checkpoint-interval-in-azure-event-hub-logstash/313548)
**Author:** [@karthik\_Ravichandran](https://discuss.elastic.co/u/karthik_Ravichandran)\ **Replies:** 0\ **Last updated:** [September 2, 2022, 11:46am UTC](https://discuss.elastic.co/t/how-to-increase-checkpoint-interval-in-azure-event-hub-logstash/313548 "2022-09-02T11:46:38Z")
input { azure\_event\_hubs { event\_hub\_connections =\> \["${CLCEHPRIMARYCONNECTIONSTRING};EntityPath=cef"\] storage\_connection =\> "${CLCSTORAGEPRIMARYENDPOINT}" checkpoint\_interval =\> 60 storage\_container =\> "offsets-par… --- ## [Problem handling null value with jdbc\_streaming filter](https://discuss.elastic.co/t/problem-handling-null-value-with-jdbc-streaming-filter/313538)
**Author:** [@Kowsalya\_Mouttou](https://discuss.elastic.co/u/Kowsalya_Mouttou)\ **Replies:** 0\ **Last updated:** [September 2, 2022, 10:20am UTC](https://discuss.elastic.co/t/problem-handling-null-value-with-jdbc-streaming-filter/313538 "2022-09-02T10:20:05Z")
Hi, here is my filter plugin in logsatsh configuration : filter { jdbc\_streaming { jdbc\_driver\_library =\> "/usr/share/logstash/logstash-core/lib/jars/postgresql-42.2.6.jar" jdbc\_driver\_class =\> "org.postgre… --- ## [Logstash giving error "Elasticsearch Unreachable](https://discuss.elastic.co/t/logstash-giving-error-elasticsearch-unreachable/313515)
**Author:** [@Amit\_sah](https://discuss.elastic.co/u/Amit_sah)\ **Replies:** 4\ **Last updated:** [September 2, 2022, 9:34am UTC](https://discuss.elastic.co/t/logstash-giving-error-elasticsearch-unreachable/313515 "2022-09-02T09:34:33Z")
\[2022-09-02T13:20:38,199\]\[WARN \]\[logstash.outputs.elasticsearch\]\[main\] Attempted to resurrect connection to dead ES instance, but got an error {:url=\>"http://elastic:xxxxxx@localhost:9200/", :exception=\>LogStash::Outputs… --- ## [Translate a CURL command into a HTTP filter request](https://discuss.elastic.co/t/translate-a-curl-command-into-a-http-filter-request/313403)
**Author:** [@Ygal\_Mizrachi](https://discuss.elastic.co/u/Ygal_Mizrachi)\ **Replies:** 2\ **Last updated:** [September 1, 2022, 10:28pm UTC](https://discuss.elastic.co/t/translate-a-curl-command-into-a-http-filter-request/313403 "2022-09-01T22:28:10Z")
Hello i need some help translating this CURL request into a http filter syntax curl -k -u username:password -X POST 'https://192.168.88.1/rest/tool/flood-ping' \\--data '{"address":"192.168.88.223","count":"12","interval… --- ## [Error starting logstash version 8.4.1](https://discuss.elastic.co/t/error-starting-logstash-version-8-4-1/313478)
**Author:** [@nathan\_ledall](https://discuss.elastic.co/u/nathan_ledall)\ **Replies:** 6\ **Last updated:** [September 1, 2022, 9:53pm UTC](https://discuss.elastic.co/t/error-starting-logstash-version-8-4-1/313478 "2022-09-01T21:53:04Z")
Hi, I'm daving an issue trying to start logstash. if I run logstash/bat -f logstash.conf everything works fine and logstash even starts but then quickly shuts down and says there is an error any ideas on how to fix it? b… --- ## [Observability stream- logstash](https://discuss.elastic.co/t/observability-stream-logstash/313487)
**Author:** [@preethi\_yogasundaram](https://discuss.elastic.co/u/preethi_yogasundaram)\ **Replies:** 0\ **Last updated:** [September 1, 2022, 9:25pm UTC](https://discuss.elastic.co/t/observability-stream-logstash/313487 "2022-09-01T21:25:54Z")
Hi, i tried to edit the observability stream configuration to include my logstash\* index pattern but i could not apply the changes, i even gave admin privileges but i still couldnt add it. Kindly help me --- ## [Logstash input from multiple redis servers](https://discuss.elastic.co/t/logstash-input-from-multiple-redis-servers/313456)
**Author:** [@kurdit](https://discuss.elastic.co/u/kurdit)\ **Replies:** 2\ **Last updated:** [September 1, 2022, 8:19pm UTC](https://discuss.elastic.co/t/logstash-input-from-multiple-redis-servers/313456 "2022-09-01T20:19:14Z")
hello community! I have the following configuration: 9 nodes, 2 redis servers and 3 logstash servers my logs will go to the redis server (it will work as a broker) -\> then to logstash -\> and to elasticsearch the questi… --- ## [Processing large CSV by file module](https://discuss.elastic.co/t/processing-large-csv-by-file-module/313291)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 4\ **Last updated:** [September 1, 2022, 7:14pm UTC](https://discuss.elastic.co/t/processing-large-csv-by-file-module/313291 "2022-09-01T19:14:03Z")
Hi, I'm facing an issue with decoding my CSV files after that when I've changes the approach to upload file. My first solution has been worked through input on multiply port to logstash. What can I observer that logstas… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=116) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=118)