# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=44 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 45 --- ## [In pipeline: translate causes logstash to crash](https://discuss.elastic.co/t/in-pipeline-translate-causes-logstash-to-crash/351386)
**Author:** [@michael\_c\_michael](https://discuss.elastic.co/u/michael_c_michael)\ **Replies:** 4\ **Last updated:** [January 22, 2024, 7:22pm UTC](https://discuss.elastic.co/t/in-pipeline-translate-causes-logstash-to-crash/351386 "2024-01-22T19:22:22Z")
I'm using a JSON dictionary to convert values to strings, it looks like this: { "1.1.0.80.1.\*.\*": "Motorcycle -\> Generic Scooter (Small)", "1.1.0.80.2.\*.\*": "Motorcycle -\> Generic Sport/Street (Mid-Size)", … --- ## [Logstash filters not working as expected](https://discuss.elastic.co/t/logstash-filters-not-working-as-expected/351536)
**Author:** [@Mansi\_Kamthane](https://discuss.elastic.co/u/Mansi_Kamthane)\ **Replies:** 0\ **Last updated:** [January 22, 2024, 11:58am UTC](https://discuss.elastic.co/t/logstash-filters-not-working-as-expected/351536 "2024-01-22T11:58:03Z")
I am working with logstash filter no filter works here here is the config file of logstash \` input { tcp { id =\> "\*\*\*" port =\> \*\*\* codec =\> json\_lines } } filter { cipher { algorithm =\> "aes-128-cbc" key =\> … --- ## [Passing input value in run time in logstash](https://discuss.elastic.co/t/passing-input-value-in-run-time-in-logstash/351514)
**Author:** [@Sat\_elk](https://discuss.elastic.co/u/Sat_elk)\ **Replies:** 0\ **Last updated:** [January 22, 2024, 5:49am UTC](https://discuss.elastic.co/t/passing-input-value-in-run-time-in-logstash/351514 "2024-01-22T05:49:18Z")
I have a few input parameters like environment, ip\_address, host\_name, path, business\_entity. I have to use this values to validate in my filter plugins. help me. how the input should i store and which format then how t… --- ## [Logstash with Kafka input - enable\_auto\_commit settings to minimise data loss?](https://discuss.elastic.co/t/logstash-with-kafka-input-enable-auto-commit-settings-to-minimise-data-loss/351505)
**Author:** [@ankh](https://discuss.elastic.co/u/ankh)\ **Replies:** 0\ **Last updated:** [January 21, 2024, 11:38pm UTC](https://discuss.elastic.co/t/logstash-with-kafka-input-enable-auto-commit-settings-to-minimise-data-loss/351505 "2024-01-21T23:38:47Z")
am using Logstash running in Kubernetes to ingest data from Kafka and write to Elasticsearch. If a Logstash instance ends abnormally while processing data, it can result in data loss. It appears there is no end-to-end a… --- ## [How to send multiline json file and send it to elasticsearch](https://discuss.elastic.co/t/how-to-send-multiline-json-file-and-send-it-to-elasticsearch/351500)
**Author:** [@mancharagopan](https://discuss.elastic.co/u/mancharagopan)\ **Replies:** 1\ **Last updated:** [January 21, 2024, 5:45pm UTC](https://discuss.elastic.co/t/how-to-send-multiline-json-file-and-send-it-to-elasticsearch/351500 "2024-01-21T17:45:09Z")
I have a json log files which is in multi line json format and i need to send it to elasticsearch as it is. how can i do it? --- ## [Logstash filter if statement when detecting multiple whitespace in middle of string](https://discuss.elastic.co/t/logstash-filter-if-statement-when-detecting-multiple-whitespace-in-middle-of-string/351457)
**Author:** [@mhoward](https://discuss.elastic.co/u/mhoward)\ **Replies:** 2\ **Last updated:** [January 19, 2024, 7:45pm UTC](https://discuss.elastic.co/t/logstash-filter-if-statement-when-detecting-multiple-whitespace-in-middle-of-string/351457 "2024-01-19T19:45:10Z")
Hey all. I'm working on a Logstash pipeline that includes processing addresses. Here's what my sample data might look like: " 123 ABC Street" "456 XYZ Street (a bunch of whitespaces here) PO Box 78… --- ## [My Timestamp in audit log using SYSLOG input plugin not in correct format](https://discuss.elastic.co/t/my-timestamp-in-audit-log-using-syslog-input-plugin-not-in-correct-format/351127)
**Author:** [@Subrato1](https://discuss.elastic.co/u/Subrato1)\ **Replies:** 9\ **Last updated:** [January 19, 2024, 7:07pm UTC](https://discuss.elastic.co/t/my-timestamp-in-audit-log-using-syslog-input-plugin-not-in-correct-format/351127 "2024-01-19T19:07:22Z")
Configuration is below : input { syslog { port =\> 5514 type =\> "scylladb" } } filter { if \[type\] == "scylladb" { grok{ match =\> {"message" =\> "%{IP:server\_ip}:%{DATA:server\_port},\\s%{DATA:categ… --- ## [Logstash Integration Plugin configuration option field issue (Logstash Input Plugin add\_filed issue)](https://discuss.elastic.co/t/logstash-integration-plugin-configuration-option-field-issue-logstash-input-plugin-add-filed-issue/351439)
**Author:** [@siva0030](https://discuss.elastic.co/u/siva0030)\ **Replies:** 3\ **Last updated:** [January 19, 2024, 6:58pm UTC](https://discuss.elastic.co/t/logstash-integration-plugin-configuration-option-field-issue-logstash-input-plugin-add-filed-issue/351439 "2024-01-19T18:58:13Z")
Hello Team, Good noon! Recently, I've upgraded my Logstash to 8.11.3 version. Starting from Logstash 8.11 version, Logstash Integration Plugin is available. I was trying to use the Logstash input plugin to receive the … --- ## [Logstash input/output plugin SSL configuration error](https://discuss.elastic.co/t/logstash-input-output-plugin-ssl-configuration-error/351447)
**Author:** [@matus.vlcek](https://discuss.elastic.co/u/matus.vlcek)\ **Replies:** 0\ **Last updated:** [January 19, 2024, 1:35pm UTC](https://discuss.elastic.co/t/logstash-input-output-plugin-ssl-configuration-error/351447 "2024-01-19T13:35:25Z")
Hi guys, I've tried to configure new default approach for logstash to logstash communication using logstash input, output plugins. It works fine without SSL, but I wasn't able to get SSL to work. It outputs this error o… --- ## [Logstash cannot read new lines that are coming from .NET error exception msg](https://discuss.elastic.co/t/logstash-cannot-read-new-lines-that-are-coming-from-net-error-exception-msg/351340)
**Author:** [@theo003](https://discuss.elastic.co/u/theo003)\ **Replies:** 3\ **Last updated:** [January 19, 2024, 8:33am UTC](https://discuss.elastic.co/t/logstash-cannot-read-new-lines-that-are-coming-from-net-error-exception-msg/351340 "2024-01-19T08:33:55Z")
Hello, Our system is throwing some error exceptions in the logs with the following format: |17 01 2024 08:22:10,614| |ERROR| CreateSession API... File: "File\_name" Line: 290System.InvalidOperationException: "Error\_msg"… --- ## [Buffer overflow issue Flunetd not able to push logs to elasticsearch cluster](https://discuss.elastic.co/t/buffer-overflow-issue-flunetd-not-able-to-push-logs-to-elasticsearch-cluster/351418)
**Author:** [@Music\_World](https://discuss.elastic.co/u/Music_World)\ **Replies:** 0\ **Last updated:** [January 19, 2024, 8:04am UTC](https://discuss.elastic.co/t/buffer-overflow-issue-flunetd-not-able-to-push-logs-to-elasticsearch-cluster/351418 "2024-01-19T08:04:30Z")
Hi @all I am using elasticsearch version: 7.16.2 and fluentd version: 1.14.4 on aws eks cluster and it's throwing bufferoverflow error like failed to flush the buffer. retry\_times=0 next\_retry\_time=2024-01-19 07:43:43… --- ## [Convert string LLA to Geo-Point](https://discuss.elastic.co/t/convert-string-lla-to-geo-point/351376)
**Author:** [@michael\_c\_michael](https://discuss.elastic.co/u/michael_c_michael)\ **Replies:** 3\ **Last updated:** [January 18, 2024, 7:47pm UTC](https://discuss.elastic.co/t/convert-string-lla-to-geo-point/351376 "2024-01-18T19:47:14Z")
I have a string field that is in Latitude, Longitude, Altitude. In the pipeline, I am taking the location in x, y, z in ECEF coordinates and converting to LLA: - pipeline.id: entity-state-processing config.str… --- ## [Codec multiline grok pattern for Logstash](https://discuss.elastic.co/t/codec-multiline-grok-pattern-for-logstash/351334)
**Author:** [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Replies:** 6\ **Last updated:** [January 18, 2024, 5:55pm UTC](https://discuss.elastic.co/t/codec-multiline-grok-pattern-for-logstash/351334 "2024-01-18T17:55:10Z")
Hey, so i am parsing multiline logs using Logstash. I need consider each log line will start with {"offset": currently it is printing into single line that's why it is coming in one message only We are not using file… --- ## [Logstash.input.imap error SSL](https://discuss.elastic.co/t/logstash-input-imap-error-ssl/349772)
**Author:** [@drissm](https://discuss.elastic.co/u/drissm)\ **Replies:** 3\ **Last updated:** [January 18, 2024, 3:51pm UTC](https://discuss.elastic.co/t/logstash-input-imap-error-ssl/349772 "2024-01-18T15:51:22Z")
Hello, i have a logstash v8.11.3 with input.imap plugin v3.2.1. Here is my pipeline and the ssl error i have input { imap { host =\> "myhost" password =\> "mypassword" port =\> 993 user =\> "myemail@mydoma… --- ## [Scale Logstash config for more than 20,000 messages per minute?](https://discuss.elastic.co/t/scale-logstash-config-for-more-than-20-000-messages-per-minute/350552)
**Author:** [@Trung\_Nguyen](https://discuss.elastic.co/u/Trung_Nguyen)\ **Replies:** 14\ **Last updated:** [January 18, 2024, 8:35am UTC](https://discuss.elastic.co/t/scale-logstash-config-for-more-than-20-000-messages-per-minute/350552 "2024-01-18T08:35:09Z")
Hi I'm using rsyslog as a syslog server to get the log from the Firewall and network devices, then it send log to Logstash (on the same host) to do the filter then push to Elastic (on other host). If I forward around 1… --- ## [@timestamp is 4 hours behind when i change timezone in advance setting kibana and database date field is ok](https://discuss.elastic.co/t/timestamp-is-4-hours-behind-when-i-change-timezone-in-advance-setting-kibana-and-database-date-field-is-ok/351107)
**Author:** [@Aslam\_Ansari](https://discuss.elastic.co/u/Aslam_Ansari)\ **Replies:** 6\ **Last updated:** [January 18, 2024, 8:11am UTC](https://discuss.elastic.co/t/timestamp-is-4-hours-behind-when-i-change-timezone-in-advance-setting-kibana-and-database-date-field-is-ok/351107 "2024-01-18T08:11:55Z")
@timestamp is 4 hours behind when I change the timezone as UTC in Kibana's advance settings and the database date field is correct. However, when I set the timezone as browser advance settings in Kibana, @timestamp is co… --- ## [Logstash parsing for dynamic fieldname](https://discuss.elastic.co/t/logstash-parsing-for-dynamic-fieldname/351237)
**Author:** [@Priyanka\_chauhan](https://discuss.elastic.co/u/Priyanka_chauhan)\ **Replies:** 1\ **Last updated:** [January 17, 2024, 7:34pm UTC](https://discuss.elastic.co/t/logstash-parsing-for-dynamic-fieldname/351237 "2024-01-17T19:34:19Z")
hi, I want to parse message: My message part is looking like after applying json filter is nodes.processes.C86BB2FAC5F22D51.user.name: value1 nodes.processes.C86BB2FAC5F22D51.user.sid: value2 nodes.files.EA68B2FAC5… --- ## [Logstash agent.\* fields](https://discuss.elastic.co/t/logstash-agent-fields/351261)
**Author:** [@mwitsas](https://discuss.elastic.co/u/mwitsas)\ **Replies:** 6\ **Last updated:** [January 17, 2024, 4:35pm UTC](https://discuss.elastic.co/t/logstash-agent-fields/351261 "2024-01-17T16:35:08Z")
Is it possible to configure logstash to populate agent.\* fields in the same way beats agents do this e.g. agent.type agent.version ... Many thanks --- ## [How to parse date field into @timestamp](https://discuss.elastic.co/t/how-to-parse-date-field-into-timestamp/351058)
**Author:** [@emoxam](https://discuss.elastic.co/u/emoxam)\ **Replies:** 9\ **Last updated:** [January 17, 2024, 12:10pm UTC](https://discuss.elastic.co/t/how-to-parse-date-field-into-timestamp/351058 "2024-01-17T12:10:13Z")
i receive the spring app logs and i want to parse time from logs to @timestamp that's what i got but timestamp is not the same. input { tcp { port =\> 5000 codec =\>plain } } filter { if \[message\] =~ /actions/ { … --- ## [After Installing Logstash version 8.11.4 it Exit because a System error](https://discuss.elastic.co/t/after-installing-logstash-version-8-11-4-it-exit-because-a-system-error/351035)
**Author:** [@jcourt2006](https://discuss.elastic.co/u/jcourt2006)\ **Replies:** 10\ **Last updated:** [January 16, 2024, 8:56pm UTC](https://discuss.elastic.co/t/after-installing-logstash-version-8-11-4-it-exit-because-a-system-error/351035 "2024-01-16T20:56:56Z")
I get the following error: \[root@app logstash\]# /usr/share/logstash/bin/logstash -t --path.settings /etc/logstash Using bundled JDK: /usr/share/logstash/jdk /usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/concurrent-… --- ## [Why is a new index created with 0001 if you restart logstash?](https://discuss.elastic.co/t/why-is-a-new-index-created-with-0001-if-you-restart-logstash/351167)
**Author:** [@emoxam](https://discuss.elastic.co/u/emoxam)\ **Replies:** 2\ **Last updated:** [January 16, 2024, 5:39pm UTC](https://discuss.elastic.co/t/why-is-a-new-index-created-with-0001-if-you-restart-logstash/351167 "2024-01-16T17:39:27Z")
Why is a new index created with 0001 if you restart logstash? And does not continue to write to the main one, without numbers. But if you restart it again, it keeps writing to 0001 without creating 0002? /etc/logstash/c… --- ## [Error management in elasticsearch output plugin](https://discuss.elastic.co/t/error-management-in-elasticsearch-output-plugin/351100)
**Author:** [@Cesar\_Garcia1](https://discuss.elastic.co/u/Cesar_Garcia1)\ **Replies:** 4\ **Last updated:** [January 16, 2024, 3:41pm UTC](https://discuss.elastic.co/t/error-management-in-elasticsearch-output-plugin/351100 "2024-01-16T15:41:28Z")
Hello everyone Hello to all of you I have a problem with logstash and elasticsearch I have this configuration file for logstash input { file { path =\> "${FILE\_TO\_SEND}" sincedb\_path =\> "/dev/null" mode =\> read… --- ## [Encountered a retryable error. Will Retry with exponential backoff code=\>400](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff-code-400/351147)
**Author:** [@DIVANSHU\_AGARWAL](https://discuss.elastic.co/u/DIVANSHU_AGARWAL)\ **Replies:** 2\ **Last updated:** [January 16, 2024, 12:03pm UTC](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff-code-400/351147 "2024-01-16T12:03:50Z")
Receiving these log traces on logstash end, unable to index documents to Elasticsearch. \[2024-01-15T11:28:39,573\]\[ERROR\]\[logstash.outputs.opensearch\] Encountered a retryable error (will retry with exponential backoff) {… --- ## [Logstash template](https://discuss.elastic.co/t/logstash-template/351131)
**Author:** [@SalehEska](https://discuss.elastic.co/u/SalehEska)\ **Replies:** 3\ **Last updated:** [January 16, 2024, 11:57am UTC](https://discuss.elastic.co/t/logstash-template/351131 "2024-01-16T11:57:08Z")
The problem is it does not create template , the template just for change things in the settings. this is the template : { "index\_patterns": \["audittrail\_transactions\_\*"\], "settings": { "number\_of\_shards": 2, "inde… --- ## [Java::JavaLang::IllegalStateException\` for \`PipelineAction::Create\](https://discuss.elastic.co/t/java-illegalstateexception-for-pipelineaction-create-main/351142)
**Author:** [@bp\_cs](https://discuss.elastic.co/u/bp_cs)\ **Replies:** 1\ **Last updated:** [January 16, 2024, 9:58am UTC](https://discuss.elastic.co/t/java-illegalstateexception-for-pipelineaction-create-main/351142 "2024-01-16T09:58:30Z")
Thread.exclusive is deprecated, use Thread::Mutex Sending Logstash logs to D:/code/logstash/logstash-7.4.2/logs which is now configured via log4j2.properties \[2024-01-16T16:51:36,256\]\[WARN \]\[logstash.config.source.multil… --- ## [Persistent queue configuration in Windows OS using File IO](https://discuss.elastic.co/t/persistent-queue-configuration-in-windows-os-using-file-io/351145)
**Author:** [@sudipta.s](https://discuss.elastic.co/u/sudipta.s)\ **Replies:** 0\ **Last updated:** [January 16, 2024, 9:56am UTC](https://discuss.elastic.co/t/persistent-queue-configuration-in-windows-os-using-file-io/351145 "2024-01-16T09:56:39Z")
Hello team, We are looking for some support on Persistent queue configuration in windows OS. We are using file IO and exposed some shared location with all write privilege. With guided configuration in elastic documenta… --- ## [Is it possible that with the help of SYSLOG we can push the present log events as well as the past history of events?](https://discuss.elastic.co/t/is-it-possible-that-with-the-help-of-syslog-we-can-push-the-present-log-events-as-well-as-the-past-history-of-events/351123)
**Author:** [@Subrato1](https://discuss.elastic.co/u/Subrato1)\ **Replies:** 0\ **Last updated:** [January 16, 2024, 6:58am UTC](https://discuss.elastic.co/t/is-it-possible-that-with-the-help-of-syslog-we-can-push-the-present-log-events-as-well-as-the-past-history-of-events/351123 "2024-01-16T06:58:28Z")
Any specific configuration required for that?? --- ## [Get error when config "value\_serializer" and "key\_serializer" in output part](https://discuss.elastic.co/t/get-error-when-config-value-serializer-and-key-serializer-in-output-part/351062)
**Author:** [@Pengcheng\_Fu](https://discuss.elastic.co/u/Pengcheng_Fu)\ **Replies:** 1\ **Last updated:** [January 15, 2024, 7:34pm UTC](https://discuss.elastic.co/t/get-error-when-config-value-serializer-and-key-serializer-in-output-part/351062 "2024-01-15T19:34:19Z")
I am testing transfer data between mutile kafka cluster my configuration is below: input { kafka { bootstrap\_servers =\> "10.62.169.206:9092,10.62.220.44:9092,10.62.220.150:9092" topics =\> \["prod-sk… --- ## [The logstash reload config manually not work](https://discuss.elastic.co/t/the-logstash-reload-config-manually-not-work/350895)
**Author:** [@jevonsnotes](https://discuss.elastic.co/u/jevonsnotes)\ **Replies:** 4\ **Last updated:** [January 15, 2024, 1:03am UTC](https://discuss.elastic.co/t/the-logstash-reload-config-manually-not-work/350895 "2024-01-15T01:03:54Z")
as the topic, i send the kill -SIGHUP xxx to the logstash ,but the config still same. version 8.11.3 linux: Linux CS-gxxt-tyzj-03 4.19.90-52.22.v2207.ky10.aarch64 #1 SMP Tue Mar 14 11:52:45 CST 2023 aarch64 aarch64 aar… --- ## [Panw.panos TCP grok errors](https://discuss.elastic.co/t/panw-panos-tcp-grok-errors/350993)
**Author:** [@CodeMonky](https://discuss.elastic.co/u/CodeMonky)\ **Replies:** 0\ **Last updated:** [January 12, 2024, 10:03pm UTC](https://discuss.elastic.co/t/panw-panos-tcp-grok-errors/350993 "2024-01-12T22:03:55Z")
Good day all. I have a question about the Palo Alto Next-Gen Firewall integration. It has two input types, TCP and UDP. We have a client that wanted to move from the UDP to the TCP/SSL connection for security, so we did… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=43) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=45)