# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=55 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 56 --- ## [How to read logs from different docker container using logstash](https://discuss.elastic.co/t/how-to-read-logs-from-different-docker-container-using-logstash/345250)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 2\ **Last updated:** [October 19, 2023, 10:34am UTC](https://discuss.elastic.co/t/how-to-read-logs-from-different-docker-container-using-logstash/345250 "2023-10-19T10:34:27Z")
I have multiple docker containers in host. For example :- tomcat process docker container, elasticsearch process docker container, postgresql process docker container. And Logstash are running in seperate docker cont… --- ## [Logstash fetching data from multiple MySQL databases](https://discuss.elastic.co/t/logstash-fetching-data-from-multiple-mysql-databases/345350)
**Author:** [@mfrob](https://discuss.elastic.co/u/mfrob)\ **Replies:** 0\ **Last updated:** [October 19, 2023, 3:48am UTC](https://discuss.elastic.co/t/logstash-fetching-data-from-multiple-mysql-databases/345350 "2023-10-19T03:48:20Z")
Hi there, I think this community is really great and helpful, especially for someone like me who is really new to the ELK stack, barely a week in. So Ive been tasked to use elasticsearch for a new company project. In or… --- ## [Ingest Pipelines in Logstash](https://discuss.elastic.co/t/ingest-pipelines-in-logstash/345316)
**Author:** [@lucasyuki](https://discuss.elastic.co/u/lucasyuki)\ **Replies:** 1\ **Last updated:** [October 18, 2023, 6:28pm UTC](https://discuss.elastic.co/t/ingest-pipelines-in-logstash/345316 "2023-10-18T18:28:32Z")
Hi, In the ingestion pipelines tab that I created and tested the dissect , how can I index it since it changes daily, in the index file I can see that the pipeline is as default, how can I change this default to the pip… --- ## [LogStash not processing log as specified in pipeline.conf](https://discuss.elastic.co/t/logstash-not-processing-log-as-specified-in-pipeline-conf/345266)
**Author:** [@AmnonH](https://discuss.elastic.co/u/AmnonH)\ **Replies:** 3\ **Last updated:** [October 18, 2023, 2:22pm UTC](https://discuss.elastic.co/t/logstash-not-processing-log-as-specified-in-pipeline-conf/345266 "2023-10-18T14:22:27Z")
I am using a pipeline.conf file to start LS It looks like this: input { file { path ==\> "C:/Elastic-stack/logstash/event-data/apache\_access.log" } } output { stdout { codec ==\> rubydebug } } However, … --- ## [Logstash - using jdbc input plugin as input. And pipiline delay as 30 seconds and batch size as 1000. Still input plugin reads all data from database doesnt get impacted by pipeline configuration of size and delay](https://discuss.elastic.co/t/logstash-using-jdbc-input-plugin-as-input-and-pipiline-delay-as-30-seconds-and-batch-size-as-1000-still-input-plugin-reads-all-data-from-database-doesnt-get-impacted-by-pipeline-configuration-of-size-and-delay/344480)
**Author:** [@Lovin\_Saini](https://discuss.elastic.co/u/Lovin_Saini)\ **Replies:** 4\ **Last updated:** [October 18, 2023, 1:56pm UTC](https://discuss.elastic.co/t/logstash-using-jdbc-input-plugin-as-input-and-pipiline-delay-as-30-seconds-and-batch-size-as-1000-still-input-plugin-reads-all-data-from-database-doesnt-get-impacted-by-pipeline-configuration-of-size-and-delay/344480 "2023-10-18T13:56:51Z")
JDBC input plugin input { jdbc { jdbc\_driver\_library =\> "/usr/share/logstash/driver/mysql-connector-java-8.0.32.jar" jdbc\_driver\_class =\> "com.mysql.cj.jdbc.Driver" jdbc\_connection\_stri… --- ## [Logstash not ready, when output goes down and then comes up during certificate renewal](https://discuss.elastic.co/t/logstash-not-ready-when-output-goes-down-and-then-comes-up-during-certificate-renewal/345269)
**Author:** [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\ **Replies:** 2\ **Last updated:** [October 18, 2023, 12:14pm UTC](https://discuss.elastic.co/t/logstash-not-ready-when-output-goes-down-and-then-comes-up-during-certificate-renewal/345269 "2023-10-18T12:14:55Z")
I was running a test scenario where I was using TTL period 15 mins. After first 15 mins, certificate got renewed, then I scaled down elasticsearch to zero and waited for another 15 mins, it showed that elasticsearch Host… --- ## [Plugin syslog output](https://discuss.elastic.co/t/plugin-syslog-output/345282)
**Author:** [@Manal\_A](https://discuss.elastic.co/u/Manal_A)\ **Replies:** 1\ **Last updated:** [October 18, 2023, 12:11pm UTC](https://discuss.elastic.co/t/plugin-syslog-output/345282 "2023-10-18T12:11:26Z")
I successfully installed the syslog output plugin on my Red Hat virtual machine. However, when I attempt to start Logstash and use the syslog output in my Logstash configuration file, I encounter the following error mess… --- ## [Logstash is restarting when trying to reload certificate](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100)
**Author:** [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\ **Replies:** 8\ **Last updated:** [October 18, 2023, 10:06am UTC](https://discuss.elastic.co/t/logstash-is-restarting-when-trying-to-reload-certificate/345100 "2023-10-18T10:06:18Z")
I am trying to reduce the TTL period to 10 mins and then after 8 mins, the certificate should reload but it does not happens, then it fails and restarts the Logstash, this process restart also fails and the complete pod … --- ## [Index Pattern](https://discuss.elastic.co/t/index-pattern/345214)
**Author:** [@Suleman\_Ahmed](https://discuss.elastic.co/u/Suleman_Ahmed)\ **Replies:** 2\ **Last updated:** [October 18, 2023, 5:40am UTC](https://discuss.elastic.co/t/index-pattern/345214 "2023-10-18T05:40:14Z")
hello! I am running ELK on my kubernetes cluster and unable to create index pattern. Any help will be highly appreciated. My logstash yaml is as under:- apiVersion: v1 kind: ConfigMap metadata: name: logstash-config … --- ## [Logstash-plugin kv useragent question](https://discuss.elastic.co/t/logstash-plugin-kv-useragent-question/345182)
**Author:** [@Mick1](https://discuss.elastic.co/u/Mick1)\ **Replies:** 2\ **Last updated:** [October 18, 2023, 12:53am UTC](https://discuss.elastic.co/t/logstash-plugin-kv-useragent-question/345182 "2023-10-18T00:53:47Z")
Dear ELK technical experts I have a basic logstash kv question to ask, experts please help. message data1:abcde,;,useragent:Mozilla/5.0 (X11; Linux x86\_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Sa… --- ## [On-prem to Cloud Via Logstash](https://discuss.elastic.co/t/on-prem-to-cloud-via-logstash/345135)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 9\ **Last updated:** [October 17, 2023, 9:01pm UTC](https://discuss.elastic.co/t/on-prem-to-cloud-via-logstash/345135 "2023-10-17T21:01:55Z")
I am moving my logs from an on-prem instance, up to Elastic Cloud. I have setup a logstash pipeline, with a very large persistent queue, like below to push events up to the Cloud. input { elasticsearch { hosts =\> … --- ## [Badly formatted index, after interpolation still contains placeholder](https://discuss.elastic.co/t/badly-formatted-index-after-interpolation-still-contains-placeholder/345205)
**Author:** [@John\_paul](https://discuss.elastic.co/u/John_paul)\ **Replies:** 14\ **Last updated:** [October 17, 2023, 5:01pm UTC](https://discuss.elastic.co/t/badly-formatted-index-after-interpolation-still-contains-placeholder/345205 "2023-10-17T17:01:21Z")
Hey guys, facing this below issue, was working fine while using indexes directly but started failing when configured to use data streams, ERROR: elasticsearch - Badly formatted index, after interpolation still contains … --- ## [Logstash jdbc plugin with MSSQL](https://discuss.elastic.co/t/logstash-jdbc-plugin-with-mssql/344912)
**Author:** [@catalin.bulancea](https://discuss.elastic.co/u/catalin.bulancea)\ **Replies:** 11\ **Last updated:** [October 17, 2023, 3:58pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-with-mssql/344912 "2023-10-17T15:58:05Z")
Hi Logstash gurus, I am trying to fetch data from MSSQL using the Logstash jdbc plugin. I am on logstash 7.17.4. I am using the below config for the input: input { jdbc { jdbc\_connection\_string =\> "jdbc:sqlserve… --- ## [Logstash doesn't parse new files in directory](https://discuss.elastic.co/t/logstash-doesnt-parse-new-files-in-directory/345197)
**Author:** [@Xhar](https://discuss.elastic.co/u/Xhar)\ **Replies:** 4\ **Last updated:** [October 17, 2023, 11:13am UTC](https://discuss.elastic.co/t/logstash-doesnt-parse-new-files-in-directory/345197 "2023-10-17T11:13:16Z")
Logstash doesn't parse new logs files in directory Here's my config input { file{ path =\> "/home/user/Documents/bdu/\*.json" sincedb\_path =\> "/dev/null" type =\> "bdu" codec =\> "json" } } output { … --- ## [Logstash / elastic-agent how to create rule on events emitted rate](https://discuss.elastic.co/t/logstash-elastic-agent-how-to-create-rule-on-events-emitted-rate/345173)
**Author:** [@antoine\_duriez](https://discuss.elastic.co/u/antoine_duriez)\ **Replies:** 0\ **Last updated:** [October 17, 2023, 8:01am UTC](https://discuss.elastic.co/t/logstash-elastic-agent-how-to-create-rule-on-events-emitted-rate/345173 "2023-10-17T08:01:10Z")
Hello community, All data collected by my elastic agents (\>4000) is processed by a pair of logstashes. In the Stack Monitoring dashboard I can see the pipeline and the number of events emitted. I would like to know wh… --- ## [Relp error: Relp::InappropriateCommand open expecting syslog](https://discuss.elastic.co/t/relp-error-relp-inappropriatecommand-open-expecting-syslog/345125)
**Author:** [@MarcoV](https://discuss.elastic.co/u/MarcoV)\ **Replies:** 2\ **Last updated:** [October 17, 2023, 7:52am UTC](https://discuss.elastic.co/t/relp-error-relp-inappropriatecommand-open-expecting-syslog/345125 "2023-10-17T07:52:35Z")
Hello everyone. I have this warning in my logstash log: Relp error: Relp::InappropriateCommand open expecting syslog My logstash configuration filter has input relp as input but with this error the log from syslog are… --- ## [Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down?](https://discuss.elastic.co/t/attempted-to-send-a-bulk-request-to-elasticsearch-but-no-there-are-no-living-connections-in-the-connection-pool-perhaps-elasticsearch-is-unreachable-or-down/345042)
**Author:** [@Yazid\_Abed\_Alqader](https://discuss.elastic.co/u/Yazid_Abed_Alqader)\ **Replies:** 3\ **Last updated:** [October 16, 2023, 12:25pm UTC](https://discuss.elastic.co/t/attempted-to-send-a-bulk-request-to-elasticsearch-but-no-there-are-no-living-connections-in-the-connection-pool-perhaps-elasticsearch-is-unreachable-or-down/345042 "2023-10-16T12:25:35Z")
Hi I have these error messages in logstash logs: \[2023-10-15T08:01:31,446\]\[WARN \]\[logstash.outputs.elasticsearch\] Marking url as dead. Last error: \[LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableErr… --- ## [Grok issues / Fingerprint issues: Value not imported into ES after 6.x - 7-x update](https://discuss.elastic.co/t/grok-issues-fingerprint-issues-value-not-imported-into-es-after-6-x-7-x-update/344357)
**Author:** [@randomnamegenerator](https://discuss.elastic.co/u/randomnamegenerator)\ **Replies:** 9\ **Last updated:** [October 16, 2023, 8:48am UTC](https://discuss.elastic.co/t/grok-issues-fingerprint-issues-value-not-imported-into-es-after-6-x-7-x-update/344357 "2023-10-16T08:48:05Z")
Hello, I am new to ELK stack especially the filtering / Grok in Logstash, We are having issues importing DATA:servicename value into ES after moving from 6.3.X to a 7.14 version. The grok below is part of our applica… --- ## [Fingerprinting source with Elastic Agent](https://discuss.elastic.co/t/fingerprinting-source-with-elastic-agent/345054)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 2\ **Last updated:** [October 16, 2023, 12:05am UTC](https://discuss.elastic.co/t/fingerprinting-source-with-elastic-agent/345054 "2023-10-16T00:05:27Z")
I am ingesting logs into Elastic Cloud using an Elastic Agent. The agent sends logs to a logstash instance where I do some custom enrichment and then it goes to the cloud to be processed by an Elastic ingest pipeline. I… --- ## [Logstash JDBC unable to run multiple statements and ingest data from different tables](https://discuss.elastic.co/t/logstash-jdbc-unable-to-run-multiple-statements-and-ingest-data-from-different-tables/345015)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 7\ **Last updated:** [October 15, 2023, 5:19pm UTC](https://discuss.elastic.co/t/logstash-jdbc-unable-to-run-multiple-statements-and-ingest-data-from-different-tables/345015 "2023-10-15T17:19:10Z")
Hi, I'm running Logstash version 8.10.2 in a Docker container to ingest data from a MySQL DB into Elastic. I don't know why the first statement is executed but the second one does not execute. Below is my logstash con… --- ## [Logstash Cloudwatch plugin error for bringing logs into Elastic](https://discuss.elastic.co/t/logstash-cloudwatch-plugin-error-for-bringing-logs-into-elastic/345037)
**Author:** [@uprashan](https://discuss.elastic.co/u/uprashan)\ **Replies:** 0\ **Last updated:** [October 15, 2023, 2:10am UTC](https://discuss.elastic.co/t/logstash-cloudwatch-plugin-error-for-bringing-logs-into-elastic/345037 "2023-10-15T02:10:51Z")
Hi all. I'm running into the Cloudwatch plugin error for Logstash when trying to bring logs (non metrics) as a stream into Logstash.. I'm running the 8.5.3 version of Logstash... Earlier cloudwatch\_logs plugin doesn't se… --- ## [Kafka should include ip](https://discuss.elastic.co/t/kafka-should-include-ip/345029)
**Author:** [@Keremcan\_Seker](https://discuss.elastic.co/u/Keremcan_Seker)\ **Replies:** 0\ **Last updated:** [October 14, 2023, 2:42pm UTC](https://discuss.elastic.co/t/kafka-should-include-ip/345029 "2023-10-14T14:42:38Z")
my logstash has 2 pipelines one is listening for http request and other one is listening kafka The data coming from http pipeline includes host:{ip} and url in \_source section however the data from kafka does not have … --- ## [Logstash - systemd-journald suppressed mensagens](https://discuss.elastic.co/t/logstash-systemd-journald-suppressed-mensagens/344080)
**Author:** [@SilasMuniz1](https://discuss.elastic.co/u/SilasMuniz1)\ **Replies:** 8\ **Last updated:** [October 14, 2023, 2:10pm UTC](https://discuss.elastic.co/t/logstash-systemd-journald-suppressed-mensagens/344080 "2023-10-14T14:10:38Z")
Hi, Always when I restarting my logstash I see this message below: Nowadays I have CPU problem and I am trying fix it. I saw a person in the forum talk about change RateLimitBurst parameter into /etc/systemd/journa… --- ## [Join Id and Name](https://discuss.elastic.co/t/join-id-and-name/344898)
**Author:** [@Samuele\_Lolli](https://discuss.elastic.co/u/Samuele_Lolli)\ **Replies:** 2\ **Last updated:** [October 13, 2023, 8:24am UTC](https://discuss.elastic.co/t/join-id-and-name/344898 "2023-10-13T08:24:04Z")
Hi everyone, im parsing with logstash some message containing an ID refering to an user, i need to add the name of the user with the specific ID. I have all the User and ID in a CSV file. Which one is the best way to a… --- ## [Facing issuse while running logstash of ELK version 8.10](https://discuss.elastic.co/t/facing-issuse-while-running-logstash-of-elk-version-8-10/344968)
**Author:** [@sandraimmaculate](https://discuss.elastic.co/u/sandraimmaculate)\ **Replies:** 0\ **Last updated:** [October 13, 2023, 6:55am UTC](https://discuss.elastic.co/t/facing-issuse-while-running-logstash-of-elk-version-8-10/344968 "2023-10-13T06:55:40Z")
Hi, i have installed elk in AWS instance with AMI Ubuntu 20.04 and hardware requirement 2vpcu, 4gb ram. i have created a Logstash configuration file like and created a log file in which contain the access logs when … --- ## [Error with Logstash on Docker](https://discuss.elastic.co/t/error-with-logstash-on-docker/344918)
**Author:** [@Samuele\_Lolli](https://discuss.elastic.co/u/Samuele_Lolli)\ **Replies:** 0\ **Last updated:** [October 12, 2023, 12:54pm UTC](https://discuss.elastic.co/t/error-with-logstash-on-docker/344918 "2023-10-12T12:54:17Z")
Hi everyone, im trying to use logstash with docker but the pipeline doesn't work. I'm using the same configuration that in logstash without docker work fine. I really need help because im stuck. I posted the log and th… --- ## [Logstash Service Restart continuously](https://discuss.elastic.co/t/logstash-service-restart-continuously/344736)
**Author:** [@Kamesh\_Pratapa](https://discuss.elastic.co/u/Kamesh_Pratapa)\ **Replies:** 9\ **Last updated:** [October 12, 2023, 12:45pm UTC](https://discuss.elastic.co/t/logstash-service-restart-continuously/344736 "2023-10-12T12:45:58Z")
Hi, All of a sudden my logstash service is restarting every 3 minutes and getting the below error \[FATAL\]\[org.logstash.Logstash \] Logstash stopped processing because of an error: (LoadError) Could not load FFI Prov… --- ## [I receive this error when trying to send index to elastic output](https://discuss.elastic.co/t/i-receive-this-error-when-trying-to-send-index-to-elastic-output/344491)
**Author:** [@alex\_base](https://discuss.elastic.co/u/alex_base)\ **Replies:** 2\ **Last updated:** [October 12, 2023, 12:43pm UTC](https://discuss.elastic.co/t/i-receive-this-error-when-trying-to-send-index-to-elastic-output/344491 "2023-10-12T12:43:20Z")
\[INFO \]\[logstash.agent \] Pipelines running {:count=\>1, :running\_pipelines=\>\[:exec\_result\], :non\_running\_pipelines=\>\[\]} \[2023-10-05T13:36:37,359\]\[ERROR\]\[logstash.javapipeline \]\[exec\_result\] Pipeline worker er… --- ## [How to config (disable) the JVM DNS caching in logstash](https://discuss.elastic.co/t/how-to-config-disable-the-jvm-dns-caching-in-logstash/344868)
**Author:** [@picadar](https://discuss.elastic.co/u/picadar)\ **Replies:** 1\ **Last updated:** [October 12, 2023, 5:47am UTC](https://discuss.elastic.co/t/how-to-config-disable-the-jvm-dns-caching-in-logstash/344868 "2023-10-12T05:47:35Z")
Hello everyone！ As title, how to config the JVM DNS caching in logstash？ I have already tried to config it from the environment variable LS\_JAVA\_OPTS, and set the value to -Dnetworkaddress.cache.ttl=N, but there is no … --- ## [Logstash daylight saving time issue](https://discuss.elastic.co/t/logstash-daylight-saving-time-issue/344846)
**Author:** [@fosuna](https://discuss.elastic.co/u/fosuna)\ **Replies:** 0\ **Last updated:** [October 11, 2023, 7:07pm UTC](https://discuss.elastic.co/t/logstash-daylight-saving-time-issue/344846 "2023-10-11T19:07:41Z")
So I'm having the exact same issue @Mahdi\_Davoodi was having here: Logstash date timezone but with the America/Mexico\_City timezone since Mexico stopped observing daylight saving time. Same behavior, the tzdata is the l… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=54) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=56)