# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=202 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 203 --- ## [Jar file version for logstash jdbc input](https://discuss.elastic.co/t/jar-file-version-for-logstash-jdbc-input/281353)
**Author:** [@6igwig](https://discuss.elastic.co/u/6igwig)\ **Replies:** 1\ **Last updated:** [August 16, 2021, 5:20am UTC](https://discuss.elastic.co/t/jar-file-version-for-logstash-jdbc-input/281353 "2021-08-16T05:20:29Z")
I am trying to pull in data from a MS SQL Server database via logstash that is hosted on a linux server. It looks like this is a great job for the JDBC input plugin. I am going through the example config posted on the d… --- ## [Understanding elasticsearch certiifcate](https://discuss.elastic.co/t/understanding-elasticsearch-certiifcate/281482)
**Author:** [@naaviin](https://discuss.elastic.co/u/naaviin)\ **Replies:** 1\ **Last updated:** [August 15, 2021, 5:00pm UTC](https://discuss.elastic.co/t/understanding-elasticsearch-certiifcate/281482 "2021-08-15T17:00:31Z")
i saw there 2 type of certificate which is elastic-stack-ca.p12 and elastic-certificates.p12. What are the differences between this 2 certificate https://www.elastic.co/guide/en/elasticsearch/reference/current/security-… --- ## [Tags for security-related encoding issues](https://discuss.elastic.co/t/tags-for-security-related-encoding-issues/281466)
**Author:** [@cknz](https://discuss.elastic.co/u/cknz)\ **Replies:** 0\ **Last updated:** [August 14, 2021, 11:58pm UTC](https://discuss.elastic.co/t/tags-for-security-related-encoding-issues/281466 "2021-08-14T23:58:42Z")
Let's say you have some logstash module or Ruby configuration that cleans up / parses / etc. logs, and it comes across something that would be illegal (Eg. illegal UTF-8 encoding sequence which if it were not sanitized c… --- ## [Parsing an existing field further](https://discuss.elastic.co/t/parsing-an-existing-field-further/281389)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 5\ **Last updated:** [August 13, 2021, 11:53pm UTC](https://discuss.elastic.co/t/parsing-an-existing-field-further/281389 "2021-08-13T23:53:05Z")
Hi, Is there way to parse the data from a single filed into 2 different fields? I have a field that contains a username and an ID that I would like to break out. So I want to assign to keep the username in the usernam… --- ## [Logstash Indexing Questions](https://discuss.elastic.co/t/logstash-indexing-questions/281415)
**Author:** [@timtom935](https://discuss.elastic.co/u/timtom935)\ **Replies:** 1\ **Last updated:** [August 13, 2021, 11:15pm UTC](https://discuss.elastic.co/t/logstash-indexing-questions/281415 "2021-08-13T23:15:23Z")
Hello I have been setting up a proof of concept elastic stack for our company. We had data flowing through the elastic stack without errors when we were just shipping the Winlogbeat data directly to Kibana. All of the W… --- ## [DLQ pipeline not showing fields added in the filter section of the Logstash pipeline in the datastream](https://discuss.elastic.co/t/dlq-pipeline-not-showing-fields-added-in-the-filter-section-of-the-logstash-pipeline-in-the-datastream/281383)
**Author:** [@mguttula](https://discuss.elastic.co/u/mguttula)\ **Replies:** 2\ **Last updated:** [August 13, 2021, 10:02pm UTC](https://discuss.elastic.co/t/dlq-pipeline-not-showing-fields-added-in-the-filter-section-of-the-logstash-pipeline-in-the-datastream/281383 "2021-08-13T22:02:25Z")
Elastic stack 7.13 version. Winlogbeat messages are ingested into Elasticsearch cluster and the dead letter queue is filling up and causing "/var/log/messages" to fill up as well with messages containing "cannot write ev… --- ## [Logstash filter for varying number of disks info in the logs](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 6\ **Last updated:** [August 13, 2021, 9:16pm UTC](https://discuss.elastic.co/t/logstash-filter-for-varying-number-of-disks-info-in-the-logs/281400 "2021-08-13T21:16:38Z")
Hello, i have a couple of log lines which go like this , that have varying number of disks 1628868082.953|7ec93a454940| disk-usage: \[1\]=46% \[2\]=44% \[3\]=52% \[4\]=52% \[5\]=46% \[6\]=40% \[7\]=45% \[8\]=48% 1628868082.953|7ec93a… --- ## [Unexpected error with Beats input plugin](https://discuss.elastic.co/t/unexpected-error-with-beats-input-plugin/281255)
**Author:** [@panagiss](https://discuss.elastic.co/u/panagiss)\ **Replies:** 8\ **Last updated:** [August 13, 2021, 7:10pm UTC](https://discuss.elastic.co/t/unexpected-error-with-beats-input-plugin/281255 "2021-08-13T19:10:40Z")
Previously i used JSON data from a file (that were previously taken by beats) to test my logstash conf and filtering. I had no problems, but when i try to read straight from auditbeat with the Beats input plugin, Logstas… --- ## [Cloudwatch log group stream using logstash to ES](https://discuss.elastic.co/t/cloudwatch-log-group-stream-using-logstash-to-es/281378)
**Author:** [@connectgeeks](https://discuss.elastic.co/u/connectgeeks)\ **Replies:** 1\ **Last updated:** [August 13, 2021, 3:27pm UTC](https://discuss.elastic.co/t/cloudwatch-log-group-stream-using-logstash-to-es/281378 "2021-08-13T15:27:29Z")
Hi I'm using logstash-7.14.0-1.x86\_64 version and would like to stream cloudwatch loggroup to ES. When I'm trying to use plugin bin/logstash-plugin install logstash-input-cloudwatch it doesn't seems to work for log grou… --- ## [Use logstash to synchronize json files to elactissearch without data](https://discuss.elastic.co/t/use-logstash-to-synchronize-json-files-to-elactissearch-without-data/281138)
**Author:** [@zhyp](https://discuss.elastic.co/u/zhyp)\ **Replies:** 3\ **Last updated:** [August 13, 2021, 2:53pm UTC](https://discuss.elastic.co/t/use-logstash-to-synchronize-json-files-to-elactissearch-without-data/281138 "2021-08-13T14:53:30Z")
There is no data, even when the file is changed, the console does not output changes. This is my conf file, test.conf input { file { path =\> "D:\\logstash\\test\\demo.log" codec =\> "json" } } output { elasticsearc… --- ## [Характеристики виртуальной машины для logstash](https://discuss.elastic.co/t/logstash/281348)
**Author:** [@111498](https://discuss.elastic.co/u/111498)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 12:12pm UTC](https://discuss.elastic.co/t/logstash/281348 "2021-08-13T12:12:38Z")
Нужна виртуальная машина для logstash. Также будет использоваться плагин lumberjack. Сколько CPU, RAM, и диска надо для этого, если учитывать что: EPS=1000, Bandwidth usage=400kb/s PPS=15 ? --- ## [Logstash import file from url](https://discuss.elastic.co/t/logstash-import-file-from-url/281067)
**Author:** [@ChinigamiHunter](https://discuss.elastic.co/u/ChinigamiHunter)\ **Replies:** 8\ **Last updated:** [August 13, 2021, 12:10pm UTC](https://discuss.elastic.co/t/logstash-import-file-from-url/281067 "2021-08-13T12:10:22Z")
I am new to Elasticsearch and Logstash, i just need to parse json data from url. Json link is : https://www.circl.lu/doc/misp/feed-osint/0165e5d7-51e6-4c2e-a382-1dd1e706f7bb.json this is my logstash conf file: input {… --- ## [Logstash 7.14.0 "invalid byte sequence in UTF-8" in logstash.javapipeline](https://discuss.elastic.co/t/logstash-7-14-0-invalid-byte-sequence-in-utf-8-in-logstash-javapipeline/280976)
**Author:** [@cknz](https://discuss.elastic.co/u/cknz)\ **Replies:** 4\ **Last updated:** [August 13, 2021, 10:56am UTC](https://discuss.elastic.co/t/logstash-7-14-0-invalid-byte-sequence-in-utf-8-in-logstash-javapipeline/280976 "2021-08-13T10:56:25Z")
Came into work today and noticed that our Logstash pipeline (on all two hosts) had halted with the following error in the logs. Aug 11 00:22:40 its-elk-p03.uod.otago.ac.nz logstash\[20536\]: \[2021-08-11T00:22:40,141\]\[ERRO… --- ## [Problems connecting logstash-input-mongodb](https://discuss.elastic.co/t/problems-connecting-logstash-input-mongodb/281186)
**Author:** [@LourdesCrespo](https://discuss.elastic.co/u/LourdesCrespo)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 10:18am UTC](https://discuss.elastic.co/t/problems-connecting-logstash-input-mongodb/281186 "2021-08-12T10:18:43Z")
In the project it is intended that the logs obtained from logstash are sent to mongodb. Docker-compose is used for this and although the containers are connected, the logs are not sent. The part of docker-compose that co… --- ## [Logstash multiline codec ignores last line of log file](https://discuss.elastic.co/t/logstash-multiline-codec-ignores-last-line-of-log-file/281318)
**Author:** [@san2597](https://discuss.elastic.co/u/san2597)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 8:19am UTC](https://discuss.elastic.co/t/logstash-multiline-codec-ignores-last-line-of-log-file/281318 "2021-08-13T08:19:51Z")
My logstash config has a mix of multiline and single line events that are being parsed correctly, reading a log file. I'm using multiline codec as follows: input { file { ..... ..... codec =\> multiline {… --- ## [Logstash does not consume Kafka](https://discuss.elastic.co/t/logstash-does-not-consume-kafka/281278)
**Author:** [@ktpktr0](https://discuss.elastic.co/u/ktpktr0)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 2:47am UTC](https://discuss.elastic.co/t/logstash-does-not-consume-kafka/281278 "2021-08-13T02:47:32Z")
I created 3 logstash + 3 Kafka, and the initial size of each topic is 1. After a log grows greatly, the Kafka disk reaches 100%. After the disk is expanded, logstash does not consume Kafka. Only when Kafka is restarted c… --- ## [Probem with file imput](https://discuss.elastic.co/t/probem-with-file-imput/281261)
**Author:** [@KarlWolf](https://discuss.elastic.co/u/KarlWolf)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 8:46pm UTC](https://discuss.elastic.co/t/probem-with-file-imput/281261 "2021-08-12T20:46:29Z")
Hello everyone, I'm having a strange behaviour with my Logstash. I'm stucked on the following problem: I had a one flow configured where I had file input and that worked brilliant. Then I added to my flow udp input...… --- ## [Logstash.bat immediately errors out with \\Java\\jdk-11.0.12 was unexpected at this time](https://discuss.elastic.co/t/logstash-bat-immediately-errors-out-with-java-jdk-11-0-12-was-unexpected-at-this-time/281264)
**Author:** [@rjcpayne699](https://discuss.elastic.co/u/rjcpayne699)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 8:41pm UTC](https://discuss.elastic.co/t/logstash-bat-immediately-errors-out-with-java-jdk-11-0-12-was-unexpected-at-this-time/281264 "2021-08-12T20:41:55Z")
Just trying my first local run of logstash after installation on windows 10. I performed the first steps to validate jvm: PS C:\\logstash-7.14.0\> Write-Host $env:JAVA\_HOME C:\\Program Files (x86)\\Java\\jdk-11.0.12 PS C:\\… --- ## [Grok works on Grok debugger but got Grokparsefailure in logstash](https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227)
**Author:** [@syunusic](https://discuss.elastic.co/u/syunusic)\ **Replies:** 5\ **Last updated:** [August 12, 2021, 6:55pm UTC](https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227 "2021-08-12T18:55:34Z")
If I go to http://grokdebug.herokuapp.com/ or the GROK debugger in Kibana and use this sample and this grok, works fine.. but if I run it in logstash, I get a grokfailure tag: Sample: NSX 2281528 - \[nsx@6876 comp=\\"nsx… --- ## [Unable to Start Logstash with Elasticsearch - "Attempted to resurrect connection to dead ES instance"](https://discuss.elastic.co/t/unable-to-start-logstash-with-elasticsearch-attempted-to-resurrect-connection-to-dead-es-instance/281241)
**Author:** [@jdurga](https://discuss.elastic.co/u/jdurga)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 5:46pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-with-elasticsearch-attempted-to-resurrect-connection-to-dead-es-instance/281241 "2021-08-12T17:46:55Z")
I've followed these steps: https://github.com/virtualtechmeetup/0003\_howto\_elasticsearch\_kibana\_install\_ubuntu I would like to send Syslog traffic from Logstash to Datadog and Elasticsearch. Really just using Elasticse… --- ## [Logstash "if statement" not working for logs from winlogbeat 7.13.2 (works on older version)](https://discuss.elastic.co/t/logstash-if-statement-not-working-for-logs-from-winlogbeat-7-13-2-works-on-older-version/281179)
**Author:** [@heikis](https://discuss.elastic.co/u/heikis)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 9:57am UTC](https://discuss.elastic.co/t/logstash-if-statement-not-working-for-logs-from-winlogbeat-7-13-2-works-on-older-version/281179 "2021-08-12T09:57:05Z")
Hello. I have a weird problem with Logstash filtering. I am sending logs from several different winlogbeat agents to logstash for processing. I have winlogbeats agents with different versions- 7.9.1 and 7.13.2. Logsta… --- ## [Logstash setup with Docker - \[exits after ~25s\]\[not able to find the problem\]](https://discuss.elastic.co/t/logstash-setup-with-docker-exits-after-25s-not-able-to-find-the-problem/281211)
**Author:** [@Jno\_Zrc](https://discuss.elastic.co/u/Jno_Zrc)\ **Replies:** 2\ **Last updated:** [August 12, 2021, 5:01pm UTC](https://discuss.elastic.co/t/logstash-setup-with-docker-exits-after-25s-not-able-to-find-the-problem/281211 "2021-08-12T17:01:09Z")
Hi all, I'm trying to insert a large csv file in Elasticsearch using Logstash. I manage to make it work, but after a while testing some different mutate transformations it stopped working. I even tried a minimal confi… --- ## [Transport Error Unauthorized](https://discuss.elastic.co/t/transport-error-unauthorized/280476)
**Author:** [@agentw](https://discuss.elastic.co/u/agentw)\ **Replies:** 11\ **Last updated:** [August 12, 2021, 4:33pm UTC](https://discuss.elastic.co/t/transport-error-unauthorized/280476 "2021-08-12T16:33:31Z")
I upgraded our cluster to 7.14, and after the update for Logstash I received "Pipeline error {:pipeline\_id=\>"main", :exception=\>#\ **Author:** [@dimitris\_sb](https://discuss.elastic.co/u/dimitris_sb)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 4:13pm UTC](https://discuss.elastic.co/t/logstash-degrading-after-a-few-days-and-restart-solves-the-issue-temporarily/281231 "2021-08-12T16:13:07Z") Hi All, We are using a cluster of two logstash instances (7.13.4) to ingest data to a three-node cluster (7.13.4). We noticed that after a few days the ingest rate falls from 30K - 50K to less than 30K and the data are … --- ## [Découpage avec split ne fonctionne pas](https://discuss.elastic.co/t/decoupage-avec-split-ne-fonctionne-pas/281225)
**Author:** [@Christophe\_Dumont](https://discuss.elastic.co/u/Christophe_Dumont)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 3:46pm UTC](https://discuss.elastic.co/t/decoupage-avec-split-ne-fonctionne-pas/281225 "2021-08-12T15:46:50Z")
Bonjour, Je souhaite indexer des paquets réseau et en extraire certaines informations mais logstash envoi tout dans un champ unique. Voici le code logstash : input { kafka { bootstrap\_servers =\> "172.16.238.204:… --- ## [How should I filter this date?](https://discuss.elastic.co/t/how-should-i-filter-this-date/281185)
**Author:** [@AbelAlejandro](https://discuss.elastic.co/u/AbelAlejandro)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 3:16pm UTC](https://discuss.elastic.co/t/how-should-i-filter-this-date/281185 "2021-08-12T15:16:50Z")
My logs are TSVs that look like the following: |timestamp\_fmt|timestamp\_ms|key|tictoc\_ms|mem\_jvm|mem\_phy\_os|tags|comment| |---|---|---|---|---|---|---|---| |2021-08-11T13-55-39.202|1628682939202|SCENARIO|TIC|4548715304|… --- ## [Kafka to logstash , with dynamic topic creation](https://discuss.elastic.co/t/kafka-to-logstash-with-dynamic-topic-creation/281181)
**Author:** [@navox\_nadhir](https://discuss.elastic.co/u/navox_nadhir)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 3:16pm UTC](https://discuss.elastic.co/t/kafka-to-logstash-with-dynamic-topic-creation/281181 "2021-08-12T15:16:43Z")
I have a question ; i need to know if logstash can handle the creation of topics kafka dynamically basing on feilds ! as un output ! the pipeline is kafka input --\> logstash --\> kafka output the use case is kubernet… --- ## [File Input codec =\> "json\_lines" won't work](https://discuss.elastic.co/t/file-input-codec-json-lines-wont-work/281196)
**Author:** [@panagiss](https://discuss.elastic.co/u/panagiss)\ **Replies:** 2\ **Last updated:** [August 12, 2021, 3:05pm UTC](https://discuss.elastic.co/t/file-input-codec-json-lines-wont-work/281196 "2021-08-12T15:05:26Z")
I try to read from a json file but codec=\>"json\_lines" won't work, can't see any output. But when i try and set codec=\>"json" it works and parses the data and i see the output. So you ask me what is the format of the js… --- ## [What are the advantages and the disadvantages of running multiple instances of logstash in the same cluster?](https://discuss.elastic.co/t/what-are-the-advantages-and-the-disadvantages-of-running-multiple-instances-of-logstash-in-the-same-cluster/281087)
**Author:** [@xemyleex](https://discuss.elastic.co/u/xemyleex)\ **Replies:** 3\ **Last updated:** [August 12, 2021, 2:13pm UTC](https://discuss.elastic.co/t/what-are-the-advantages-and-the-disadvantages-of-running-multiple-instances-of-logstash-in-the-same-cluster/281087 "2021-08-12T14:13:21Z")
What are the advantages and the disadvantages of running multiple instances of logstash in the same cluster? --- ## [How to define what data is sent to Elastic from Logstash?](https://discuss.elastic.co/t/how-to-define-what-data-is-sent-to-elastic-from-logstash/279995)
**Author:** [@profsteve](https://discuss.elastic.co/u/profsteve)\ **Replies:** 8\ **Last updated:** [August 12, 2021, 1:37pm UTC](https://discuss.elastic.co/t/how-to-define-what-data-is-sent-to-elastic-from-logstash/279995 "2021-08-12T13:37:12Z")
Hello, I have ELK running on Ubuntu Server with the intention of it receiving Syslog from firewalls and Filebeat from DNS/DHCP/AD services. The primary issue is the firewalls make a lot of Syslog noise without there be… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=201) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=203)