# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=265 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 266 --- ## [java.lang.OutOfMemoryError: Java heap space](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/259773)
**Author:** [@rknd](https://discuss.elastic.co/u/rknd)\ **Replies:** 3\ **Last updated:** [January 9, 2021, 8:18am UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space/259773 "2021-01-09T08:18:36Z")
Hello everyone I've got this error. java.lang.OutOfMemoryError: Java heap space When I run my pipeline. I set jvm option the below. But still getting this jvm error. -Xms8M -Xmx8M How can i solve ? Can someone expl… --- ## [Field.keyword empty in Discover](https://discuss.elastic.co/t/field-keyword-empty-in-discover/260534)
**Author:** [@HerveSavard](https://discuss.elastic.co/u/HerveSavard)\ **Replies:** 1\ **Last updated:** [January 11, 2021, 6:52am UTC](https://discuss.elastic.co/t/field-keyword-empty-in-discover/260534 "2021-01-11T06:52:56Z")
Hello All, In Discover, i see all my value do i need. sysvalval 2 When i add a filter like sysvalval.keyword the list of value is empty. I use Kibana version v7.10 Many thanks in advance --- ## [Logstash doesnt start after upgrade to 6.8.13 (from 6.2.3)](https://discuss.elastic.co/t/logstash-doesnt-start-after-upgrade-to-6-8-13-from-6-2-3/260599)
**Author:** [@cfu](https://discuss.elastic.co/u/cfu)\ **Replies:** 1\ **Last updated:** [January 11, 2021, 6:39am UTC](https://discuss.elastic.co/t/logstash-doesnt-start-after-upgrade-to-6-8-13-from-6-2-3/260599 "2021-01-11T06:39:26Z")
After upgrading logstash 6.3.2 to 6.8.13 it doesn´t start. Looking into the log file shows: \[2021-01-09T11:24:12,770\]\[DEBUG\]\[logstash.modules.scaffold\] Found module {:module\_name=\>"fb\_apache", :directory=\>"/products/el… --- ## [Set \_type in logstash](https://discuss.elastic.co/t/set-type-in-logstash/260643)
**Author:** [@navin1093](https://discuss.elastic.co/u/navin1093)\ **Replies:** 2\ **Last updated:** [January 11, 2021, 1:21am UTC](https://discuss.elastic.co/t/set-type-in-logstash/260643 "2021-01-11T01:21:57Z")
Hi experts. As per my research index (Elastic Search) -\> database (RDMS) type (Elastic Search) -\> table (RDMS) Can i know how i can set the type? input { beats { port =\> 5044 ... type =\> "%{\[fields\]\[na… --- ## [java.lang.OutOfMemoryError: Java heap space in Logstash](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-in-logstash/260596)
**Author:** [@rknd](https://discuss.elastic.co/u/rknd)\ **Replies:** 1\ **Last updated:** [January 11, 2021, 12:13am UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-in-logstash/260596 "2021-01-11T00:13:43Z")
I have too many configuration files. I run these files together via pipelines.yml. I use these types of settings likewise below; - pipeline.id: pipeline1 path.config: "path\_to\_pipeline1" - pipeline.id: pipeline2 pat… --- ## [Filebeat fields value unable to use it in logstash configuration file](https://discuss.elastic.co/t/filebeat-fields-value-unable-to-use-it-in-logstash-configuration-file/260614)
**Author:** [@navin1093](https://discuss.elastic.co/u/navin1093)\ **Replies:** 8\ **Last updated:** [January 10, 2021, 11:20pm UTC](https://discuss.elastic.co/t/filebeat-fields-value-unable-to-use-it-in-logstash-configuration-file/260614 "2021-01-10T23:20:24Z")
I have set custom name in filebeat input section. However, in logstash configuration file it was unable to be use Filebeat config file - type: log enabled: true paths: ... fields: - name\_of\_index: group-1… --- ## [Scripted\_upsert not working when logtash looses connection to ES and reconnects](https://discuss.elastic.co/t/scripted-upsert-not-working-when-logtash-looses-connection-to-es-and-reconnects/260651)
**Author:** [@Pachidermus](https://discuss.elastic.co/u/Pachidermus)\ **Replies:** 1\ **Last updated:** [January 10, 2021, 6:40pm UTC](https://discuss.elastic.co/t/scripted-upsert-not-working-when-logtash-looses-connection-to-es-and-reconnects/260651 "2021-01-10T18:40:18Z")
I am experiencing the exact same problem as here: The setup is very basic. We are using the ES output plugin with a basic logic: output { if \[@metadata\]\[scripted\_upsert\] == "true" { elasticsearch { ... … --- ## [Logstash high CPU on JDBC plugin](https://discuss.elastic.co/t/logstash-high-cpu-on-jdbc-plugin/260610)
**Author:** [@Kevin\_Juliano](https://discuss.elastic.co/u/Kevin_Juliano)\ **Replies:** 1\ **Last updated:** [January 10, 2021, 2:08am UTC](https://discuss.elastic.co/t/logstash-high-cpu-on-jdbc-plugin/260610 "2021-01-10T02:08:40Z")
Hi! I am running Logstash with JDBC plugin postgreSQL, as I check the performance of our server, because I are having some timeout issues, I found out that Logstash when running is having more than 100% CPU usage. Wh… --- ## [How to sent syslog from mikrotik to ELK](https://discuss.elastic.co/t/how-to-sent-syslog-from-mikrotik-to-elk/260601)
**Author:** [@Zhamax](https://discuss.elastic.co/u/Zhamax)\ **Replies:** 1\ **Last updated:** [January 9, 2021, 3:46pm UTC](https://discuss.elastic.co/t/how-to-sent-syslog-from-mikrotik-to-elk/260601 "2021-01-09T15:46:26Z")
Hi everyone i have some questions to ask. i want to sent syslog from mikrotik to Elastic But I'm confused about how to write code into Input, Filter and output ? Thank you in advance for your help --- ## [Logstash Database pipeline resiliency](https://discuss.elastic.co/t/logstash-database-pipeline-resiliency/260608)
**Author:** [@adityaPsl](https://discuss.elastic.co/u/adityaPsl)\ **Replies:** 3\ **Last updated:** [January 9, 2021, 3:28pm UTC](https://discuss.elastic.co/t/logstash-database-pipeline-resiliency/260608 "2021-01-09T15:28:24Z")
Hello Team, I have a use case where Ineed to deploy database pipeline on 2 nodes, but i want to implement in a way that if one node(machine) goes down then only 2nd will pick up from that point and until the second goes… --- ## [Inconsistent elasticsearch input vs output plugin settings](https://discuss.elastic.co/t/inconsistent-elasticsearch-input-vs-output-plugin-settings/260586)
**Author:** [@GuillaumeN](https://discuss.elastic.co/u/GuillaumeN)\ **Replies:** 0\ **Last updated:** [January 8, 2021, 10:58pm UTC](https://discuss.elastic.co/t/inconsistent-elasticsearch-input-vs-output-plugin-settings/260586 "2021-01-08T22:58:49Z")
Hi, The Logstash Elasticsearch output supports the "ssl\_certificate\_verification" parameter whereas the input one doesn't. This makes it more difficult when consuming data from an Elasticsearch cluster using self-signe… --- ## [Help with installing logstash - no such file to load --logstash/build](https://discuss.elastic.co/t/help-with-installing-logstash-no-such-file-to-load-logstash-build/260572)
**Author:** [@Grant1999](https://discuss.elastic.co/u/Grant1999)\ **Replies:** 4\ **Last updated:** [January 8, 2021, 9:37pm UTC](https://discuss.elastic.co/t/help-with-installing-logstash-no-such-file-to-load-logstash-build/260572 "2021-01-08T21:37:32Z")
When in the logstash folder I have tried to install logstash with the command. logstash -f logstash.conf and get the error no such file to load --logstash/build. I have supplied a screenshot. If anyone could help that wo… --- ## [Logstash 7.9 not writing to output file](https://discuss.elastic.co/t/logstash-7-9-not-writing-to-output-file/260260)
**Author:** [@satish92](https://discuss.elastic.co/u/satish92)\ **Replies:** 2\ **Last updated:** [January 8, 2021, 8:25pm UTC](https://discuss.elastic.co/t/logstash-7-9-not-writing-to-output-file/260260 "2021-01-08T20:25:43Z")
I have logstash 7.9 running in centos hosts. I'm facing a strange behavior. Logstash is not writing to a file in the given path. I want to make sure logstash is working or not so I provided a path in output section but … --- ## [Rabbitmq output plugin : dynamic values on headers not working](https://discuss.elastic.co/t/rabbitmq-output-plugin-dynamic-values-on-headers-not-working/260475)
**Author:** [@MarineW29](https://discuss.elastic.co/u/MarineW29)\ **Replies:** 5\ **Last updated:** [January 8, 2021, 4:25pm UTC](https://discuss.elastic.co/t/rabbitmq-output-plugin-dynamic-values-on-headers-not-working/260475 "2021-01-08T16:25:16Z")
Hello, I have a pipeline with elasticsearch input and rabbitmq output. I would like to set dynamic values to headers in properties of the message published in RabbitMQ, but the field are not replaces with their values. … --- ## [Question on Elastic Search Filtering](https://discuss.elastic.co/t/question-on-elastic-search-filtering/260504)
**Author:** [@Sampson\_Light](https://discuss.elastic.co/u/Sampson_Light)\ **Replies:** 1\ **Last updated:** [January 8, 2021, 3:17pm UTC](https://discuss.elastic.co/t/question-on-elastic-search-filtering/260504 "2021-01-08T15:17:04Z")
Hi, I have duplicates in my system but i intend to do some form of filtering on the logstash side to prevent any future duplicates from occurring (due to server restart incorrectly). For reference, my current duplicate… --- ## [Logstash not parsing metadata from logfile](https://discuss.elastic.co/t/logstash-not-parsing-metadata-from-logfile/260496)
**Author:** [@Rahul3](https://discuss.elastic.co/u/Rahul3)\ **Replies:** 1\ **Last updated:** [January 8, 2021, 3:13pm UTC](https://discuss.elastic.co/t/logstash-not-parsing-metadata-from-logfile/260496 "2021-01-08T15:13:44Z")
Hi, I am very new to ELK Stack, I have a requirement to parse specific metadata from a log entry, Here is a sample log entry, \[ERROR\] \[2021-01-04 14:56:41,566\] \[http-nio-8080-exec-4\] \[com.blocks.bear.server.exception.Ex… --- ## [Why logstash's redis output plugin send list data with \\n?](https://discuss.elastic.co/t/why-logstashs-redis-output-plugin-send-list-data-with-n/260525)
**Author:** [@iooi](https://discuss.elastic.co/u/iooi)\ **Replies:** 1\ **Last updated:** [January 8, 2021, 11:24am UTC](https://discuss.elastic.co/t/why-logstashs-redis-output-plugin-send-list-data-with-n/260525 "2021-01-08T11:24:46Z")
Using Redis output pluginedit to send data to Redis. output { redis { host =\> \["${REDIS\_URL}"\] data\_type =\> "list" key =\> "ID" codec =\> line { format =\> "%{\[@metadata\]\[\_id\]}"} } } Check Redis' data: … --- ## [Logstash parse json file content failed, but ok when enter one line with input plugin](https://discuss.elastic.co/t/logstash-parse-json-file-content-failed-but-ok-when-enter-one-line-with-input-plugin/260503)
**Author:** [@bo\_bo](https://discuss.elastic.co/u/bo_bo)\ **Replies:** 0\ **Last updated:** [January 8, 2021, 7:01am UTC](https://discuss.elastic.co/t/logstash-parse-json-file-content-failed-but-ok-when-enter-one-line-with-input-plugin/260503 "2021-01-08T07:01:20Z")
File content is: {"#account\_id":"43DDBF5E46FBC68F2B9C395DA7C3443B","#distinct\_id":"205aa1ec49b21576da4ee3beb994f132","#type":"track","#ip":"122.157.17 5.52","#time":"2020-01-06 19:44:54","#event\_name":"TALENT\_CAPTURE",… --- ## [Trouble With Multiline Codec Logstash](https://discuss.elastic.co/t/trouble-with-multiline-codec-logstash/260483)
**Author:** [@bigbobolue](https://discuss.elastic.co/u/bigbobolue)\ **Replies:** 1\ **Last updated:** [January 7, 2021, 7:51pm UTC](https://discuss.elastic.co/t/trouble-with-multiline-codec-logstash/260483 "2021-01-07T19:51:55Z")
I am having trouble with a particular multiline log 2021-01-07T07:17:59.942 INFO (129c-2b04) \[LogonMonitor::LogSummary\] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* Session Summary (User: User1234, Session: 1) \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* 2021-01-07T07:17:… --- ## [Pipeline sending to itself](https://discuss.elastic.co/t/pipeline-sending-to-itself/260413)
**Author:** [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\ **Replies:** 3\ **Last updated:** [January 7, 2021, 5:27pm UTC](https://discuss.elastic.co/t/pipeline-sending-to-itself/260413 "2021-01-07T17:27:52Z")
Hello all, We have the following usecase: We want to read data from a service provider that requires paging when polling data(they also support pushing data but we are not accessible from the internet). I found that th… --- ## [Question about logstash output plugin 7.9 and ecs](https://discuss.elastic.co/t/question-about-logstash-output-plugin-7-9-and-ecs/259154)
**Author:** [@gerbdla](https://discuss.elastic.co/u/gerbdla)\ **Replies:** 1\ **Last updated:** [January 7, 2021, 4:53pm UTC](https://discuss.elastic.co/t/question-about-logstash-output-plugin-7-9-and-ecs/259154 "2021-01-07T16:53:14Z")
I am not understanding the ecs changes in the logstash elasticsearch output plugin version 7.9 ecs compatibility. I have an issue when I try to index an event that has an object mapping for host. object mapping for \[ho… --- ## [Logstash fails to run: ES\_TEMPDIR does not get translated properly?](https://discuss.elastic.co/t/logstash-fails-to-run-es-tempdir-does-not-get-translated-properly/260441)
**Author:** [@tarkhil](https://discuss.elastic.co/u/tarkhil)\ **Replies:** 3\ **Last updated:** [January 7, 2021, 4:18pm UTC](https://discuss.elastic.co/t/logstash-fails-to-run-es-tempdir-does-not-get-translated-properly/260441 "2021-01-07T16:18:31Z")
logstash7-7.10.0 openjdk15-15.0.1 Attempt to run logstash fails with # su -m logstash -c 'sh -c "env JAVA\_HOME=/usr/local/openjdk15 /usr/local/logstash/bin/logstash --path.settings /usr/local/etc/logstash -l /var… --- ## [使用logstash7.4.2往es发送数据得用户信息如何创建用户呢](https://discuss.elastic.co/t/logstash7-4-2-es/260411)
**Author:** [@111351](https://discuss.elastic.co/u/111351)\ **Replies:** 0\ **Last updated:** [January 7, 2021, 6:36am UTC](https://discuss.elastic.co/t/logstash7-4-2-es/260411 "2021-01-07T06:36:44Z")
你好我用es里自带得logstsah\_system用户向es发数据不成功请问有人知道是怎么回事吗 --- ## [Logstash trying to write to read-only index endlessly](https://discuss.elastic.co/t/logstash-trying-to-write-to-read-only-index-endlessly/260356)
**Author:** [@alexbde](https://discuss.elastic.co/u/alexbde)\ **Replies:** 3\ **Last updated:** [January 7, 2021, 9:52am UTC](https://discuss.elastic.co/t/logstash-trying-to-write-to-read-only-index-endlessly/260356 "2021-01-07T09:52:04Z")
Hello, we're using BELK-Stack with 5 Filebeats =\> 1 Logstash =\> 1 Elasticsearch \<= 1 Kibana. During the last weeks we experienced some downtime of 2 Filebeat services (root cause doesn't matter) which led to some buffere… --- ## [\[Threat Intelligence\]: Avoid redundancy of information in the same index](https://discuss.elastic.co/t/threat-intelligence-avoid-redundancy-of-information-in-the-same-index/260303)
**Author:** [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\ **Replies:** 4\ **Last updated:** [January 7, 2021, 8:11am UTC](https://discuss.elastic.co/t/threat-intelligence-avoid-redundancy-of-information-in-the-same-index/260303 "2021-01-07T08:11:59Z")
Hello, I am using this pipeline to enrich my SIEM with URLhaus information: input { exec { command =\> 'curl https://urlhaus.abuse.ch/downloads/csv/ --output text.zip && unzip -c text.zip' interval =\> 86400 … --- ## [Logstash: Resource consumption of multiple Logstash servers are not even](https://discuss.elastic.co/t/logstash-resource-consumption-of-multiple-logstash-servers-are-not-even/260406)
**Author:** [@Hung\_Phan\_Huy](https://discuss.elastic.co/u/Hung_Phan_Huy)\ **Replies:** 1\ **Last updated:** [January 7, 2021, 6:55am UTC](https://discuss.elastic.co/t/logstash-resource-consumption-of-multiple-logstash-servers-are-not-even/260406 "2021-01-07T06:55:19Z")
Hi, I'm running my Logstash services on Docker Swarm with 4 replicas. They subscribe to the same Kafka topic and output the data to Elasticsearch. When I inspect the resource consumption of those 4 containers, I found … --- ## [Logstash open http server received message grabled code](https://discuss.elastic.co/t/logstash-open-http-server-received-message-grabled-code/260297)
**Author:** [@111351](https://discuss.elastic.co/u/111351)\ **Replies:** 3\ **Last updated:** [January 7, 2021, 1:15am UTC](https://discuss.elastic.co/t/logstash-open-http-server-received-message-grabled-code/260297 "2021-01-07T01:15:10Z")
I use logstash 7.4.2 start pipeline input{ http{ } } output{ stdout{} } by browser send http request have grabled code but by postman send message can normal display. thanks you give me help. --- ## [Logstash output syslog : how to remove added {host} field?](https://discuss.elastic.co/t/logstash-output-syslog-how-to-remove-added-host-field/260246)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 5\ **Last updated:** [January 6, 2021, 5:48pm UTC](https://discuss.elastic.co/t/logstash-output-syslog-how-to-remove-added-host-field/260246 "2021-01-06T17:48:45Z")
Hello there ! I have to forward kafka logs to syslog relay. I know it's weird but I have to. Here is logstash conf : #kafka input input { kafka { topics =\> \["test"\] codec =\> json bootstrap\_servers =\> "ka… --- ## [Logstash - how to configure CSV filter for joining 2 CSV files based on a common field (mapping - one to many data) and ingest the data into single index](https://discuss.elastic.co/t/logstash-how-to-configure-csv-filter-for-joining-2-csv-files-based-on-a-common-field-mapping-one-to-many-data-and-ingest-the-data-into-single-index/259994)
**Author:** [@Jenisha\_Ramanathan](https://discuss.elastic.co/u/Jenisha_Ramanathan)\ **Replies:** 5\ **Last updated:** [January 6, 2021, 4:51pm UTC](https://discuss.elastic.co/t/logstash-how-to-configure-csv-filter-for-joining-2-csv-files-based-on-a-common-field-mapping-one-to-many-data-and-ingest-the-data-into-single-index/259994 "2021-01-06T16:51:21Z")
Hi Team, I have 2 CVS files which contains one similar column header say "faculty\_id", For the rows which has same "faculty\_id" value, I want the below steps to be done Join/combine the data from both the csv files i… --- ## [Denormalizing with filter in Logstash](https://discuss.elastic.co/t/denormalizing-with-filter-in-logstash/259846)
**Author:** [@Tim\_Zachow](https://discuss.elastic.co/u/Tim_Zachow)\ **Replies:** 7\ **Last updated:** [January 6, 2021, 4:09pm UTC](https://discuss.elastic.co/t/denormalizing-with-filter-in-logstash/259846 "2021-01-06T16:09:58Z")
Hello dear community, it's nice to be able to post my first topic. So I just started to work with Elasticsearch. Thus I'm very inexperienced and after some time I sadly was not able to find a solution by myself. I nee… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=264) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=266)