# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=191 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 192 --- ## [Logstash Rabbitmq connection refused?](https://discuss.elastic.co/t/logstash-rabbitmq-connection-refused/285801)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 0\ **Last updated:** [October 4, 2021, 10:15am UTC](https://discuss.elastic.co/t/logstash-rabbitmq-connection-refused/285801 "2021-10-04T10:15:56Z")
I am using multiple instances using logstash, First I get an error multiple instance error when I resolved that error. Now I am getting a new error. \[ERROR\]\[logstash.inputs.rabbitmq \] RabbitMQ connection error, will ret… --- ## [Insert records for parent/child in same index of elastic search](https://discuss.elastic.co/t/insert-records-for-parent-child-in-same-index-of-elastic-search/285774)
**Author:** [@manjurgani](https://discuss.elastic.co/u/manjurgani)\ **Replies:** 0\ **Last updated:** [October 3, 2021, 6:52pm UTC](https://discuss.elastic.co/t/insert-records-for-parent-child-in-same-index-of-elastic-search/285774 "2021-10-03T18:52:49Z")
I have created a mapping in Elasticsearch like below POST /INDEX { "mappings": { "properties": { "id": { "type": "keyword" }, "join\_field": { "typ… --- ## [Data not showing in elastic and kibana for one of the index pattern out of three indexes](https://discuss.elastic.co/t/data-not-showing-in-elastic-and-kibana-for-one-of-the-index-pattern-out-of-three-indexes/284464)
**Author:** [@vasanthyvns](https://discuss.elastic.co/u/vasanthyvns)\ **Replies:** 3\ **Last updated:** [October 3, 2021, 3:45pm UTC](https://discuss.elastic.co/t/data-not-showing-in-elastic-and-kibana-for-one-of-the-index-pattern-out-of-three-indexes/284464 "2021-10-03T15:45:01Z")
I have enabled logstash to monitor 3 different logs, but somehow data is not showing up for one of the index pattern, checked almost everything and couldn't find anything. Not sure if i am missing something here. --- ## [How to make an logstash pipeline to run contuously (Elasticsearch input plugin)](https://discuss.elastic.co/t/how-to-make-an-logstash-pipeline-to-run-contuously-elasticsearch-input-plugin/285714)
**Author:** [@Thanura\_Kannangara](https://discuss.elastic.co/u/Thanura_Kannangara)\ **Replies:** 3\ **Last updated:** [October 2, 2021, 3:22pm UTC](https://discuss.elastic.co/t/how-to-make-an-logstash-pipeline-to-run-contuously-elasticsearch-input-plugin/285714 "2021-10-02T15:22:33Z")
Hey Folks, I'm running an elastic pipeline to read from Elasticsearch and write documents to an S3 bucket input { elasticsearch { ssl =\> true hosts =\> \["\"\] user =\> "\" password =\> "\<… --- ## [How to accept rsyslog on logstash with filebeat](https://discuss.elastic.co/t/how-to-accept-rsyslog-on-logstash-with-filebeat/285734)
**Author:** [@sandy12](https://discuss.elastic.co/u/sandy12)\ **Replies:** 0\ **Last updated:** [October 2, 2021, 12:10pm UTC](https://discuss.elastic.co/t/how-to-accept-rsyslog-on-logstash-with-filebeat/285734 "2021-10-02T12:10:00Z")
I want to send logs from rsyslog server to logstash and then to Elasticsearch. But I want to send rsyslog with installing filebeat . Can you please tell me how to accept the rsyslog on logstash server. What settings I… --- ## [ES index date format for date/timezone offset](https://discuss.elastic.co/t/es-index-date-format-for-date-timezone-offset/285718)
**Author:** [@earlsanchez](https://discuss.elastic.co/u/earlsanchez)\ **Replies:** 1\ **Last updated:** [October 2, 2021, 1:06am UTC](https://discuss.elastic.co/t/es-index-date-format-for-date-timezone-offset/285718 "2021-10-02T01:06:10Z")
Hello, We're using Elasticsearch v7.9. The date format of our mongod logs changed to this: 2021-09-26T03:23:46.515-0700 Now we're getting "\_dateparsefailure" during Logstash processing. I can't find a specific examp… --- ## [Ruby filter to extract different data elements from a string based on pattern](https://discuss.elastic.co/t/ruby-filter-to-extract-different-data-elements-from-a-string-based-on-pattern/285717)
**Author:** [@Indigo\_Star](https://discuss.elastic.co/u/Indigo_Star)\ **Replies:** 2\ **Last updated:** [October 2, 2021, 12:32am UTC](https://discuss.elastic.co/t/ruby-filter-to-extract-different-data-elements-from-a-string-based-on-pattern/285717 "2021-10-02T00:32:38Z")
Hi, I need to extract data from a field transaction which contains value as: Transaction = "CHE\_ECSTATE '-259297525' 19802619:-259297525{0}BEACH.TCN4(TCN4)ECN4/UTR023/ECTBR@20210929.162523.985 I need fields like CHE… --- ## [Logstash error - Bundler::GemNotFound: Could not find jar-dependencies-0.4.1 in any of the sources](https://discuss.elastic.co/t/logstash-error-bundler-could-not-find-jar-dependencies-0-4-1-in-any-of-the-sources/285658)
**Author:** [@Mario\_Sabetta](https://discuss.elastic.co/u/Mario_Sabetta)\ **Replies:** 0\ **Last updated:** [October 1, 2021, 8:15am UTC](https://discuss.elastic.co/t/logstash-error-bundler-could-not-find-jar-dependencies-0-4-1-in-any-of-the-sources/285658 "2021-10-01T08:15:08Z")
Hello everyone, I started logstash 7.15.0 and I encountered an event with the following error message and it tells me that the bundler cannot find the gem jar-dependencies-0.4.1. What needs to be done? Thanks for the at… --- ## [How can i parse log file and match specific strings with grok filter](https://discuss.elastic.co/t/how-can-i-parse-log-file-and-match-specific-strings-with-grok-filter/285115)
**Author:** [@hagaluly](https://discuss.elastic.co/u/hagaluly)\ **Replies:** 4\ **Last updated:** [October 1, 2021, 9:57pm UTC](https://discuss.elastic.co/t/how-can-i-parse-log-file-and-match-specific-strings-with-grok-filter/285115 "2021-10-01T21:57:55Z")
i have a log that looks like this BUILD\_DISPLAY\_NAME=#998 BUILD\_NUMBER=998 JENKINS\_URL=https:///jenkins.com BUILD\_ID=998 DEVICE\_CRED\_PSW=pass GIT\_PREVIOUS\_SUCCESSFUL\_COMMIT=f847h56934875f6239487562j3498 JOB\_BASE\_NAME=ma… --- ## [If "keyword" in message not working for logstash](https://discuss.elastic.co/t/if-keyword-in-message-not-working-for-logstash/285622)
**Author:** [@kriss332](https://discuss.elastic.co/u/kriss332)\ **Replies:** 7\ **Last updated:** [October 1, 2021, 6:47pm UTC](https://discuss.elastic.co/t/if-keyword-in-message-not-working-for-logstash/285622 "2021-10-01T18:47:41Z")
Hello all... I am receiving logs from 5 different sources on one single port. In fact it is a collection of files being sent through syslog from a server in realtime. The server stores logs from 4 VPN servers and one DN… --- ## [Strange behavior in logstash after remove field message](https://discuss.elastic.co/t/strange-behavior-in-logstash-after-remove-field-message/285524)
**Author:** [@gbeltramelli](https://discuss.elastic.co/u/gbeltramelli)\ **Replies:** 1\ **Last updated:** [October 1, 2021, 6:11pm UTC](https://discuss.elastic.co/t/strange-behavior-in-logstash-after-remove-field-message/285524 "2021-10-01T18:11:25Z")
Hi, first of all sorry for my english! im using version 7.14 of ELK and i\`m having some strange behavior when using mutate { remove\_field =\> \[ "message" \] } I have some logs from a DataPower then I use GROK to make … --- ## [Failed to start logstash](https://discuss.elastic.co/t/failed-to-start-logstash/285278)
**Author:** [@ts5366](https://discuss.elastic.co/u/ts5366)\ **Replies:** 6\ **Last updated:** [October 1, 2021, 4:27pm UTC](https://discuss.elastic.co/t/failed-to-start-logstash/285278 "2021-10-01T16:27:11Z")
input { kinesis { application\_name =\> "stat\_events" kinesis\_stream\_name =\> "events" region =\> "us-east-1" codec =\> gzip\_lines {} initial\_position\_in\_stream =\> "LATEST" } } output { stdout { … --- ## [ASA Single Grok Filter Not Working](https://discuss.elastic.co/t/asa-single-grok-filter-not-working/285627)
**Author:** [@Micah\_Barsness](https://discuss.elastic.co/u/Micah_Barsness)\ **Replies:** 3\ **Last updated:** [October 1, 2021, 12:51pm UTC](https://discuss.elastic.co/t/asa-single-grok-filter-not-working/285627 "2021-10-01T12:51:31Z")
Hi, so I'm using several grok filters to parse out different messages. My config is a bit messy now because I feel like i've tried everything. Logs are being properly parsed through pre-made filters and one custom one s… --- ## [Is it better to have one pipeline or multiple pipelines?](https://discuss.elastic.co/t/is-it-better-to-have-one-pipeline-or-multiple-pipelines/285493)
**Author:** [@riahc3](https://discuss.elastic.co/u/riahc3)\ **Replies:** 6\ **Last updated:** [September 30, 2021, 5:12pm UTC](https://discuss.elastic.co/t/is-it-better-to-have-one-pipeline-or-multiple-pipelines/285493 "2021-09-30T17:12:41Z")
Hello From the start, Ive implemented the Elastic Stack using Logstash as the reciever and sender of logs to Logstash. Ive always implemented it using various pipelines. Each pipeline is organized by a different config… --- ## [Collecting syslog (514/udp) with Logstash in Docker swarm](https://discuss.elastic.co/t/collecting-syslog-514-udp-with-logstash-in-docker-swarm/285610)
**Author:** [@antoine.brun](https://discuss.elastic.co/u/antoine.brun)\ **Replies:** 0\ **Last updated:** [September 30, 2021, 3:29pm UTC](https://discuss.elastic.co/t/collecting-syslog-514-udp-with-logstash-in-docker-swarm/285610 "2021-09-30T15:29:28Z")
Hello, we are trying to test Logstash to collect syslogs to replace our in-house syslog collector. This works fine when our application runs in Docker, the syslogs are sent to the nginx container that forwards them to … --- ## [How to modify a single value from timestamp](https://discuss.elastic.co/t/how-to-modify-a-single-value-from-timestamp/285567)
**Author:** [@Jose\_E](https://discuss.elastic.co/u/Jose_E)\ **Replies:** 3\ **Last updated:** [September 30, 2021, 3:08pm UTC](https://discuss.elastic.co/t/how-to-modify-a-single-value-from-timestamp/285567 "2021-09-30T15:08:13Z")
Hi all, I'm trying to find out a way to substitute a single value from @timestamp, from another one, for instance, the hour. So far I know you can get the timestamp separated values using %{+HH}, but I can't find a way… --- ## [Logstash misinterprets incoming data](https://discuss.elastic.co/t/logstash-misinterprets-incoming-data/285200)
**Author:** [@hellotty](https://discuss.elastic.co/u/hellotty)\ **Replies:** 4\ **Last updated:** [September 30, 2021, 2:45pm UTC](https://discuss.elastic.co/t/logstash-misinterprets-incoming-data/285200 "2021-09-30T14:45:55Z")
Hello. I am streaming mdaemon logs using filebeat and using multiline.pattern to concatenate lines into one event. Using logstash, I create fields "mail from", "mail to", "message id", etc. Is it possible to make grok … --- ## [TCP plugin tuning](https://discuss.elastic.co/t/tcp-plugin-tuning/285510)
**Author:** [@vaseemQA](https://discuss.elastic.co/u/vaseemQA)\ **Replies:** 5\ **Last updated:** [September 30, 2021, 2:40pm UTC](https://discuss.elastic.co/t/tcp-plugin-tuning/285510 "2021-09-30T14:40:16Z")
Hi Team, I'm Using TCP plugin inside input and opened port =\> 12345 and then I've started logstash which works a listener, on the other side I've generated a sample shell script which works as a client and opened a TCP … --- ## [Unexpected end of ZLIB input stream on parsing gzip files](https://discuss.elastic.co/t/unexpected-end-of-zlib-input-stream-on-parsing-gzip-files/285603)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 0\ **Last updated:** [September 30, 2021, 2:35pm UTC](https://discuss.elastic.co/t/unexpected-end-of-zlib-input-stream-on-parsing-gzip-files/285603 "2021-09-30T14:35:55Z")
hello , I have a few MBs of gzip files ,which logstash fails to parse for some reason . I have 3 configs set up in pipelines.yml and 1 succeeds first and the other 2 fail the same error . \[2021-09-30T14:29:47,055\]\[ERRO… --- ## [Logstash persistent queue max capacity](https://discuss.elastic.co/t/logstash-persistent-queue-max-capacity/285588)
**Author:** [@samia](https://discuss.elastic.co/u/samia)\ **Replies:** 0\ **Last updated:** [September 30, 2021, 12:19pm UTC](https://discuss.elastic.co/t/logstash-persistent-queue-max-capacity/285588 "2021-09-30T12:19:27Z")
Hello, I would like to know if there's a maximum for the queue size of Logstash persistent queue ? I read that its default value is 1GB and it must not exceed the local disk size. I wander if it can handle volumes of … --- ## [Logstash memcached filter plugin can not open more then 4 connection](https://discuss.elastic.co/t/logstash-memcached-filter-plugin-can-not-open-more-then-4-connection/285010)
**Author:** [@ramakrushna.sahu](https://discuss.elastic.co/u/ramakrushna.sahu)\ **Replies:** 5\ **Last updated:** [September 30, 2021, 9:27am UTC](https://discuss.elastic.co/t/logstash-memcached-filter-plugin-can-not-open-more-then-4-connection/285010 "2021-09-30T09:27:35Z")
Hi Team, Need help on logstash Memcached filter plugin. I see the logstash memcached filter plugin can not open more than 4 connections to Memcached server. is there any configuration required for this memcached filte… --- ## [Logstash cannot reach Elasticsearch / filebeat security](https://discuss.elastic.co/t/logstash-cannot-reach-elasticsearch-filebeat-security/284733)
**Author:** [@smam](https://discuss.elastic.co/u/smam)\ **Replies:** 11\ **Last updated:** [September 30, 2021, 8:20am UTC](https://discuss.elastic.co/t/logstash-cannot-reach-elasticsearch-filebeat-security/284733 "2021-09-30T08:20:07Z")
Hello, I am encountering some Problems while trying to secure Filebeat&Logstash. Scenario: I created my own RootCA and signed certificates with it. My Kibana-Webserver for example is signed with this Certificate, but … --- ## [Failed to execute action](https://discuss.elastic.co/t/failed-to-execute-action/285464)
**Author:** [@Busra\_Duygu](https://discuss.elastic.co/u/Busra_Duygu)\ **Replies:** 2\ **Last updated:** [September 30, 2021, 6:27am UTC](https://discuss.elastic.co/t/failed-to-execute-action/285464 "2021-09-30T06:27:44Z")
Hi firends i am getting an error like below but there is no error in the first line of my conf file. Does anyone have an idea ? Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exce… --- ## [ElasticSearch Output Plugin partial update results in entire document substitution](https://discuss.elastic.co/t/elasticsearch-output-plugin-partial-update-results-in-entire-document-substitution/285512)
**Author:** [@samir\_varandas](https://discuss.elastic.co/u/samir_varandas)\ **Replies:** 3\ **Last updated:** [September 29, 2021, 11:19pm UTC](https://discuss.elastic.co/t/elasticsearch-output-plugin-partial-update-results-in-entire-document-substitution/285512 "2021-09-29T23:19:32Z")
Hi friends, I am having some trouble with the Elasticsearch Output Plugin, I am trying to perform a partial update on a document inside my Elastic index, I want to update certain fields only (the ones mapped in my event)… --- ## [Parse contents of a specific field and store in new fields](https://discuss.elastic.co/t/parse-contents-of-a-specific-field-and-store-in-new-fields/285527)
**Author:** [@rickyhicky](https://discuss.elastic.co/u/rickyhicky)\ **Replies:** 0\ **Last updated:** [September 29, 2021, 10:41pm UTC](https://discuss.elastic.co/t/parse-contents-of-a-specific-field-and-store-in-new-fields/285527 "2021-09-29T22:41:33Z")
Hello all, I have few log entries like below, which I am parsing using grok/regex. How can I again parse contents of a field and store it to new fields within the same index? Log example: \`2021-09-24T17:05:52,777 INFO… --- ## [How to run down source of mutate error?](https://discuss.elastic.co/t/how-to-run-down-source-of-mutate-error/285500)
**Author:** [@mistrhanky](https://discuss.elastic.co/u/mistrhanky)\ **Replies:** 5\ **Last updated:** [September 29, 2021, 9:55pm UTC](https://discuss.elastic.co/t/how-to-run-down-source-of-mutate-error/285500 "2021-09-29T21:55:26Z")
I have an error like the one below. In fact I get a fair number of them, lets say a few hundred each day on a production system(7.12) that logs fairly heavy, 800 million or so records a day. So the amount of these errors… --- ## [Logstash doesnot ingest data when started as a service](https://discuss.elastic.co/t/logstash-doesnot-ingest-data-when-started-as-a-service/285491)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 5\ **Last updated:** [September 29, 2021, 5:57pm UTC](https://discuss.elastic.co/t/logstash-doesnot-ingest-data-when-started-as-a-service/285491 "2021-09-29T17:57:33Z")
Hi All, I am trying to ingest csv data into Elasticsearch by using logstash, when i run the conf file manually the data is getting ingested, but when i add the conf file into pipelines.yml and then start logstash as a s… --- ## [Pipeline worker error in logstash config](https://discuss.elastic.co/t/pipeline-worker-error-in-logstash-config/285496)
**Author:** [@kriss332](https://discuss.elastic.co/u/kriss332)\ **Replies:** 2\ **Last updated:** [September 29, 2021, 5:24pm UTC](https://discuss.elastic.co/t/pipeline-worker-error-in-logstash-config/285496 "2021-09-29T17:24:05Z")
Hello all. I am trying to parse a VPN device logs. t send data in KV format. The data received as I see in netcat's output is- \<13\>Sep 29 21:52:00 172.xx.43.101 488 \<134\>1 2021-09-29T21:52:00+05:30 vpn.com2 PulseSecure… --- ## [Parsing JSON with a string at the beginning of each JSON Object](https://discuss.elastic.co/t/parsing-json-with-a-string-at-the-beginning-of-each-json-object/285480)
**Author:** [@pkward](https://discuss.elastic.co/u/pkward)\ **Replies:** 7\ **Last updated:** [September 29, 2021, 2:57pm UTC](https://discuss.elastic.co/t/parsing-json-with-a-string-at-the-beginning-of-each-json-object/285480 "2021-09-29T14:57:03Z")
Hello, I'm trying to parse a json file with a string at the beginning of each object. What is the easiest way to bypass the initial string or parse the json with string at the beginning? Here is an example of the JSON: … --- ## [How to concatenate two fields in logstash?](https://discuss.elastic.co/t/how-to-concatenate-two-fields-in-logstash/285455)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 2\ **Last updated:** [September 29, 2021, 12:13pm UTC](https://discuss.elastic.co/t/how-to-concatenate-two-fields-in-logstash/285455 "2021-09-29T12:13:35Z")
I have json format data. I want to concatenate firstName and LastName. I have tried by using mutate filter filter{ mutate { add\_field =\> { "Full Name" =\> "%{FirstName}%{LastName}" } … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=190) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=192)