# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=113 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 114 --- ## [Facing communication issue while sending data from SQL to logstash](https://discuss.elastic.co/t/facing-communication-issue-while-sending-data-from-sql-to-logstash/315269)
**Author:** [@Bhanuji\_paluri](https://discuss.elastic.co/u/Bhanuji_paluri)\ **Replies:** 0\ **Last updated:** [September 27, 2022, 2:08pm UTC](https://discuss.elastic.co/t/facing-communication-issue-while-sending-data-from-sql-to-logstash/315269 "2022-09-27T14:08:41Z")
find the logs from logstash side. \[2022-09-27T15:47:00,258\]\[ERROR\]\[logstash.inputs.jdbc \]\[test\]\[f4db26278744478bf3c5e76b57171244092a989c2d32ce74f9029d5a6e668706\] Unable to connect to database. Tried 1 times {:error\_… --- ## [Default Index Pattern (logstash-yyyy.MM.dd) Failing](https://discuss.elastic.co/t/default-index-pattern-logstash-yyyy-mm-dd-failing/315195)
**Author:** [@scriner](https://discuss.elastic.co/u/scriner)\ **Replies:** 7\ **Last updated:** [September 27, 2022, 1:33pm UTC](https://discuss.elastic.co/t/default-index-pattern-logstash-yyyy-mm-dd-failing/315195 "2022-09-27T13:33:43Z")
Hi all. We are using the following pipeline config file to ship syslogs to Logstash: input { tcp { port =\> 1514 codec =\> "json" type =\> "syslog" } udp { port =\> 1514 codec =\> "json" type =\> … --- ## [How to chain multiple input in Logstash](https://discuss.elastic.co/t/how-to-chain-multiple-input-in-logstash/315179)
**Author:** [@anand\_tripathi](https://discuss.elastic.co/u/anand_tripathi)\ **Replies:** 3\ **Last updated:** [September 27, 2022, 12:50pm UTC](https://discuss.elastic.co/t/how-to-chain-multiple-input-in-logstash/315179 "2022-09-27T12:50:47Z")
My use case is something like below First Input(Mysql JDBC) I'm selecting some data from MySQL like select pk\_id, score, other\_id from \ Second Chained Input (Elasticsearch) Then I want to pass the data p… --- ## [Update document structure without duplication](https://discuss.elastic.co/t/update-document-structure-without-duplication/315244)
**Author:** [@reed](https://discuss.elastic.co/u/reed)\ **Replies:** 0\ **Last updated:** [September 27, 2022, 9:27am UTC](https://discuss.elastic.co/t/update-document-structure-without-duplication/315244 "2022-09-27T09:27:26Z")
Hi all, I need to read alla index documents and for all add a new field with default value, my code is this: input { elasticsearch { hosts =\> \["http://myelastic:9200"\] index =\> "myindex-txt" user =\> aaa passwor… --- ## [Elasticsearch index\_search\_slowlog.log has error](https://discuss.elastic.co/t/elasticsearch-index-search-slowlog-log-has-error/315240)
**Author:** [@hellocomputer](https://discuss.elastic.co/u/hellocomputer)\ **Replies:** 0\ **Last updated:** [September 27, 2022, 7:26am UTC](https://discuss.elastic.co/t/elasticsearch-index-search-slowlog-log-has-error/315240 "2022-09-27T07:26:36Z")
Hello, I'm working with Elasticsearch 7.14.0 version in ubuntu 18.04 version. We want to gain "search keyword" from Elasticsearch slow log. Suddenly, Elasticsearch doesn't make new search log. What is the problem in thi… --- ## [How to ignore exceptions and read the next offset of kafka](https://discuss.elastic.co/t/how-to-ignore-exceptions-and-read-the-next-offset-of-kafka/315235)
**Author:** [@lythen](https://discuss.elastic.co/u/lythen)\ **Replies:** 0\ **Last updated:** [September 27, 2022, 6:49am UTC](https://discuss.elastic.co/t/how-to-ignore-exceptions-and-read-the-next-offset-of-kafka/315235 "2022-09-27T06:49:03Z")
My Logstash is used in reading the data of kafka and sending them to Elasticsearch. But now I 'm facing some problems,sucsh as when the index was frozen in ES, the Logstash would throw an exception and then, the Logstash… --- ## [Trying to install output mongodb](https://discuss.elastic.co/t/trying-to-install-output-mongodb/314309)
**Author:** [@Niels434](https://discuss.elastic.co/u/Niels434)\ **Replies:** 6\ **Last updated:** [September 27, 2022, 5:23am UTC](https://discuss.elastic.co/t/trying-to-install-output-mongodb/314309 "2022-09-27T05:23:08Z")
I'm trying to get the output plugin MongoDB for the Logstash pipeline installed. But i'm getting an error code: jruby: warning: unknown property jruby.mx1g jruby: warning: unknown property jruby.ms1g jruby: unknown o… --- ## [Logstash circuit breaking](https://discuss.elastic.co/t/logstash-circuit-breaking/315049)
**Author:** [@Harper\_S1](https://discuss.elastic.co/u/Harper_S1)\ **Replies:** 2\ **Last updated:** [September 27, 2022, 3:24am UTC](https://discuss.elastic.co/t/logstash-circuit-breaking/315049 "2022-09-27T03:24:11Z")
Hi, Do anyone know how to solve this circuit breaking exception in logstash (7.10). \[2022-09-23T14:38:22,920\]\[INFO \]\[logstash.outputs.elasticsearch\]\[main\]\[299ec4f1e5994d0fe7b59d4e4d29f50e734f0d6401d909dc198ecbc402ca398… --- ## [Error multiples inputs - block in multifilter, block in start\_workers, ParseException: No close tag for /log4j:event/log4j:throwable](https://discuss.elastic.co/t/error-multiples-inputs-block-in-multifilter-block-in-start-workers-parseexception-no-close-tag-for-log4j-event-log4j-throwable/315106)
**Author:** [@erwin339](https://discuss.elastic.co/u/erwin339)\ **Replies:** 4\ **Last updated:** [September 26, 2022, 6:54pm UTC](https://discuss.elastic.co/t/error-multiples-inputs-block-in-multifilter-block-in-start-workers-parseexception-no-close-tag-for-log4j-event-log4j-throwable/315106 "2022-09-26T18:54:15Z")
Context: I have 4 folders, one contains logstash, the other 3 each contain filebeat, the first acts as if an application launched log, the second with 2 applications that send logs for which filebeat has 2 entries, and t… --- ## [Why is the error log being sent divided?](https://discuss.elastic.co/t/why-is-the-error-log-being-sent-divided/315192)
**Author:** [@erwin339](https://discuss.elastic.co/u/erwin339)\ **Replies:** 1\ **Last updated:** [September 26, 2022, 6:08pm UTC](https://discuss.elastic.co/t/why-is-the-error-log-being-sent-divided/315192 "2022-09-26T18:08:56Z")
I am sending error and fatal logs, these have multiple lines, so the multiline is configured like this: filebeat multiline: This is throwing me an error: my error and fatal log have this structure: \ **Author:** [@shivani\_aggarwal](https://discuss.elastic.co/u/shivani_aggarwal)\ **Replies:** 4\ **Last updated:** [September 26, 2022, 5:38pm UTC](https://discuss.elastic.co/t/logstash-not-working-with-jdk-11-0-16/315105 "2022-09-26T17:38:35Z") Hi, Version details: Logstash - logstash-oss:7.17.3 OS: centos7 java version: java-11-openjdk-11.0.16.0.8 Background: On a centos-based docker image, I install java, followed by the logstash-oss rpm and use this in … --- ## [Remove\_fields doesn't remove field in json](https://discuss.elastic.co/t/remove-fields-doesnt-remove-field-in-json/315176)
**Author:** [@Kostyantyn\_Dobriohlo](https://discuss.elastic.co/u/Kostyantyn_Dobriohlo)\ **Replies:** 2\ **Last updated:** [September 26, 2022, 4:53pm UTC](https://discuss.elastic.co/t/remove-fields-doesnt-remove-field-in-json/315176 "2022-09-26T16:53:23Z")
Cannot remove fields in json, 'cause always have a json parsing error. Tried many different ways, but inside the json nothing changes. And there is a warning in logs: Error parsing json Config file: input { tcp { … --- ## [OutOfDirectMemoryError: failed in Logstash logs](https://discuss.elastic.co/t/outofdirectmemoryerror-failed-in-logstash-logs/315145)
**Author:** [@madurad](https://discuss.elastic.co/u/madurad)\ **Replies:** 2\ **Last updated:** [September 26, 2022, 4:45pm UTC](https://discuss.elastic.co/t/outofdirectmemoryerror-failed-in-logstash-logs/315145 "2022-09-26T16:45:13Z")
Hi, Continuously getting OutOfDirectMemoryError: failed error on Logstash, I chekced the host memory usage and CPU all good and memory is around 60% used, but no clear idea how this issue is being happened. In jvm.opt… --- ## [\[ERROR\]\[logstash.javapipeline](https://discuss.elastic.co/t/error-logstash-javapipeline/314746)
**Author:** [@wxhgwh](https://discuss.elastic.co/u/wxhgwh)\ **Replies:** 18\ **Last updated:** [September 26, 2022, 3:44am UTC](https://discuss.elastic.co/t/error-logstash-javapipeline/314746 "2022-09-26T03:44:05Z")
\`\`\` \[ERROR\]\[logstash.licensechecker.licensereader\] Unable to retrieve license information fromlicense server {:message=\>"Unsupported or unrecognized SSL message"}Preformatted text \[ERROR\]\[logstash.javapipeline \]\[mai… --- ## [Parsing a list of lists with logstash filter](https://discuss.elastic.co/t/parsing-a-list-of-lists-with-logstash-filter/314747)
**Author:** [@amirfarsi](https://discuss.elastic.co/u/amirfarsi)\ **Replies:** 2\ **Last updated:** [September 24, 2022, 12:52pm UTC](https://discuss.elastic.co/t/parsing-a-list-of-lists-with-logstash-filter/314747 "2022-09-24T12:52:49Z")
Hello friends. I have a list of lists in the form below: doc 1: \[\[40004, 10\], \[40005, 12\]\] doc 2: \[\[40004, 8\], \[40006, 12\], \[20002, 3\]\] I want to change them to the following form: doc 1: { "40004": 10, "… --- ## [Preparing Logs analytics with Logstash (for kibana)](https://discuss.elastic.co/t/preparing-logs-analytics-with-logstash-for-kibana/314834)
**Author:** [@Mark\_S](https://discuss.elastic.co/u/Mark_S)\ **Replies:** 8\ **Last updated:** [September 24, 2022, 8:35am UTC](https://discuss.elastic.co/t/preparing-logs-analytics-with-logstash-for-kibana/314834 "2022-09-24T08:35:18Z")
When creating analytics (for kibana) I need to create keywords Because unfortunately simple text cannot be analyzed in kibana Dashboards. So I am trying to break up the logs. Grok constructor works fine when the logs … --- ## [Logstash Error Failed to Execute action](https://discuss.elastic.co/t/logstash-error-failed-to-execute-action/315051)
**Author:** [@Technolust](https://discuss.elastic.co/u/Technolust)\ **Replies:** 7\ **Last updated:** [September 23, 2022, 9:38pm UTC](https://discuss.elastic.co/t/logstash-error-failed-to-execute-action/315051 "2022-09-23T21:38:01Z")
Team, I've lost all my hair... For whatever reason, I can't figure this out. \[2022-09-23T15:09:46,265\]\[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:m… --- ## [VMWare NSX Parsing](https://discuss.elastic.co/t/vmware-nsx-parsing/315054)
**Author:** [@groth](https://discuss.elastic.co/u/groth)\ **Replies:** 2\ **Last updated:** [September 23, 2022, 9:18pm UTC](https://discuss.elastic.co/t/vmware-nsx-parsing/315054 "2022-09-23T21:18:42Z")
I'm currently working on a Logstash pipeline for VMWare NSX firewall logs coming in over syslog forwarding. Some of the ICMP logs have a couple numbers between the protocol name and the source and destination IPs that VM… --- ## [Block in coverage\_state Error in Logstash Pipeline](https://discuss.elastic.co/t/block-in-coverage-state-error-in-logstash-pipeline/315040)
**Author:** [@andreakatie](https://discuss.elastic.co/u/andreakatie)\ **Replies:** 2\ **Last updated:** [September 23, 2022, 5:25pm UTC](https://discuss.elastic.co/t/block-in-coverage-state-error-in-logstash-pipeline/315040 "2022-09-23T17:25:42Z")
Hello all, I am running an ELK stack (Elasticsearch v. 7.1.2, Filebeat v. 7.8.0, and Logstash v. 7.16.3) and recently added the following code to a Logstash .conf template (one of many in the stack which has been succes… --- ## [Elk does not see indexes](https://discuss.elastic.co/t/elk-does-not-see-indexes/315012)
**Author:** [@holpa](https://discuss.elastic.co/u/holpa)\ **Replies:** 1\ **Last updated:** [September 23, 2022, 12:38pm UTC](https://discuss.elastic.co/t/elk-does-not-see-indexes/315012 "2022-09-23T12:38:56Z")
Hello. I'm just learning how to work at ELK. I want to set up the transfer of logs from AD Installed on 1 server: elasticsearch, kibana, logstash 8.4 v 2 server - windows forwarding event - winlogbeat winlogbeat.event… --- ## [SNMP Pipeline didn't work](https://discuss.elastic.co/t/snmp-pipeline-didnt-work/315003)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 0\ **Last updated:** [September 23, 2022, 9:29am UTC](https://discuss.elastic.co/t/snmp-pipeline-didnt-work/315003 "2022-09-23T09:29:58Z")
Hello everyone, i'm struggling with snmp pipeline here. i have configured the pipeline like below picture. i used walk and define the tables too. when i run the logstash, there is no error log about this pipeline. it's … --- ## [Object mapping for \[data.value\] tried to parse field \[value\] as object, but found a concrete value](https://discuss.elastic.co/t/object-mapping-for-data-value-tried-to-parse-field-value-as-object-but-found-a-concrete-value/314867)
**Author:** [@shivendra95](https://discuss.elastic.co/u/shivendra95)\ **Replies:** 2\ **Last updated:** [September 23, 2022, 5:44am UTC](https://discuss.elastic.co/t/object-mapping-for-data-value-tried-to-parse-field-value-as-object-but-found-a-concrete-value/314867 "2022-09-23T05:44:03Z")
Hi, I'm trying to read log files from s3 bucket, but for some log lines I'm getting the below error \[WARN \] 2022-09-21 10:43:42.756 \[\[main\]\>worker0\] elasticsearch - Could not index event to Elasticsearch. {:status=\>4… --- ## [Issue with output plugin type elasticsearch in logstash](https://discuss.elastic.co/t/issue-with-output-plugin-type-elasticsearch-in-logstash/314967)
**Author:** [@sbk-elk](https://discuss.elastic.co/u/sbk-elk)\ **Replies:** 4\ **Last updated:** [September 22, 2022, 8:39pm UTC](https://discuss.elastic.co/t/issue-with-output-plugin-type-elasticsearch-in-logstash/314967 "2022-09-22T20:39:39Z")
Hello All, Below is my logstash configuration. I have a few questions with respect to how i added the fields and how they carry over in the stages of my pipeline. Do the fields set in hash in my input plugin are added… --- ## [Is there anyway to use Logstash on Cloud?](https://discuss.elastic.co/t/is-there-anyway-to-use-logstash-on-cloud/314975)
**Author:** [@SamuelSMendes](https://discuss.elastic.co/u/SamuelSMendes)\ **Replies:** 1\ **Last updated:** [September 22, 2022, 9:37pm UTC](https://discuss.elastic.co/t/is-there-anyway-to-use-logstash-on-cloud/314975 "2022-09-22T21:37:02Z")
So far I'm aware of the Ingest Pipelines and they are really useful, but I've been using a lot of Logstash plugins (mostly JDBC and HTTP) plusm mutates in a daily basis and so I started wondering if there is also a solut… --- ## [Convert my doc.timestamp as date](https://discuss.elastic.co/t/convert-my-doc-timestamp-as-date/314973)
**Author:** [@erwin339](https://discuss.elastic.co/u/erwin339)\ **Replies:** 3\ **Last updated:** [September 22, 2022, 9:08pm UTC](https://discuss.elastic.co/t/convert-my-doc-timestamp-as-date/314973 "2022-09-22T21:08:07Z")
I am currently trying to get the date the logs were generated since I only have the date they were processed, I am receiving the logs in log4j xml format. I am unable to convert the UNIX date to this format Sep 22, 2022 … --- ## [Trying to upload a csv to an index fails](https://discuss.elastic.co/t/trying-to-upload-a-csv-to-an-index-fails/314954)
**Author:** [@alter1](https://discuss.elastic.co/u/alter1)\ **Replies:** 1\ **Last updated:** [September 22, 2022, 3:53pm UTC](https://discuss.elastic.co/t/trying-to-upload-a-csv-to-an-index-fails/314954 "2022-09-22T15:53:54Z")
I am trying to load a csv file to an index associated with ILM policies, the first time it was executed it was successful, then I have deleted and recreated the index and the load has not worked again. This is the pipel… --- ## [Logstash filter for records of nested json messages from eventhub](https://discuss.elastic.co/t/logstash-filter-for-records-of-nested-json-messages-from-eventhub/314930)
**Author:** [@Sivaramakrishhna\_Amb](https://discuss.elastic.co/u/Sivaramakrishhna_Amb)\ **Replies:** 2\ **Last updated:** [September 22, 2022, 2:50pm UTC](https://discuss.elastic.co/t/logstash-filter-for-records-of-nested-json-messages-from-eventhub/314930 "2022-09-22T14:50:03Z")
Input message looks like below { "records": \[ { "level": "Informational", "properties": { "Keywords": 0, "ProviderName": "Core.LogRecord", "Message": "2022-09-08 08:17:02,935 \[23\] INFO {"Message":"9/8/2022 8:17:0… --- ## [\`block in converge\_state' when setting up output to gelf host](https://discuss.elastic.co/t/block-in-converge-state-when-setting-up-output-to-gelf-host/314862)
**Author:** [@Heffie](https://discuss.elastic.co/u/Heffie)\ **Replies:** 1\ **Last updated:** [September 22, 2022, 8:14am UTC](https://discuss.elastic.co/t/block-in-converge-state-when-setting-up-output-to-gelf-host/314862 "2022-09-22T08:14:33Z")
Hi, Im getting this error: \[2022-09-21T11:32:55,738\]\[FATAL\]\[logstash.runner \] An unexpected error occurred! {:error=\>#\ **Author:** [@SamuelSMendes](https://discuss.elastic.co/u/SamuelSMendes)\ **Replies:** 3\ **Last updated:** [September 21, 2022, 8:50pm UTC](https://discuss.elastic.co/t/is-it-possible-to-format-logstash-output-in-a-way-that-it-shows-in-uptime/314883 "2022-09-21T20:50:25Z") I was thinking if, by using mutate in the output of my pipeline on logstash in a way that "mimics" the heartbeat output, I could have an uptime application to see the return of my pseudo-heartbeat (generated by the logst… --- ## [Avoid Timestamp and host from the logstash file output](https://discuss.elastic.co/t/avoid-timestamp-and-host-from-the-logstash-file-output/314870)
**Author:** [@evgrajesh](https://discuss.elastic.co/u/evgrajesh)\ **Replies:** 4\ **Last updated:** [September 21, 2022, 4:38pm UTC](https://discuss.elastic.co/t/avoid-timestamp-and-host-from-the-logstash-file-output/314870 "2022-09-21T16:38:35Z")
Hi I am using very simple configuration. Receive strings from TCP input and write to a file Sent string This is new string 8 Written in to file 2022-09-21T12:36:28.278922Z %{host} This is new string 8 I want to avo… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=112) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=114)