# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=149 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 150 --- ## [Logstash pipeline to pipeline - send\_to field variable](https://discuss.elastic.co/t/logstash-pipeline-to-pipeline-send-to-field-variable/301625)
**Author:** [@ahmed\_charafouddine](https://discuss.elastic.co/u/ahmed_charafouddine)\ **Replies:** 2\ **Last updated:** [April 5, 2022, 2:10pm UTC](https://discuss.elastic.co/t/logstash-pipeline-to-pipeline-send-to-field-variable/301625 "2022-04-05T14:10:30Z")
Hello, I am working on the use of pipeline-to-pipeline. I wanted to value a variable and then use it in send\_to of the pipeline plugin. idea to avoid the use of several if..else conditions which surely have an impact o… --- ## [Continuous display of time difference](https://discuss.elastic.co/t/continuous-display-of-time-difference/301645)
**Author:** [@santy1](https://discuss.elastic.co/u/santy1)\ **Replies:** 0\ **Last updated:** [April 5, 2022, 1:48pm UTC](https://discuss.elastic.co/t/continuous-display-of-time-difference/301645 "2022-04-05T13:48:02Z")
Hello Team, I have 2 events , START and END, and using elapsed filter I am able to calculate the time difference and display it in Kibana using Metric chart. However, the END event occurs after 3 to 4 hours, and for th… --- ## [Persistent queue back pressure](https://discuss.elastic.co/t/persistent-queue-back-pressure/301570)
**Author:** [@Ahmad\_Manzoor](https://discuss.elastic.co/u/Ahmad_Manzoor)\ **Replies:** 0\ **Last updated:** [April 5, 2022, 12:26am UTC](https://discuss.elastic.co/t/persistent-queue-back-pressure/301570 "2022-04-05T00:26:50Z")
I am using logstash with jdbc input plugin and elastic app search output plugin. I have around 9,600,000 records. Out of which around 8,700,000 records are successfully uploaded to app search but remaining are pending si… --- ## [Batch size in Logstash](https://discuss.elastic.co/t/batch-size-in-logstash/301487)
**Author:** [@MuthulakshmiS](https://discuss.elastic.co/u/MuthulakshmiS)\ **Replies:** 3\ **Last updated:** [April 5, 2022, 9:57am UTC](https://discuss.elastic.co/t/batch-size-in-logstash/301487 "2022-04-05T09:57:04Z")
Hi Team, There is a confusion regarding the usage of pipeline.batch.size parameter in Logstash. Can anyone please clarify? From the documentations, got to understand that pipeline.batch.size is used to control the num… --- ## [ELK+Filebeat errors - multiline\_codec\_max\_lines\_reached, \_xmlparsefailure, \_split\_type\_failure - Large XML Files - pushing Child nodes as separate events](https://discuss.elastic.co/t/elk-filebeat-errors-multiline-codec-max-lines-reached-xmlparsefailure-split-type-failure-large-xml-files-pushing-child-nodes-as-separate-events/301538)
**Author:** [@rubberband](https://discuss.elastic.co/u/rubberband)\ **Replies:** 0\ **Last updated:** [April 4, 2022, 3:38pm UTC](https://discuss.elastic.co/t/elk-filebeat-errors-multiline-codec-max-lines-reached-xmlparsefailure-split-type-failure-large-xml-files-pushing-child-nodes-as-separate-events/301538 "2022-04-04T15:38:24Z")
Dear Team, Requesting your guidance to solve this problem. Big Problem: There's a set of old, HUGE XML files (100+ MB per file, not pretty-formatted, and over 100 million characters) with logs that I have been trying t… --- ## [Logstash does not execute config file](https://discuss.elastic.co/t/logstash-does-not-execute-config-file/301594)
**Author:** [@Niya](https://discuss.elastic.co/u/Niya)\ **Replies:** 0\ **Last updated:** [April 5, 2022, 6:46am UTC](https://discuss.elastic.co/t/logstash-does-not-execute-config-file/301594 "2022-04-05T06:46:55Z")
I want some data in a postgresql database to be indexed to an Elasticsearch index. To do so I decided to use Logstash. I installed Logstash and JDBC. I perform the following config: jdbc { jdbc\_connection\_s… --- ## [Logstash aborted when run in multipipeline](https://discuss.elastic.co/t/logstash-aborted-when-run-in-multipipeline/301459)
**Author:** [@Vikrant1](https://discuss.elastic.co/u/Vikrant1)\ **Replies:** 2\ **Last updated:** [April 5, 2022, 6:23am UTC](https://discuss.elastic.co/t/logstash-aborted-when-run-in-multipipeline/301459 "2022-04-05T06:23:49Z")
Hi All, When am trying to run multi-pipeline one - two logstash pipe-line getting abort by error - Errno::ENOENT: No such file or directory - /home/user/.logstash\_jdbc\_last\_run\> am not able to find this mentioned file … --- ## [Data stopped coming after DST change (Logstash JDBC input)](https://discuss.elastic.co/t/data-stopped-coming-after-dst-change-logstash-jdbc-input/301562)
**Author:** [@pk.241011](https://discuss.elastic.co/u/pk.241011)\ **Replies:** 4\ **Last updated:** [April 5, 2022, 2:25am UTC](https://discuss.elastic.co/t/data-stopped-coming-after-dst-change-logstash-jdbc-input/301562 "2022-04-05T02:25:25Z")
Hi, We had the DST change in Australia. \[ It ends at 2am (which is 3am Daylight Saving Time) on the first Sunday in April , when clocks are put back one hour.\] The data came in till 1:59 AM in night (April 3rd, 2022) … --- ## [Save only unique events that are not yet stored in ES based on field value](https://discuss.elastic.co/t/save-only-unique-events-that-are-not-yet-stored-in-es-based-on-field-value/301466)
**Author:** [@Lupul\_Dacic](https://discuss.elastic.co/u/Lupul_Dacic)\ **Replies:** 4\ **Last updated:** [April 4, 2022, 7:57pm UTC](https://discuss.elastic.co/t/save-only-unique-events-that-are-not-yet-stored-in-es-based-on-field-value/301466 "2022-04-04T19:57:44Z")
Hi all, I am using Logstash with a http\_poller. Basically I am querying an API and save the data in ELK. All is fine, but the situation is that I am saving the same events and I need unique ones. I would like to save o… --- ## [Logstash is not sending data to the output](https://discuss.elastic.co/t/logstash-is-not-sending-data-to-the-output/301432)
**Author:** [@Bkumar](https://discuss.elastic.co/u/Bkumar)\ **Replies:** 6\ **Last updated:** [April 4, 2022, 6:34pm UTC](https://discuss.elastic.co/t/logstash-is-not-sending-data-to-the-output/301432 "2022-04-04T18:34:19Z")
Hi, This is the message source is sending to the logstash \<132\>Mar 13 23:50:32 k8s-93f066ee-9b76 {\\"projectid\\":\\"JHAR\_BV\\",\\"\_kmd\\":{\\"lmt\\":\\"2022-03-13T23:50:26:388Z\\",\\"ecf\\":\\"2022-03-13T23:50:26:388Z\\"},\\type\\":\\… --- ## [Issue with logstash (1:7.17.1-1) service on Ubuntu 18.04.3 LTS](https://discuss.elastic.co/t/issue-with-logstash-1-7-17-1-1-service-on-ubuntu-18-04-3-lts/299460)
**Author:** [@rahulgupta18](https://discuss.elastic.co/u/rahulgupta18)\ **Replies:** 1\ **Last updated:** [April 4, 2022, 6:30pm UTC](https://discuss.elastic.co/t/issue-with-logstash-1-7-17-1-1-service-on-ubuntu-18-04-3-lts/299460 "2022-04-04T18:30:06Z")
Hi Team, I have deployed a 2-node ELK stack cluster where I can successfully login to kibana instance with my superuser credentials. All of the ELK components (running on same Ubuntu VM) including metricbeats is runnin… --- ## [Unable to send data from metricbeat to logstash](https://discuss.elastic.co/t/unable-to-send-data-from-metricbeat-to-logstash/301523)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 1\ **Last updated:** [April 4, 2022, 5:46pm UTC](https://discuss.elastic.co/t/unable-to-send-data-from-metricbeat-to-logstash/301523 "2022-04-04T17:46:53Z")
Hello , Can someone help in this,I'm unable to send data from metricbeat to logstash. Below are the configurations: metricbeat.config.modules: path: ${path.config}/modules.d/\*.yml reload.enabled: false setup.kiban… --- ## [Logstash kv plugin is not working](https://discuss.elastic.co/t/logstash-kv-plugin-is-not-working/300548)
**Author:** [@mmk1995](https://discuss.elastic.co/u/mmk1995)\ **Replies:** 10\ **Last updated:** [April 4, 2022, 3:36pm UTC](https://discuss.elastic.co/t/logstash-kv-plugin-is-not-working/300548 "2022-04-04T15:36:42Z")
I am using ELK 7.6.2 The problem is when I remove the kv plugin, there is log loaded to Elasticsearch, when I add kv plugin, there is zero data can be loaded to Elasticsearch. Moreover, there is no error is logged in t… --- ## [Docker Compose file with 3 ES nodes Logstash and Kibana -updated 2022](https://discuss.elastic.co/t/docker-compose-file-with-3-es-nodes-logstash-and-kibana-updated-2022/301527)
**Author:** [@Lupul\_Dacic](https://discuss.elastic.co/u/Lupul_Dacic)\ **Replies:** 0\ **Last updated:** [April 4, 2022, 2:34pm UTC](https://discuss.elastic.co/t/docker-compose-file-with-3-es-nodes-logstash-and-kibana-updated-2022/301527 "2022-04-04T14:34:54Z")
Hi team, Could you please provide me an example of a docker-compose.yml file with an updated version without certs? I was able to find only obsolete version that in reality do not work correctly. In my case neither th… --- ## [Logstash queue.drain: true setting not being honored](https://discuss.elastic.co/t/logstash-queue-drain-true-setting-not-being-honored/301460)
**Author:** [@kannan\_raj](https://discuss.elastic.co/u/kannan_raj)\ **Replies:** 0\ **Last updated:** [April 4, 2022, 7:19am UTC](https://discuss.elastic.co/t/logstash-queue-drain-true-setting-not-being-honored/301460 "2022-04-04T07:19:33Z")
I've enabled the queue.drain: true setting on one of my logstash instances which has a persistent queue. The behavior I would expect is that when I stop the logstash process ( systemctl logstash stop ) the page files wo… --- ## [Beat(Nomad cluster\>Logstash\>Elasticsearch\>Graylog](https://discuss.elastic.co/t/beat-nomad-cluster-logstash-elasticsearch-graylog/300145)
**Author:** [@Uzmasaman\_Chanderki](https://discuss.elastic.co/u/Uzmasaman_Chanderki)\ **Replies:** 0\ **Last updated:** [March 20, 2022, 11:18pm UTC](https://discuss.elastic.co/t/beat-nomad-cluster-logstash-elasticsearch-graylog/300145 "2022-03-20T23:18:31Z")
I am new to Elasticsearch. I have filebeat running on Nomad cluster as system job. I have to collect logs and ship it to a centralized server which has Logstash, Elasticsearch and graylog running on docker-compose. Some … --- ## [Resurrect Connection to dead ES Instance](https://discuss.elastic.co/t/resurrect-connection-to-dead-es-instance/300265)
**Author:** [@karewise\_tech](https://discuss.elastic.co/u/karewise_tech)\ **Replies:** 0\ **Last updated:** [March 22, 2022, 5:08am UTC](https://discuss.elastic.co/t/resurrect-connection-to-dead-es-instance/300265 "2022-03-22T05:08:52Z")
\[logstash.outputs.Elasticsearch\]\[main\] Attempted to resurrect connection to dead ES instance, but got an error {:url=\>"http://elastic:xxxxxx@elasticsearch:9200/", :exception=\>LogStash::Outputs::Elasticsearch::HttpClient:… --- ## [Filebeat logstash encoding problem](https://discuss.elastic.co/t/filebeat-logstash-encoding-problem/301454)
**Author:** [@joie](https://discuss.elastic.co/u/joie)\ **Replies:** 0\ **Last updated:** [April 4, 2022, 6:33am UTC](https://discuss.elastic.co/t/filebeat-logstash-encoding-problem/301454 "2022-04-04T06:33:31Z")
Hi Thank you in advance. i am very stuggle to deal with this problem. push the oracle alert.log from the filebeat to logstash and see int ths slack message. my data source in alert log (Mon Apr 4 15:11:52 KST 2022 한글… --- ## [Docker Logstash Default(Bulit-in) plugins](https://discuss.elastic.co/t/docker-logstash-default-bulit-in-plugins/301451)
**Author:** [@leeseojune53](https://discuss.elastic.co/u/leeseojune53)\ **Replies:** 0\ **Last updated:** [April 4, 2022, 6:07am UTC](https://discuss.elastic.co/t/docker-logstash-default-bulit-in-plugins/301451 "2022-04-04T06:07:38Z")
I want know Logstash's default plugins Below is the list of plugins for logstash. Do you have any instructions on the basic plug-in? logstash-codec-avro logstash-codec-cef logstash-codec-collectd logstash-codec-dots l… --- ## [Logstash pipeline running but not loading the data](https://discuss.elastic.co/t/logstash-pipeline-running-but-not-loading-the-data/301427)
**Author:** [@acozme](https://discuss.elastic.co/u/acozme)\ **Replies:** 2\ **Last updated:** [April 3, 2022, 3:02pm UTC](https://discuss.elastic.co/t/logstash-pipeline-running-but-not-loading-the-data/301427 "2022-04-03T15:02:27Z")
Hello! I'm starting to learn Elastick Stack. Right now I'm trying to load some data from a csv file from a tutorial. I think everything is ok, the pipeline is running but no data is loaded to Elastic. This is the conf f… --- ## [The new variable in the filter ruby event must be set() before it can be used?](https://discuss.elastic.co/t/the-new-variable-in-the-filter-ruby-event-must-be-set-before-it-can-be-used/301406)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 2\ **Last updated:** [April 3, 2022, 4:08am UTC](https://discuss.elastic.co/t/the-new-variable-in-the-filter-ruby-event-must-be-set-before-it-can-be-used/301406 "2022-04-03T04:08:51Z")
ruby { code =\> " pubtime\_new = event.get('\[pubtime\]').to\_i \* 1000 event.set('pubtime\_new',pubtime\_new) \>\>\>\> If I don't register pubtime\_new then below can't get the value of pubtime\_new.… --- ## [Logstash JSON parser Error](https://discuss.elastic.co/t/logstash-json-parser-error/301389)
**Author:** [@Lupul\_Dacic](https://discuss.elastic.co/u/Lupul_Dacic)\ **Replies:** 5\ **Last updated:** [April 2, 2022, 5:41pm UTC](https://discuss.elastic.co/t/logstash-json-parser-error/301389 "2022-04-02T17:41:38Z")
Hi guys, I am new to ELK and I need your help in order to parse a JSON file with Logstash. I want to make it available in Kibana. My logstash.conf is input { file { path =\> "/usr/share/logstash/c.json" star… --- ## [Sending Data from Metricbeat to Logstash not working](https://discuss.elastic.co/t/sending-data-from-metricbeat-to-logstash-not-working/301320)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 3\ **Last updated:** [April 2, 2022, 4:27pm UTC](https://discuss.elastic.co/t/sending-data-from-metricbeat-to-logstash-not-working/301320 "2022-04-02T16:27:15Z")
Hello All, In current implementation the data is going well from metricbeat to elastic but now while changing it from metricbeat-\>logstash-\>elastic it dosent works,data is not received by logstash.Can someone help wher… --- ## [Mutate replace using if](https://discuss.elastic.co/t/mutate-replace-using-if/301393)
**Author:** [@travlest](https://discuss.elastic.co/u/travlest)\ **Replies:** 0\ **Last updated:** [April 2, 2022, 3:09pm UTC](https://discuss.elastic.co/t/mutate-replace-using-if/301393 "2022-04-02T15:09:05Z")
Hi, I have a question regarding updating and removing logstash. My situation is, that I am collecting logs using API, for the first time data is being shipped into logstash, the "GeoIP" is empty space, but it will have u… --- ## [Logstash-keystore error](https://discuss.elastic.co/t/logstash-keystore-error/301216)
**Author:** [@jack3368](https://discuss.elastic.co/u/jack3368)\ **Replies:** 1\ **Last updated:** [April 1, 2022, 7:49pm UTC](https://discuss.elastic.co/t/logstash-keystore-error/301216 "2022-04-01T19:49:35Z")
I am upgrading logstash v7.5.2 to v7.17.1. The step in docker file that using logstash-keystore to create keystore always return the following error message: Using bundled JDK: /usr/share/logstash/jdk OpenJDK 64-Bit S… --- ## [How to improve Logstash configuration for outputs](https://discuss.elastic.co/t/how-to-improve-logstash-configuration-for-outputs/301332)
**Author:** [@alytkowski](https://discuss.elastic.co/u/alytkowski)\ **Replies:** 1\ **Last updated:** [April 1, 2022, 4:25pm UTC](https://discuss.elastic.co/t/how-to-improve-logstash-configuration-for-outputs/301332 "2022-04-01T16:25:27Z")
Dear team, can you advise on how to improve my configuration for outputs from Logstash? In our environment, the only output is Elasticsearch, and we have around 30 sources which are sent to Elasticsearch from logstash … --- ## [Parse nested json - Logstash v16.2 on Windows 10](https://discuss.elastic.co/t/parse-nested-json-logstash-v16-2-on-windows-10/301296)
**Author:** [@Ely\_96](https://discuss.elastic.co/u/Ely_96)\ **Replies:** 1\ **Last updated:** [April 1, 2022, 4:15pm UTC](https://discuss.elastic.co/t/parse-nested-json-logstash-v16-2-on-windows-10/301296 "2022-04-01T16:15:36Z")
Hi guys, I'm trying to import a nested json into Elasticsearch v7.16, but I haven't been successful so far. I managed successfully to import a "normal" (not nested) json, but for the nested one I'm in trouble. Could you … --- ## [Logstash pipeline development tool 2022](https://discuss.elastic.co/t/logstash-pipeline-development-tool-2022/301325)
**Author:** [@epacke](https://discuss.elastic.co/u/epacke)\ **Replies:** 0\ **Last updated:** [April 1, 2022, 1:16pm UTC](https://discuss.elastic.co/t/logstash-pipeline-development-tool-2022/301325 "2022-04-01T13:16:31Z")
Hi there! It's time for the yearly release. :slight\_smile: For those that have not seen it it's a tool to help with the development of logstash filters. You simply enter raw log lines into an input field, choose which … --- ## [Enrichment via Elasticsearch Lookup](https://discuss.elastic.co/t/enrichment-via-elasticsearch-lookup/301295)
**Author:** [@shinobu](https://discuss.elastic.co/u/shinobu)\ **Replies:** 1\ **Last updated:** [April 1, 2022, 10:37am UTC](https://discuss.elastic.co/t/enrichment-via-elasticsearch-lookup/301295 "2022-04-01T10:37:05Z")
Hello, I tried to enrich alerting information with data from a logindex. In Logstash I configured the following as a filter: elasticsearch { hosts =\> \["http://localhost:9200"\] index =\> "logfile" query =\> … --- ## [Is logstash 7.16.2 effected with CVE-2022-22965.A remote code execution vulnerability](https://discuss.elastic.co/t/is-logstash-7-16-2-effected-with-cve-2022-22965-a-remote-code-execution-vulnerability/301305)
**Author:** [@rama543](https://discuss.elastic.co/u/rama543)\ **Replies:** 0\ **Last updated:** [April 1, 2022, 10:09am UTC](https://discuss.elastic.co/t/is-logstash-7-16-2-effected-with-cve-2022-22965-a-remote-code-execution-vulnerability/301305 "2022-04-01T10:09:54Z")
We deployed Logstash 7.16.2 version in amazon EC2 instances . Is logstash imacted with this new vulnerability CVE-2022-22965(Spring4shell) A remote code execution vulnerability Thanks [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=148) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=150)