# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=290 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 291 --- ## [Not able to process UDP traffic](https://discuss.elastic.co/t/not-able-to-process-udp-traffic/252838)
**Author:** [@joao.matias](https://discuss.elastic.co/u/joao.matias)\ **Replies:** 0\ **Last updated:** [October 21, 2020, 12:31pm UTC](https://discuss.elastic.co/t/not-able-to-process-udp-traffic/252838 "2020-10-21T12:31:35Z")
Hello everyone. I have a host with the following specs CentOS 7 16GB RAM 2 CPUs 100GB storage 2 network interfaces And I'm trying to set up logstash to receive and filter the following types of logs Netflow over UD… --- ## [Weird logs pushed by logstash after going secure](https://discuss.elastic.co/t/weird-logs-pushed-by-logstash-after-going-secure/250179)
**Author:** [@Kwiskas](https://discuss.elastic.co/u/Kwiskas)\ **Replies:** 4\ **Last updated:** [October 21, 2020, 10:29am UTC](https://discuss.elastic.co/t/weird-logs-pushed-by-logstash-after-going-secure/250179 "2020-10-21T10:29:18Z")
Hello Our elk stack is on version 7.6.2 A few days ago we pushed on our team logstash as secure to enable x-pack we dit same for elastic nodes also kibana Since we did that logstash is sending weird logs and paramete… --- ## [Logstash-7.8.1 - Couldn't connect MySQL jdbc on plugin logstash-integration-jdbc (5.0.5)](https://discuss.elastic.co/t/logstash-7-8-1-couldnt-connect-mysql-jdbc-on-plugin-logstash-integration-jdbc-5-0-5/252767)
**Author:** [@matthewdeepblue](https://discuss.elastic.co/u/matthewdeepblue)\ **Replies:** 2\ **Last updated:** [October 21, 2020, 7:57am UTC](https://discuss.elastic.co/t/logstash-7-8-1-couldnt-connect-mysql-jdbc-on-plugin-logstash-integration-jdbc-5-0-5/252767 "2020-10-21T07:57:13Z")
Hi guys. I got error message below when I tried to connect MySQL RDS through logstash plugin. please post here if you guys have any idea. Thanks. \[2020-10-21T04:27:23,538\]\[ERROR\]\[logstash.inputs.jdbc \] Unable to … --- ## [To push updated data from mongodb using logstash logstash-input-mongodb plugin](https://discuss.elastic.co/t/to-push-updated-data-from-mongodb-using-logstash-logstash-input-mongodb-plugin/252791)
**Author:** [@Durga\_AK](https://discuss.elastic.co/u/Durga_AK)\ **Replies:** 0\ **Last updated:** [October 21, 2020, 7:46am UTC](https://discuss.elastic.co/t/to-push-updated-data-from-mongodb-using-logstash-logstash-input-mongodb-plugin/252791 "2020-10-21T07:46:01Z")
Im trying to sync updated data from mongodb using the logstash-input-mongodb plugin. New data inserted into mongodb is getting indexed in Elastic search. Already existing data and updated data is not moving to Elastic se… --- ## [Data Loss with HTTP Input Plugin](https://discuss.elastic.co/t/data-loss-with-http-input-plugin/252782)
**Author:** [@mbrill](https://discuss.elastic.co/u/mbrill)\ **Replies:** 0\ **Last updated:** [October 21, 2020, 7:11am UTC](https://discuss.elastic.co/t/data-loss-with-http-input-plugin/252782 "2020-10-21T07:11:55Z")
Hi, i'm currently running into the following issue with the http input plugin for logstash: Whenever a request is sent to the pipeline while the pipeline is busy, this request is dropped. I have read about the blocking… --- ## [How to store & get data with different time interval](https://discuss.elastic.co/t/how-to-store-get-data-with-different-time-interval/252775)
**Author:** [@James\_Liu](https://discuss.elastic.co/u/James_Liu)\ **Replies:** 0\ **Last updated:** [October 21, 2020, 6:04am UTC](https://discuss.elastic.co/t/how-to-store-get-data-with-different-time-interval/252775 "2020-10-21T06:04:30Z")
Our team are currently focusing on log datas and visualizing them. We are building a platform analyzing every audit log by log4j in our java project: log -\> filebeat -\> logstash -\> elasticsearch -\> our platform CURREN… --- ## [SSL options not working in logstash-output-mongodb (LS v7.9.0)](https://discuss.elastic.co/t/ssl-options-not-working-in-logstash-output-mongodb-ls-v7-9-0/252481)
**Author:** [@ksathya](https://discuss.elastic.co/u/ksathya)\ **Replies:** 4\ **Last updated:** [October 21, 2020, 5:01am UTC](https://discuss.elastic.co/t/ssl-options-not-working-in-logstash-output-mongodb-ls-v7-9-0/252481 "2020-10-21T05:01:33Z")
I am using logstash v7.9.0 for Windows. I am using logstash-output-mongodb plugin v 3.1.5. My output code is as below: mongodb { collection =\> "TestColection" database =\> "TestDB" … --- ## [Grok filter by source](https://discuss.elastic.co/t/grok-filter-by-source/252623)
**Author:** [@andi](https://discuss.elastic.co/u/andi)\ **Replies:** 4\ **Last updated:** [October 21, 2020, 4:49am UTC](https://discuss.elastic.co/t/grok-filter-by-source/252623 "2020-10-21T04:49:48Z")
Hi All, Sorry i'm new on ELK, currently i'm use ELK 6.8.10, it's working properly, with filter by tags, this is my grok. filter { if "nameservice-out-staging" in \[tags\] { grok { match =\> { "message" =\> … --- ## [Problem with parsing json in syslog format](https://discuss.elastic.co/t/problem-with-parsing-json-in-syslog-format/252042)
**Author:** [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\ **Replies:** 4\ **Last updated:** [October 21, 2020, 1:16am UTC](https://discuss.elastic.co/t/problem-with-parsing-json-in-syslog-format/252042 "2020-10-21T01:16:34Z")
Hi, can you please help me with logstash config? input { #tcp { # port =\> 5014 # type =\> syslog #} udp { port =\> 5014 type =\> syslog } } filter { … --- ## [Multiple entries for single file in sincedb](https://discuss.elastic.co/t/multiple-entries-for-single-file-in-sincedb/252438)
**Author:** [@chandramouli\_sriniva](https://discuss.elastic.co/u/chandramouli_sriniva)\ **Replies:** 14\ **Last updated:** [October 20, 2020, 9:29pm UTC](https://discuss.elastic.co/t/multiple-entries-for-single-file-in-sincedb/252438 "2020-10-20T21:29:21Z")
Hi, I read a single file from server(C:/Users/xx.log - its a rotating log file daily) and route to kafka. In a single sincedb file, we see multiple entries like below. Is this valid. we see dupes on Kafka topic. Is it … --- ## [Known Issue Logstash 7.7.1-7.9.1: Logstash Keystore may fail when used with multiple pipelines](https://discuss.elastic.co/t/known-issue-logstash-7-7-1-7-9-1-logstash-keystore-may-fail-when-used-with-multiple-pipelines/252711)
**Author:** [@RobBavey](https://discuss.elastic.co/u/RobBavey)\ **Replies:** 3\ **Last updated:** [October 20, 2020, 7:51pm UTC](https://discuss.elastic.co/t/known-issue-logstash-7-7-1-7-9-1-logstash-keystore-may-fail-when-used-with-multiple-pipelines/252711 "2020-10-20T19:51:44Z")
A performance related change made in Logstash 7.7.1 may cause Logstash configurations with multiple pipelines to fail with the following error: "Unable to configure plugins: (ConfigurationError) Cannot evaluate \`${varia… --- ## [Logstash pipelines dose not work independently](https://discuss.elastic.co/t/logstash-pipelines-dose-not-work-independently/252704)
**Author:** [@chamiduz](https://discuss.elastic.co/u/chamiduz)\ **Replies:** 1\ **Last updated:** [October 20, 2020, 3:00pm UTC](https://discuss.elastic.co/t/logstash-pipelines-dose-not-work-independently/252704 "2020-10-20T15:00:18Z")
Hi, I want to implement a logstash pipeline to parallelly send unprocessed data to s3 and processed data to ES. I currently have this implementation but data is not publishing to s3 when ES is not responding. how can … --- ## [\[ERROR\]\[logstash.pipeline \] Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"\\\"/usr/share/logstash/logstash-core/lib/jars/sqljdbc42.jar\\", jdbc\_connection\_string](https://discuss.elastic.co/t/error-logstash-pipeline-error-registering-plugin-pipeline-id-main-plugin-logstash-jdbc-jdbc-driver-library-usr-share-logstash-logstash-core-lib-jars-sqljdbc42-jar-jdbc-connection-string/252562)
**Author:** [@lynda](https://discuss.elastic.co/u/lynda)\ **Replies:** 11\ **Last updated:** [October 20, 2020, 2:27pm UTC](https://discuss.elastic.co/t/error-logstash-pipeline-error-registering-plugin-pipeline-id-main-plugin-logstash-jdbc-jdbc-driver-library-usr-share-logstash-logstash-core-lib-jars-sqljdbc42-jar-jdbc-connection-string/252562 "2020-10-20T14:27:14Z")
Hello, I have been trying to run logstash as a service but it keeps failing then restarting in a loop. I tried looking for a solution in similar topics without finding anything useful in my case. Here are the logs when… --- ## [Configuring Logstash to index nested xml data](https://discuss.elastic.co/t/configuring-logstash-to-index-nested-xml-data/252692)
**Author:** [@andrew.laraia](https://discuss.elastic.co/u/andrew.laraia)\ **Replies:** 1\ **Last updated:** [October 20, 2020, 1:55pm UTC](https://discuss.elastic.co/t/configuring-logstash-to-index-nested-xml-data/252692 "2020-10-20T13:55:36Z")
With Logstash, is there a way to index nested serialized XML using the XML plugin? Can you provide an example? We have some data that we migrate from an existing SQL database via Logstash. These records have a handful o… --- ## [Logstash](https://discuss.elastic.co/t/logstash/252689)
**Author:** [@Syed.Ubaid](https://discuss.elastic.co/u/Syed.Ubaid)\ **Replies:** 0\ **Last updated:** [October 20, 2020, 1:24pm UTC](https://discuss.elastic.co/t/logstash/252689 "2020-10-20T13:24:56Z")
After installing xpack logstash is giving error my configuration file is input { file { path =\> "D:/csv/countriesdata.csv" start\_position =\> "beginning" sincedb\_path =\> "NUL" } } filter { csv { separator =\> ",… --- ## [Need help with a 2XX HTTP code 400 Error](https://discuss.elastic.co/t/need-help-with-a-2xx-http-code-400-error/252682)
**Author:** [@Ratch54](https://discuss.elastic.co/u/Ratch54)\ **Replies:** 0\ **Last updated:** [October 20, 2020, 12:46pm UTC](https://discuss.elastic.co/t/need-help-with-a-2xx-http-code-400-error/252682 "2020-10-20T12:46:31Z")
I am receiving the \[ERROR\]\[logstash.outputs.http\]\[HTTP Output Failure\] Encountered non-2XX HTTP code 400 {:response\_code=\>400, :url"https://some url:8086/write?db=db", :event=\>%{host} %{message}, will\_retry=\>false The c… --- ## [How to extract data from Json](https://discuss.elastic.co/t/how-to-extract-data-from-json/252595)
**Author:** [@rildo](https://discuss.elastic.co/u/rildo)\ **Replies:** 4\ **Last updated:** [October 20, 2020, 12:25pm UTC](https://discuss.elastic.co/t/how-to-extract-data-from-json/252595 "2020-10-20T12:25:36Z")
input { stdin { } } output { stdout { codec =\> rubydebug } } filter { json { source =\> "message" } mutate { gsub =\> \[ "message" , "\\n", "," \] } mutate { gsub =\> \[ "message" , "\[\\\]", "" \] gsub =\> \[ "other\_… --- ## [JMX Input Plugin - Error in connecting to Weblogic JMX](https://discuss.elastic.co/t/jmx-input-plugin-error-in-connecting-to-weblogic-jmx/252670)
**Author:** [@abidub](https://discuss.elastic.co/u/abidub)\ **Replies:** 0\ **Last updated:** [October 20, 2020, 11:30am UTC](https://discuss.elastic.co/t/jmx-input-plugin-error-in-connecting-to-weblogic-jmx/252670 "2020-10-20T11:30:20Z")
Hi there, I am using the JMX Input Plugin and trying to connect to a Weblogic JMX service URL and getting the following error: \[ERROR\]\[logstash.inputs.jmx\] "failed to retrieve RMIServe stub: javax.naming.NoInitialConte… --- ## [Can Logstash implement a data flow from Elastic to other database?](https://discuss.elastic.co/t/can-logstash-implement-a-data-flow-from-elastic-to-other-database/252672)
**Author:** [@Fernando1](https://discuss.elastic.co/u/Fernando1)\ **Replies:** 0\ **Last updated:** [October 20, 2020, 11:45am UTC](https://discuss.elastic.co/t/can-logstash-implement-a-data-flow-from-elastic-to-other-database/252672 "2020-10-20T11:45:33Z")
Hello!! How are you? Can Logstash implement a reverse flow? Like instead of pulling data from, for example, PostgreSQL into Elasticsearch, do the contrary? Like pulling from Elasticsearch into PostgreSQL? Best Wishes, … --- ## [Syslog plugin multiple grok](https://discuss.elastic.co/t/syslog-plugin-multiple-grok/252665)
**Author:** [@mutt13y](https://discuss.elastic.co/u/mutt13y)\ **Replies:** 0\ **Last updated:** [October 20, 2020, 10:27am UTC](https://discuss.elastic.co/t/syslog-plugin-multiple-grok/252665 "2020-10-20T10:27:13Z")
Hi, The syslog plugin parameter grok\_pattern only accepts a string. Is there a reason it cant be changed to accept a list ? Currently we have to trap for parse failure and parse alternate patterns manually. --- ## [LogStash::Json::ParserError](https://discuss.elastic.co/t/logstash-parsererror/250809)
**Author:** [@Raiderume](https://discuss.elastic.co/u/Raiderume)\ **Replies:** 9\ **Last updated:** [October 20, 2020, 9:08am UTC](https://discuss.elastic.co/t/logstash-parsererror/250809 "2020-10-20T09:08:59Z")
Hello, I have problems with parsing json logs to logstash, there are tons of errors at log: Error parsing json {:source=\>"message", :raw=\>"\*\*\*\*", :exception=\>#\ **Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 3\ **Last updated:** [October 20, 2020, 9:07am UTC](https://discuss.elastic.co/t/deletion-of-indexed-documents-while-doing-the-indexing-with-action-set-to-index/251846 "2020-10-20T09:07:30Z") I have seen a peculiar behavior while using the logstash JDBC input plugin and Elasticsearch output plugin. Initial indexing has been completed and was trying to do the partial update for few columns. When I have sent … --- ## [Multilple field names are not getting renamed using logstash](https://discuss.elastic.co/t/multilple-field-names-are-not-getting-renamed-using-logstash/252646)
**Author:** [@abhay\_deshmukh](https://discuss.elastic.co/u/abhay_deshmukh)\ **Replies:** 0\ **Last updated:** [October 20, 2020, 8:04am UTC](https://discuss.elastic.co/t/multilple-field-names-are-not-getting-renamed-using-logstash/252646 "2020-10-20T08:04:10Z")
mutate { rename =\> { "10" =\> "CheckSum" "103" =\> "OrdRejReason" "107" =\> "SecurityDesc" "108" =\> "HeartBtInt" "109" =\> "ClientID" "11" =\> "ClOrdID" "112" =\> "TestReqID" } I'm trying to rename these fields i tri… --- ## [\[ERROR\]\[logstash.outputs.elasticsearch\] Failed to install template. {:message=\>"Got response code '400' contacting Elasticsearch at URL 'http://localhost:9200/\_template/jose\_prueba\_v14'"](https://discuss.elastic.co/t/error-logstash-outputs-elasticsearch-failed-to-install-template-message-got-response-code-400-contacting-elasticsearch-at-url-http-localhost-9200-template-jose-prueba-v14/252582)
**Author:** [@joseantonio.sanchez](https://discuss.elastic.co/u/joseantonio.sanchez)\ **Replies:** 3\ **Last updated:** [October 20, 2020, 7:54am UTC](https://discuss.elastic.co/t/error-logstash-outputs-elasticsearch-failed-to-install-template-message-got-response-code-400-contacting-elasticsearch-at-url-http-localhost-9200-template-jose-prueba-v14/252582 "2020-10-20T07:54:26Z")
Good afternoon, I'm having problems with logstash when putting a template file to perform a mapping. in my logstash .conf file, I put "output { elasticsearch { hosts =\> \['localhost: 9200'\] index =\> 'jose\_test\_v14' … --- ## [Extract first 50 characters from message "body" into new field](https://discuss.elastic.co/t/extract-first-50-characters-from-message-body-into-new-field/252574)
**Author:** [@SecretSquizza](https://discuss.elastic.co/u/SecretSquizza)\ **Replies:** 4\ **Last updated:** [October 20, 2020, 7:44am UTC](https://discuss.elastic.co/t/extract-first-50-characters-from-message-body-into-new-field/252574 "2020-10-20T07:44:15Z")
Hello everyone, Firstly, apologies for any formatting mistakes in this post - its my first one... i'm relatively new to the Elastic stack and need to get a shortened version of the message body into another field so it… --- ## [Using json in logstash](https://discuss.elastic.co/t/using-json-in-logstash/252643)
**Author:** [@igersht](https://discuss.elastic.co/u/igersht)\ **Replies:** 0\ **Last updated:** [October 20, 2020, 7:08am UTC](https://discuss.elastic.co/t/using-json-in-logstash/252643 "2020-10-20T07:08:37Z")
Hey, I'm trying to send JSON to logstash looking like this: "build\_url" =\> "https://www.build.url", "fields" =\> { "log\_source" =\> "log\_json" }, "@timestamp" =\> 2020-10-19T11:… --- ## [Delete Sync Operation using Logstash JDBC](https://discuss.elastic.co/t/delete-sync-operation-using-logstash-jdbc/252636)
**Author:** [@sharath\_sai](https://discuss.elastic.co/u/sharath_sai)\ **Replies:** 0\ **Last updated:** [October 20, 2020, 6:28am UTC](https://discuss.elastic.co/t/delete-sync-operation-using-logstash-jdbc/252636 "2020-10-20T06:28:26Z")
Hi all, We have configured JDBC logstash, to migrate data from Cassandra to logstash. we have able to move data from Cassandra to logstash, we are also updating the data in elastic search based on document\_id, but we n… --- ## [Logstash elasticsearch input plugin observes frequent Faraday::TimeoutError. How can we increase the default timeout settings for the request?](https://discuss.elastic.co/t/logstash-elasticsearch-input-plugin-observes-frequent-faraday-timeouterror-how-can-we-increase-the-default-timeout-settings-for-the-request/252621)
**Author:** [@vikrant\_baraiya1](https://discuss.elastic.co/u/vikrant_baraiya1)\ **Replies:** 0\ **Last updated:** [October 20, 2020, 2:37am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input-plugin-observes-frequent-faraday-timeouterror-how-can-we-increase-the-default-timeout-settings-for-the-request/252621 "2020-10-20T02:37:42Z")
We are using Logstash with an elastic search input plugin and elastic search output plugin. The data volume increased heavily during the peak hours between 10 AM - 5 PM Eastern time. And during this time frame, we observ… --- ## [How to setup logstash to send whole xml event log to elasticsearch](https://discuss.elastic.co/t/how-to-setup-logstash-to-send-whole-xml-event-log-to-elasticsearch/252098)
**Author:** [@muru1](https://discuss.elastic.co/u/muru1)\ **Replies:** 15\ **Last updated:** [October 20, 2020, 12:26am UTC](https://discuss.elastic.co/t/how-to-setup-logstash-to-send-whole-xml-event-log-to-elasticsearch/252098 "2020-10-20T00:26:04Z")
I am trying to setup logstash to send xml event logs to elasticseach, however it is inserting each line as a separate entry into es, I tried using multiline codec but its buffering that many lines and inserting into es i… --- ## [Unable to start Logstash docker](https://discuss.elastic.co/t/unable-to-start-logstash-docker/252614)
**Author:** [@tamilarasanbravo](https://discuss.elastic.co/u/tamilarasanbravo)\ **Replies:** 0\ **Last updated:** [October 19, 2020, 9:26pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-docker/252614 "2020-10-19T21:26:05Z")
Hi Team, I am unable to start my logstash docker stack and below is the config and log Config logstash: image: logstash:7.9.1 hostname: "{{.Node.Hostname}}-logstash" environment: - XPACK\_MONITORING… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=289) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=291)