# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=407 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 408 --- ## [Make logstash upload then exit](https://discuss.elastic.co/t/make-logstash-upload-then-exit/218421)
**Author:** [@Goishin](https://discuss.elastic.co/u/Goishin)\ **Replies:** 2\ **Last updated:** [February 10, 2020, 3:59pm UTC](https://discuss.elastic.co/t/make-logstash-upload-then-exit/218421 "2020-02-10T15:59:02Z")
I've got what appears to be a unique situation. I am using Kibana to dashboard test pass data after a dev commits code. Because of that, my log data isn't received in real time. I basically have a file with all my loggin… --- ## [Grok pattern doesn't match path](https://discuss.elastic.co/t/grok-pattern-doesnt-match-path/218592)
**Author:** [@hrmzaqa](https://discuss.elastic.co/u/hrmzaqa)\ **Replies:** 5\ **Last updated:** [February 10, 2020, 3:27pm UTC](https://discuss.elastic.co/t/grok-pattern-doesnt-match-path/218592 "2020-02-10T15:27:59Z")
Hi, I have a problem that I cannot solve for too long so I'm using this forum to get a help :slight\_smile: i need to to get name of application server parsed from path. I have this simple pattern but i'm getting "\_gro… --- ## [ArgumentError when consuming a message with the jms plugin](https://discuss.elastic.co/t/argumenterror-when-consuming-a-message-with-the-jms-plugin/217398)
**Author:** [@ghost\_recon](https://discuss.elastic.co/u/ghost_recon)\ **Replies:** 4\ **Last updated:** [February 10, 2020, 3:03pm UTC](https://discuss.elastic.co/t/argumenterror-when-consuming-a-message-with-the-jms-plugin/217398 "2020-02-10T15:03:49Z")
Hello everyone, I'am new to logstash and i've been trying to establish using the jms plugin a connection to a jboss queue and it worked, but whenever a message is been consumed instead of displaying it logstash throws t… --- ## [High availability of logstash using jdbc plugin](https://discuss.elastic.co/t/high-availability-of-logstash-using-jdbc-plugin/216043)
**Author:** [@naveenbangalore](https://discuss.elastic.co/u/naveenbangalore)\ **Replies:** 2\ **Last updated:** [February 10, 2020, 2:52pm UTC](https://discuss.elastic.co/t/high-availability-of-logstash-using-jdbc-plugin/216043 "2020-02-10T14:52:01Z")
I have a 2 node logstash setup populating the elastic cluster. There are few configs that use the jdbc plugin in the input. Is there a way to setup high availability among the logstash nodes when one of them fails? My qu… --- ## [Logstash not formatting date to yyyyMMdd](https://discuss.elastic.co/t/logstash-not-formatting-date-to-yyyymmdd/218506)
**Author:** [@krishna\_bhargav](https://discuss.elastic.co/u/krishna_bhargav)\ **Replies:** 1\ **Last updated:** [February 10, 2020, 2:51pm UTC](https://discuss.elastic.co/t/logstash-not-formatting-date-to-yyyymmdd/218506 "2020-02-10T14:51:10Z")
Here is the snippet, the date in load file comes as string as 2020-02-06-07.51.19.413216, we need to load it to elastic in format yyyyMMdd. But the below code only converts String to date and not the format. filter { … --- ## [Date filter cause docker logstash to crash](https://discuss.elastic.co/t/date-filter-cause-docker-logstash-to-crash/218611)
**Author:** [@zebulyon](https://discuss.elastic.co/u/zebulyon)\ **Replies:** 0\ **Last updated:** [February 10, 2020, 2:45pm UTC](https://discuss.elastic.co/t/date-filter-cause-docker-logstash-to-crash/218611 "2020-02-10T14:45:54Z")
Hi, Adding the date filter causes Logstash to crash. I'm using the premade Logstash docker image, docker.elastic.co/logstash/logstash:7.5.2, I have managed to get the grok filter to work just fine, but when I try to ad… --- ## [Logstash ingestion performance for log management](https://discuss.elastic.co/t/logstash-ingestion-performance-for-log-management/217435)
**Author:** [@AlessandroKP](https://discuss.elastic.co/u/AlessandroKP)\ **Replies:** 6\ **Last updated:** [February 10, 2020, 10:58am UTC](https://discuss.elastic.co/t/logstash-ingestion-performance-for-log-management/217435 "2020-02-10T10:58:54Z")
Hi guys, I have some issues with the performance of the ingestion of data into Elasticsearch through Logstash. My ingestion setup is the following: Data ingested are log lines read from file through Filebeat Log line… --- ## [Unable to start logstash configuration file](https://discuss.elastic.co/t/unable-to-start-logstash-configuration-file/218503)
**Author:** [@Charles\_Yuliansen](https://discuss.elastic.co/u/Charles_Yuliansen)\ **Replies:** 4\ **Last updated:** [February 10, 2020, 8:28am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-configuration-file/218503 "2020-02-10T08:28:57Z")
this is my first time running logstash on container. Im running logstash on the same container elasticsearch + kibana. Its running on ubuntu. i run my conf file by using /usr/share/logstash/bin/logstash -f conf.d/logs… --- ## [How to retrieve headers from udp data via input plugin logstash](https://discuss.elastic.co/t/how-to-retrieve-headers-from-udp-data-via-input-plugin-logstash/218496)
**Author:** [@sukku77](https://discuss.elastic.co/u/sukku77)\ **Replies:** 0\ **Last updated:** [February 10, 2020, 1:45am UTC](https://discuss.elastic.co/t/how-to-retrieve-headers-from-udp-data-via-input-plugin-logstash/218496 "2020-02-10T01:45:27Z")
Hi All, Can someone advice the options to retrieve the source IP information for UDP data. We are currently getting the data via input udp plugin from logstash. Usually, source ip address info comes as part of udp heade… --- ## [\[Logstash\]\[jdbc input plugin\]sql\_last\_value with timestamp type support milliseconds?](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-sql-last-value-with-timestamp-type-support-milliseconds/218520)
**Author:** [@ztkmkoo](https://discuss.elastic.co/u/ztkmkoo)\ **Replies:** 0\ **Last updated:** [February 10, 2020, 6:33am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-sql-last-value-with-timestamp-type-support-milliseconds/218520 "2020-02-10T06:33:43Z")
Hi, I try to collect my data from mysql (or cubrid) to elasticsearch and hadoop. (test with logstash 7.5.2 on my local windows 10 desktop) And found sql\_last\_value is not worked as i expected. Its my config input { … --- ## [Create 1 document from multiple documents](https://discuss.elastic.co/t/create-1-document-from-multiple-documents/218519)
**Author:** [@hofrichterovak](https://discuss.elastic.co/u/hofrichterovak)\ **Replies:** 0\ **Last updated:** [February 10, 2020, 6:27am UTC](https://discuss.elastic.co/t/create-1-document-from-multiple-documents/218519 "2020-02-10T06:27:52Z")
Hello, I have an input field that contains values. For further processing I need to divide this field with a separator (";") and then work with it. In my case, this is the text of the script in which I search for consta… --- ## [Restore arcsight\_logs\_template index template created by ArcSight Module?](https://discuss.elastic.co/t/restore-arcsight-logs-template-index-template-created-by-arcsight-module/218103)
**Author:** [@undelete](https://discuss.elastic.co/u/undelete)\ **Replies:** 3\ **Last updated:** [February 10, 2020, 6:01am UTC](https://discuss.elastic.co/t/restore-arcsight-logs-template-index-template-created-by-arcsight-module/218103 "2020-02-10T06:01:19Z")
Hi, not sure if Elasticsearch is the correct category. However I used the Dev Tools in Kibana to update the pre-created index template arcsight\_logs\_template, created by the ArcSight Module. I updated the number\_of\_repli… --- ## [Logstash Configuarion file issue](https://discuss.elastic.co/t/logstash-configuarion-file-issue/218346)
**Author:** [@Syed.Ubaid](https://discuss.elastic.co/u/Syed.Ubaid)\ **Replies:** 9\ **Last updated:** [February 10, 2020, 5:06am UTC](https://discuss.elastic.co/t/logstash-configuarion-file-issue/218346 "2020-02-10T05:06:44Z")
I have connect elk stack with mysql configuration file showing error . My configuration file is following input { jdbc { jdbc\_connection\_string =\> "jdbc:mysql://localhost:3306/db\_ublfunds\_cc" jdbc\_user =\> "abcd" jd… --- ## [IMAP Input Plugin and Gmail](https://discuss.elastic.co/t/imap-input-plugin-and-gmail/218155)
**Author:** [@manunc](https://discuss.elastic.co/u/manunc)\ **Replies:** 3\ **Last updated:** [February 10, 2020, 2:46am UTC](https://discuss.elastic.co/t/imap-input-plugin-and-gmail/218155 "2020-02-10T02:46:23Z")
This question has already been posted but wihtout answer. I got this error when tryiing to use IMAP plugin in order to collect email from Gmail Imap SSL account: \[2020-02-06T22:04:16,642\]\[ERROR\]\[logstash.inputs.imap … --- ## [There is Expected one of \[ \\\\t\\\\r\\\\n\] error in docker](https://discuss.elastic.co/t/there-is-expected-one-of-t-r-n-error-in-docker/218287)
**Author:** [@fuzes](https://discuss.elastic.co/u/fuzes)\ **Replies:** 4\ **Last updated:** [February 10, 2020, 2:23am UTC](https://discuss.elastic.co/t/there-is-expected-one-of-t-r-n-error-in-docker/218287 "2020-02-10T02:23:53Z")
when i running logstash in mac os, it works well. but when i running docker container with same logstash.conf file, logstash container die with "exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of \[ \\… --- ## [Why won't Logstash rename a parsed JSON field?](https://discuss.elastic.co/t/why-wont-logstash-rename-a-parsed-json-field/218447)
**Author:** [@Sylancer](https://discuss.elastic.co/u/Sylancer)\ **Replies:** 6\ **Last updated:** [February 10, 2020, 12:04am UTC](https://discuss.elastic.co/t/why-wont-logstash-rename-a-parsed-json-field/218447 "2020-02-10T00:04:24Z")
So I've got some Zeek logs. In particular I've got http.log which Filebeat is adding the tag "Zeek-HTTP" to. That's all working fine. I want to use the JSON plugin to parse the "message" field in Logstash to a new target… --- ## [Parsed JSON object/hash requires a target configuration option with json message](https://discuss.elastic.co/t/parsed-json-object-hash-requires-a-target-configuration-option-with-json-message/218472)
**Author:** [@richzw](https://discuss.elastic.co/u/richzw)\ **Replies:** 1\ **Last updated:** [February 9, 2020, 4:06pm UTC](https://discuss.elastic.co/t/parsed-json-object-hash-requires-a-target-configuration-option-with-json-message/218472 "2020-02-09T16:06:54Z")
We met the following warning message as below under logstash 7.3 Parsed JSON object/hash requires a target configuration option {:source=\>"message", :raw=\>"\[{\\"itemBefore\\":0,\\"itemAward\\":1,\\"itemType\\":\\"hint\\"}\]"} a… --- ## [Logstash.inputs.twitter Twitter client error Connection reset exception IO Error](https://discuss.elastic.co/t/logstash-inputs-twitter-twitter-client-error-connection-reset-exception-io-error/218462)
**Author:** [@shakesperoo](https://discuss.elastic.co/u/shakesperoo)\ **Replies:** 0\ **Last updated:** [February 9, 2020, 8:08am UTC](https://discuss.elastic.co/t/logstash-inputs-twitter-twitter-client-error-connection-reset-exception-io-error/218462 "2020-02-09T08:08:55Z")
I keep get below error from time to time with logstash twitter input and i dont know whats the issue or what shall i do ! \[2020-02-08T21:55:33,230\]\[WARN \]\[logstash.inputs.twitter \]\[main\] Twitter client error {:message… --- ## [Pattern works on the grok debugger but fails in logstash](https://discuss.elastic.co/t/pattern-works-on-the-grok-debugger-but-fails-in-logstash/218410)
**Author:** [@moderable](https://discuss.elastic.co/u/moderable)\ **Replies:** 5\ **Last updated:** [February 8, 2020, 4:06pm UTC](https://discuss.elastic.co/t/pattern-works-on-the-grok-debugger-but-fails-in-logstash/218410 "2020-02-08T16:06:49Z")
Can someone please tell me how I can debug this pattern and corresponding log entry that works on the grok debugger but fails in logstash Here is the code that works on the grok debugger: (.\*)\\\]%{SPACE}\[-\]%{SPACE}(.\*)\[… --- ## [Multiple pipeline not processing with no errors](https://discuss.elastic.co/t/multiple-pipeline-not-processing-with-no-errors/218439)
**Author:** [@vijay\_kaali](https://discuss.elastic.co/u/vijay_kaali)\ **Replies:** 0\ **Last updated:** [February 8, 2020, 12:38pm UTC](https://discuss.elastic.co/t/multiple-pipeline-not-processing-with-no-errors/218439 "2020-02-08T12:38:01Z")
I have two input plug in kafka and csv . pipeline.id: main path.config: "/etc/logstash/conf.d/\*.conf" pipeline.id: hpux path.config: "/etc/logstash/conf.d/hpux\_conf/hpux.conf" but only main is working hpux it not pr… --- ## [Remove nested documents from index using logstash](https://discuss.elastic.co/t/remove-nested-documents-from-index-using-logstash/217606)
**Author:** [@Mohan\_vel](https://discuss.elastic.co/u/Mohan_vel)\ **Replies:** 4\ **Last updated:** [February 8, 2020, 11:14am UTC](https://discuss.elastic.co/t/remove-nested-documents-from-index-using-logstash/217606 "2020-02-08T11:14:31Z")
Hi All i am trying to index the data from MS SQL server to elastic search as nested documents it's working fine but the challenge is i have to remove the nested documents and main documnet if the is\_active column retriev… --- ## [Logstash breaks my log line into multiple documets](https://discuss.elastic.co/t/logstash-breaks-my-log-line-into-multiple-documets/215156)
**Author:** [@kasra](https://discuss.elastic.co/u/kasra)\ **Replies:** 14\ **Last updated:** [February 8, 2020, 7:23am UTC](https://discuss.elastic.co/t/logstash-breaks-my-log-line-into-multiple-documets/215156 "2020-02-08T07:23:15Z")
I have nested JSON log files which I'm trying to parse and ship to my ES v.5 My logstash version is also 7.5. The problem is among for instance 3000 logs I get 250 \_jsonparsefailure! My log format is nested JSON. her… --- ## [Multiline is not displayed in the "Visualize" or "Dashboard" screen](https://discuss.elastic.co/t/multiline-is-not-displayed-in-the-visualize-or-dashboard-screen/216427)
**Author:** [@totopo\_loco](https://discuss.elastic.co/u/totopo_loco)\ **Replies:** 6\ **Last updated:** [February 7, 2020, 11:51pm UTC](https://discuss.elastic.co/t/multiline-is-not-displayed-in-the-visualize-or-dashboard-screen/216427 "2020-02-07T23:51:34Z")
Hello Dear Community, I am having problems to display multi line messages after they were parsed in Kibana 7.5.2. In the visualization and dashboard in Kibana the "message.keyword" is not shown when multi line messag… --- ## [Indexing question related to pipeline outputs](https://discuss.elastic.co/t/indexing-question-related-to-pipeline-outputs/218417)
**Author:** [@\_finack](https://discuss.elastic.co/u/_finack)\ **Replies:** 0\ **Last updated:** [February 7, 2020, 9:35pm UTC](https://discuss.elastic.co/t/indexing-question-related-to-pipeline-outputs/218417 "2020-02-07T21:35:21Z")
Issue: I have a Logstash pipeline output testing if \[agent\]\[type\] == "metricbeat". I am expecting such documents to end up in indices matching the pattern metricbeat-\*; however, some are ending up in indices matching the… --- ## [Logs parsed from Ubuntu but not from server with same configurations](https://discuss.elastic.co/t/logs-parsed-from-ubuntu-but-not-from-server-with-same-configurations/218412)
**Author:** [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)\ **Replies:** 0\ **Last updated:** [February 7, 2020, 8:55pm UTC](https://discuss.elastic.co/t/logs-parsed-from-ubuntu-but-not-from-server-with-same-configurations/218412 "2020-02-07T20:55:47Z")
Filebeat is sending and logstahs is applying grok filter perfectly from Ubuntu, but from windows server it only passed logs but logstash isnt applying filters. I have the same config file for filebeat and logstash on ser… --- ## [Elasticsearch / Logstash isnt indexing all lines of json file](https://discuss.elastic.co/t/elasticsearch-logstash-isnt-indexing-all-lines-of-json-file/218411)
**Author:** [@richk516](https://discuss.elastic.co/u/richk516)\ **Replies:** 0\ **Last updated:** [February 7, 2020, 8:55pm UTC](https://discuss.elastic.co/t/elasticsearch-logstash-isnt-indexing-all-lines-of-json-file/218411 "2020-02-07T20:55:11Z")
I am running a very simple config, which should index a json file. It works fine when the json is small (4 or 5 lines) but then when it gets any longer, I get an error and it only indexes the last few lines. To test. To… --- ## [Logstash thread model - workaround for multiple destinations](https://discuss.elastic.co/t/logstash-thread-model-workaround-for-multiple-destinations/218401)
**Author:** [@g.le](https://discuss.elastic.co/u/g.le)\ **Replies:** 0\ **Last updated:** [February 7, 2020, 8:23pm UTC](https://discuss.elastic.co/t/logstash-thread-model-workaround-for-multiple-destinations/218401 "2020-02-07T20:23:24Z")
A while ago, I had created a thread regarding cloned events and delivery to two distinct Elasticsearch clusters. As it turned out, the thread model of Logstash does not guarantee delivery of an event to its proper outpu… --- ## [Keyname in json is causing an error](https://discuss.elastic.co/t/keyname-in-json-is-causing-an-error/218372)
**Author:** [@rich516](https://discuss.elastic.co/u/rich516)\ **Replies:** 0\ **Last updated:** [February 7, 2020, 3:27pm UTC](https://discuss.elastic.co/t/keyname-in-json-is-causing-an-error/218372 "2020-02-07T15:27:55Z")
I have a strange error that I cant figure out. I am using logstash 7.3.2 and have a simple config. Input is a json log file I am creating manually (just testing right now) Output is to elsasticsearch. When my test fi… --- ## [Rename nested field name](https://discuss.elastic.co/t/rename-nested-field-name/218398)
**Author:** [@sai\_kiran1](https://discuss.elastic.co/u/sai_kiran1)\ **Replies:** 2\ **Last updated:** [February 7, 2020, 8:11pm UTC](https://discuss.elastic.co/t/rename-nested-field-name/218398 "2020-02-07T20:11:46Z")
Hello, Trying to rename the fields \[field\]\[subfield\] to \[subfield\] for all the dynamic subfields I receive under my field. "field" : { "subfield\_1" : "value1", "subfield\_2" : "value2", . . "subfield\_n" : "value3" … --- ## [Logstash not finding files](https://discuss.elastic.co/t/logstash-not-finding-files/218393)
**Author:** [@kberry](https://discuss.elastic.co/u/kberry)\ **Replies:** 0\ **Last updated:** [February 7, 2020, 6:59pm UTC](https://discuss.elastic.co/t/logstash-not-finding-files/218393 "2020-02-07T18:59:37Z")
I am running a logstash instance looking at 10000 gz files between kbs to mb in size. The files are stored in a folder structure of /var/log/prd/\/\/\/\/GZFILES/the\_files.gz. my file opti… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=406) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=408)