# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=424 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 425 --- ## [Date parse failure message strange behavior](https://discuss.elastic.co/t/date-parse-failure-message-strange-behavior/213862)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 2\ **Last updated:** [January 6, 2020, 6:14pm UTC](https://discuss.elastic.co/t/date-parse-failure-message-strange-behavior/213862 "2020-01-06T18:14:48Z")
Hello, Using the following filter : #log timestamp filter { date { match =\> \[ "timestamp","UNIX" \] target =\> "log\_timestamp" timezone =\> "Europe/Paris" } } I have a \[0\] "\_dateparsefailure", but target fiel… --- ## [Logstash is getting terminated after succesfully running 3-4 hours but not completing its job](https://discuss.elastic.co/t/logstash-is-getting-terminated-after-succesfully-running-3-4-hours-but-not-completing-its-job/213735)
**Author:** [@Avinaba\_Ghosh\_Hajra](https://discuss.elastic.co/u/Avinaba_Ghosh_Hajra)\ **Replies:** 4\ **Last updated:** [January 6, 2020, 5:54pm UTC](https://discuss.elastic.co/t/logstash-is-getting-terminated-after-succesfully-running-3-4-hours-but-not-completing-its-job/213735 "2020-01-06T17:54:08Z")
We are running a pipeline that will fetch the data from oracle db table and send it to elasticsearch. We are using jdbc input plugin for that. As it is a one time activity we are running it in the following way nohup lo… --- ## [Unusually rare error with regard to logstash](https://discuss.elastic.co/t/unusually-rare-error-with-regard-to-logstash/213913)
**Author:** [@PhobosDthorga](https://discuss.elastic.co/u/PhobosDthorga)\ **Replies:** 1\ **Last updated:** [January 6, 2020, 5:53pm UTC](https://discuss.elastic.co/t/unusually-rare-error-with-regard-to-logstash/213913 "2020-01-06T17:53:36Z")
Hello everyone! We at GekkoFyre Networks are encountering an issue with logstash where we keep experiencing the same error over and over \[ 1 \], causing the service to restart repeatedly until eventually, systemd termina… --- ## [DLQ rotate or retention](https://discuss.elastic.co/t/dlq-rotate-or-retention/213860)
**Author:** [@Aviad\_Bitton](https://discuss.elastic.co/u/Aviad_Bitton)\ **Replies:** 1\ **Last updated:** [January 6, 2020, 5:40pm UTC](https://discuss.elastic.co/t/dlq-rotate-or-retention/213860 "2020-01-06T17:40:46Z")
Hi, I am using Logstash 6.8 and i intend to use dlq for this purpose: If an event is not entered the Elastic cluster because of mapping conflict or something else, i want to show that log in a specific index that tell… --- ## [Combining two documents by aggregating common field](https://discuss.elastic.co/t/combining-two-documents-by-aggregating-common-field/213622)
**Author:** [@JeremyP](https://discuss.elastic.co/u/JeremyP)\ **Replies:** 9\ **Last updated:** [January 6, 2020, 3:40pm UTC](https://discuss.elastic.co/t/combining-two-documents-by-aggregating-common-field/213622 "2020-01-06T15:40:39Z")
Hello, I'm using the JDBC input plugin to ingest vulnerability data from a Rapid7 postgres database. Everything is fine, but one of the tables I'm referencing has many rows which is causing significant duplication of my… --- ## [Parse a gmail file](https://discuss.elastic.co/t/parse-a-gmail-file/213889)
**Author:** [@Yasso](https://discuss.elastic.co/u/Yasso)\ **Replies:** 0\ **Last updated:** [January 6, 2020, 12:41pm UTC](https://discuss.elastic.co/t/parse-a-gmail-file/213889 "2020-01-06T12:41:15Z")
hello guys .. can i parse a gmail file (i have download my gmail count(mygmailfile.mbox) ) with logstrash?.. what i need exactly is: what is the structure of the gmail .. how my cofig file should look like ..what patter… --- ## [\[Use Case\] Last Login Time for User](https://discuss.elastic.co/t/use-case-last-login-time-for-user/213700)
**Author:** [@rohus24](https://discuss.elastic.co/u/rohus24)\ **Replies:** 2\ **Last updated:** [January 6, 2020, 12:35pm UTC](https://discuss.elastic.co/t/use-case-last-login-time-for-user/213700 "2020-01-06T12:35:00Z")
Hi, I have created an index which is populated with event logs from my application. I would like to attempt to do the following using logstash: Determine all user login events (using pre-defined tags in logstash confi… --- ## [USE the data provide JDBC INPUT to send output email and SNS](https://discuss.elastic.co/t/use-the-data-provide-jdbc-input-to-send-output-email-and-sns/213880)
**Author:** [@reneroy](https://discuss.elastic.co/u/reneroy)\ **Replies:** 0\ **Last updated:** [January 6, 2020, 11:29am UTC](https://discuss.elastic.co/t/use-the-data-provide-jdbc-input-to-send-output-email-and-sns/213880 "2020-01-06T11:29:50Z")
Hello i want to use event provide to my statement in jdbc input oracle database ; extract the data and send alert case the value of data ( Email and SMS ) to a user ; please if you have a solution help me ! --- ## [Logstash RabbitMQ input with SAC](https://discuss.elastic.co/t/logstash-rabbitmq-input-with-sac/213866)
**Author:** [@cenemoi](https://discuss.elastic.co/u/cenemoi)\ **Replies:** 0\ **Last updated:** [January 6, 2020, 9:49am UTC](https://discuss.elastic.co/t/logstash-rabbitmq-input-with-sac/213866 "2020-01-06T09:49:25Z")
Hello, I need to add SAC on a queue. My conf : input { rabbitmq { ... arguments =\> {"x-single-active-consumer" =\> true} #passive =\> true (SAC working by another way) } } The warn : \[WARN \] 2020-01-06 09:44:50.… --- ## [How to Edit logs with gork?](https://discuss.elastic.co/t/how-to-edit-logs-with-gork/213850)
**Author:** [@Coder\_HK](https://discuss.elastic.co/u/Coder_HK)\ **Replies:** 1\ **Last updated:** [January 6, 2020, 9:23am UTC](https://discuss.elastic.co/t/how-to-edit-logs-with-gork/213850 "2020-01-06T09:23:53Z")
I have a log winlog.evend\_data.RuleName : technique\_Id=T2343,techniqueName=asdfdasd I want to change it to winlog.evend\_data.RuleName : technique\_Id : T4343 , techniquw\_name : asdfdasd I just need the regular expressio… --- ## [Too many socket connection to coordinating nodes?](https://discuss.elastic.co/t/too-many-socket-connection-to-coordinating-nodes/211689)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 5\ **Last updated:** [January 6, 2020, 9:22am UTC](https://discuss.elastic.co/t/too-many-socket-connection-to-coordinating-nodes/211689 "2020-01-06T09:22:41Z")
Hello, While investigating on "\[main\] Marking url as dead." errors, I noticed that I have lots of socket connection established from my logstash to the two coordinating nodes : netstat -nat | grep 9200 | wc -l 93 Lo… --- ## [Logstash keystore Permisson denied](https://discuss.elastic.co/t/logstash-keystore-permisson-denied/213590)
**Author:** [@zeleznypa](https://discuss.elastic.co/u/zeleznypa)\ **Replies:** 5\ **Last updated:** [January 6, 2020, 7:40am UTC](https://discuss.elastic.co/t/logstash-keystore-permisson-denied/213590 "2020-01-06T07:40:59Z")
We have configured smb shared folder between elastic stack servers, that contain the logstash.keystore file. This storage is mounted in for example /mnt/elastic folder the file /mnt/elastic/logstash.keystore is symlink… --- ## [Duplicate logs In Kibana](https://discuss.elastic.co/t/duplicate-logs-in-kibana/213562)
**Author:** [@Shlok\_Srivastava1](https://discuss.elastic.co/u/Shlok_Srivastava1)\ **Replies:** 2\ **Last updated:** [January 6, 2020, 6:51am UTC](https://discuss.elastic.co/t/duplicate-logs-in-kibana/213562 "2020-01-06T06:51:33Z")
I am getting Double logs for every api hits in Kibana. Following is my logstash configuration assuming the issue is related to logstash input { beats { port =\> 5044 ssl =\> false } } output { elastic… --- ## [Aws S3 subfolder structure data pull](https://discuss.elastic.co/t/aws-s3-subfolder-structure-data-pull/213661)
**Author:** [@suraj\_kumar3](https://discuss.elastic.co/u/suraj_kumar3)\ **Replies:** 1\ **Last updated:** [January 6, 2020, 5:52am UTC](https://discuss.elastic.co/t/aws-s3-subfolder-structure-data-pull/213661 "2020-01-06T05:52:32Z")
Hi there, I want to pull data from aws bucket having multiple subfolders but wildcard entry is not working. please help me with this one. Regards, Suraj --- ## [How to configure Logstash and Syslog ng to send and recieve logs?](https://discuss.elastic.co/t/how-to-configure-logstash-and-syslog-ng-to-send-and-recieve-logs/213773)
**Author:** [@Coder\_HK](https://discuss.elastic.co/u/Coder_HK)\ **Replies:** 5\ **Last updated:** [January 5, 2020, 7:20pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-and-syslog-ng-to-send-and-recieve-logs/213773 "2020-01-05T19:20:43Z")
I want to send syslog ng logs to logstash, I have installed security onion as a VM and want to send those logs to logstash which is on Ubuntu (another VM) kindly tell me how to configure both syslog-ng.log and logstash.c… --- ## [Beats not parsed](https://discuss.elastic.co/t/beats-not-parsed/213815)
**Author:** [@omrip](https://discuss.elastic.co/u/omrip)\ **Replies:** 0\ **Last updated:** [January 5, 2020, 5:30pm UTC](https://discuss.elastic.co/t/beats-not-parsed/213815 "2020-01-05T17:30:37Z")
when using beats as input the data show constantly to the console and seems all scrambled, what can be the reason? input { beats { port =\> 5044 } } filter { if "beats\_input\_codec\_plain\_applied" in \[tags\] { mutate… --- ## [Parsing Pulling data from EventHub](https://discuss.elastic.co/t/parsing-pulling-data-from-eventhub/213814)
**Author:** [@omrip](https://discuss.elastic.co/u/omrip)\ **Replies:** 0\ **Last updated:** [January 5, 2020, 5:15pm UTC](https://discuss.elastic.co/t/parsing-pulling-data-from-eventhub/213814 "2020-01-05T17:15:09Z")
when pulling data from event hub the data is all mixed up is there a built parser for the events? can i have some assistance with parsing the events? 0981Z", "userDisplayName": "bb king", "userPrincipa… --- ## [Logstash pull data from Event Hub And output to several LogAnalytics](https://discuss.elastic.co/t/logstash-pull-data-from-event-hub-and-output-to-several-loganalytics/213813)
**Author:** [@omrip](https://discuss.elastic.co/u/omrip)\ **Replies:** 0\ **Last updated:** [January 5, 2020, 5:10pm UTC](https://discuss.elastic.co/t/logstash-pull-data-from-event-hub-and-output-to-several-loganalytics/213813 "2020-01-05T17:10:13Z")
Hi, i am trying to send data from 2 pipelines inputs of Eventhub and forward the data to 2Log Analytics instances. there are 2 configuration file to read data from each event hub. the output of the data is send to 2 L… --- ## [Logstash Monitoring API -](https://discuss.elastic.co/t/logstash-monitoring-api/213286)
**Author:** [@Gidi\_Kalef](https://discuss.elastic.co/u/Gidi_Kalef)\ **Replies:** 13\ **Last updated:** [January 5, 2020, 8:46am UTC](https://discuss.elastic.co/t/logstash-monitoring-api/213286 "2020-01-05T08:46:59Z")
Hi Guys! i have 3 logstash servers installed for my data pipeline. i have enabled the xpack.monitoring for each one of the servers, when i use Kibana and go to "Monitoring" tab, i see only one instance although i have … --- ## [Delete documents loaded in the index in elastic via logstash](https://discuss.elastic.co/t/delete-documents-loaded-in-the-index-in-elastic-via-logstash/213624)
**Author:** [@Elvis\_Goncalves](https://discuss.elastic.co/u/Elvis_Goncalves)\ **Replies:** 1\ **Last updated:** [January 5, 2020, 4:39am UTC](https://discuss.elastic.co/t/delete-documents-loaded-in-the-index-in-elastic-via-logstash/213624 "2020-01-05T04:39:27Z")
I am using the following configuration file, but it does not delete the CUITS numbers that I get from my SQL database in the elasticsearch index. This is the code in the file input { jdbc { jdbc\_connection\_string =\> … --- ## [How to view the XML in Kibana](https://discuss.elastic.co/t/how-to-view-the-xml-in-kibana/213680)
**Author:** [@Chandan1989](https://discuss.elastic.co/u/Chandan1989)\ **Replies:** 1\ **Last updated:** [January 5, 2020, 4:30am UTC](https://discuss.elastic.co/t/how-to-view-the-xml-in-kibana/213680 "2020-01-05T04:30:05Z")
Hi, I want to see the the whole XML which I am trying to send from Cloud Foundry as input and the output is also converted into XML. Any suggestions to view the both XML. Regards, Chandan --- ## [Provided Grok patterns do not match data in the input](https://discuss.elastic.co/t/provided-grok-patterns-do-not-match-data-in-the-input/213601)
**Author:** [@Yasso](https://discuss.elastic.co/u/Yasso)\ **Replies:** 4\ **Last updated:** [January 5, 2020, 4:27am UTC](https://discuss.elastic.co/t/provided-grok-patterns-do-not-match-data-in-the-input/213601 "2020-01-05T04:27:38Z")
hello guys i want to parse that log : 55.3.244.1 GET /index.html 10024 0.043 \[2012-01-31 07:19:54 \] when i'm using this patters : %{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration} … --- ## [Logstash HTTP POST : Filter or Output (which would be better) also not wait for response?](https://discuss.elastic.co/t/logstash-http-post-filter-or-output-which-would-be-better-also-not-wait-for-response/213776)
**Author:** [@stcdarrell](https://discuss.elastic.co/u/stcdarrell)\ **Replies:** 0\ **Last updated:** [January 4, 2020, 3:48pm UTC](https://discuss.elastic.co/t/logstash-http-post-filter-or-output-which-would-be-better-also-not-wait-for-response/213776 "2020-01-04T15:48:35Z")
hi, i'm working on an interesting project. I have a honeypot (tpot19 https://github.com/dtag-dev-sec/tpotce) that gathers threat intelligence info. Its back end is ELK. i need to send this info to another platform that … --- ## [Failure mechanism in logstash](https://discuss.elastic.co/t/failure-mechanism-in-logstash/213733)
**Author:** [@Pramod\_Thakur](https://discuss.elastic.co/u/Pramod_Thakur)\ **Replies:** 0\ **Last updated:** [January 3, 2020, 5:35pm UTC](https://discuss.elastic.co/t/failure-mechanism-in-logstash/213733 "2020-01-03T17:35:55Z")
Hi Folks, I am trying to push data from oracle to ES or rabbitmq and I am able to do that as well. My problem when I am pushing those data and that time my ES or rabbitmq goes down then I am not able to figure out that … --- ## [How to ACTUALLY install plugins locally?](https://discuss.elastic.co/t/how-to-actually-install-plugins-locally/213759)
**Author:** [@Philip\_Brown](https://discuss.elastic.co/u/Philip_Brown)\ **Replies:** 0\ **Last updated:** [January 3, 2020, 11:59pm UTC](https://discuss.elastic.co/t/how-to-actually-install-plugins-locally/213759 "2020-01-03T23:59:25Z")
I've done a bunch of googling on this... spent many HOURS trying to get this to work.. with no luck. I need to install a plugin on a no-internet machine. specifically, logstash-output-zabbix. the docs claim I can just… --- ## [Grok sssd messages, like "\[sssd\[krb5\_child\[7502\]\]\]"](https://discuss.elastic.co/t/grok-sssd-messages-like-sssd-krb5-child-7502/213751)
**Author:** [@Willl](https://discuss.elastic.co/u/Willl)\ **Replies:** 0\ **Last updated:** [January 3, 2020, 10:11pm UTC](https://discuss.elastic.co/t/grok-sssd-messages-like-sssd-krb5-child-7502/213751 "2020-01-03T22:11:58Z")
I'm testing a Grok pattern for sssd messages like this one: 2020-01-03T16:28:24.968540-05:00 ny2dev10 \[sssd\[krb5\_child\[7312\]\]\]: Preauthentication failed I ran Grok Debugger with the following pattern and it looks fin… --- ## [Nested repeating group](https://discuss.elastic.co/t/nested-repeating-group/213638)
**Author:** [@nad](https://discuss.elastic.co/u/nad)\ **Replies:** 2\ **Last updated:** [January 3, 2020, 8:48pm UTC](https://discuss.elastic.co/t/nested-repeating-group/213638 "2020-01-03T20:48:12Z")
Hi, I'm starting using filebeat-\>logstash-\>ES to ship logs data. Here is bellow a sample of the application logs: 2020-01-01 01:00:01.590556482 : TOTO \[AAA\] GroupX1 RED : 2225714 BLUE : 9119409 GREEN : 231229… --- ## [Filebeat/Elastic/ECS source field conflict](https://discuss.elastic.co/t/filebeat-elastic-ecs-source-field-conflict/213634)
**Author:** [@mistrhanky](https://discuss.elastic.co/u/mistrhanky)\ **Replies:** 6\ **Last updated:** [January 3, 2020, 6:16pm UTC](https://discuss.elastic.co/t/filebeat-elastic-ecs-source-field-conflict/213634 "2020-01-03T18:16:19Z")
I am having an issue that seems it may be configurable, but I am not sure where to look next. I have filebeat setup up and most of my traffic is standard syslog stuff. I have one feed that is using a non-typical path tho… --- ## [Setting log4j settings; dynamic setting works, setting in log4j.properties causes malfunction](https://discuss.elastic.co/t/setting-log4j-settings-dynamic-setting-works-setting-in-log4j-properties-causes-malfunction/213594)
**Author:** [@tomj](https://discuss.elastic.co/u/tomj)\ **Replies:** 2\ **Last updated:** [January 3, 2020, 5:36pm UTC](https://discuss.elastic.co/t/setting-log4j-settings-dynamic-setting-works-setting-in-log4j-properties-causes-malfunction/213594 "2020-01-03T17:36:53Z")
Logstash 7.1.1 running on Amazon Linux 2 jdbc\_static filter plugin version 1.0.6 Elasticsearch 7.1.1 I have a test stream that produces about 125-150 messages per second, which I enrich using the jdbc\_static filter pl… --- ## [Grok debugger says good but logstash doesn't match](https://discuss.elastic.co/t/grok-debugger-says-good-but-logstash-doesnt-match/213620)
**Author:** [@Willl](https://discuss.elastic.co/u/Willl)\ **Replies:** 2\ **Last updated:** [January 3, 2020, 4:33pm UTC](https://discuss.elastic.co/t/grok-debugger-says-good-but-logstash-doesnt-match/213620 "2020-01-03T16:33:02Z")
Hi, Running filebeat with system module on /var/log/message. I tested the following in Kibana Grok Debugger, Log: Jan 2 14:42:52 dev10 sudo: pam\_unix(sudo:session): session opened for user root by root(uid=0) wit… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=423) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=425)