# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=429 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 430 --- ## [Logstash output to elastic-search fails](https://discuss.elastic.co/t/logstash-output-to-elastic-search-fails/212695)
**Author:** [@J\_Warner](https://discuss.elastic.co/u/J_Warner)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 5:52pm UTC](https://discuss.elastic.co/t/logstash-output-to-elastic-search-fails/212695 "2019-12-20T17:52:28Z")
Hey recently deployed Elasticsearch via ECK and Logstash with deployment the problem I'm getting is when the connection between them two when I check the logs I get the following error: \[2019-12-20T17:47:53,592\]\[WARN… --- ## [Winlogbeat flooding logstash with warnings. Handler in the pipeline did not handle the exception. Invalid version of beats protocol](https://discuss.elastic.co/t/winlogbeat-flooding-logstash-with-warnings-handler-in-the-pipeline-did-not-handle-the-exception-invalid-version-of-beats-protocol/212681)
**Author:** [@chunkychunkz](https://discuss.elastic.co/u/chunkychunkz)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 4:08pm UTC](https://discuss.elastic.co/t/winlogbeat-flooding-logstash-with-warnings-handler-in-the-pipeline-did-not-handle-the-exception-invalid-version-of-beats-protocol/212681 "2019-12-20T16:08:56Z")
Newbie to ELK stack here. However, I was able to get winlogbeat --\> Logstash --\> Elastic. I can see winlogbeat data from kibana, but my logs in logstash are flooded with warnings... should there be a concern? winlogbeat… --- ## [Queue on output](https://discuss.elastic.co/t/queue-on-output/212677)
**Author:** [@mkalinowski](https://discuss.elastic.co/u/mkalinowski)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 3:01pm UTC](https://discuss.elastic.co/t/queue-on-output/212677 "2019-12-20T15:01:08Z")
Hello, I read recently, there is something like "Persistent Queues". I also saw information about DLQ. And here my misunderstanding begins. I care about the problem when the cluster is down. It is important to me that… --- ## [Adding new field to existing document](https://discuss.elastic.co/t/adding-new-field-to-existing-document/212671)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 1:58pm UTC](https://discuss.elastic.co/t/adding-new-field-to-existing-document/212671 "2019-12-20T13:58:34Z")
Can I use logstash to read elasticsearch index create new field and write same document back to same index? --- ## [Getting error when run logstash 5.0 in ibm suse 12.4 os and ibm java 8](https://discuss.elastic.co/t/getting-error-when-run-logstash-5-0-in-ibm-suse-12-4-os-and-ibm-java-8/212620)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 4\ **Last updated:** [December 20, 2019, 1:31pm UTC](https://discuss.elastic.co/t/getting-error-when-run-logstash-5-0-in-ibm-suse-12-4-os-and-ibm-java-8/212620 "2019-12-20T13:31:47Z")
please let me know the solution why is it getting? logstash logs : NotImplementedError: stat.st\_dev unsupported or native support failed to load dev\_major at org/jruby/RubyFileStat.java:214 nix\_inode at /home/elastic… --- ## [GROOK FILTER HOW TO SPLIT THE MULTI VALUE COLUMN](https://discuss.elastic.co/t/grook-filter-how-to-split-the-multi-value-column/211462)
**Author:** [@saravana\_hariharan](https://discuss.elastic.co/u/saravana_hariharan)\ **Replies:** 0\ **Last updated:** [December 11, 2019, 12:14pm UTC](https://discuss.elastic.co/t/grook-filter-how-to-split-the-multi-value-column/211462 "2019-12-11T12:14:41Z")
GROOK FILTER %{NUMBER:TIME} \\t %{WORD:SESSIONID} \\t %{WORD:REQUESTID} \\t %{WORD:CRUD} \\t %{GREEDYDATA:FIELDVALUE} My question is how to split one column,which has multiple values of (CREATEDBY,CREATEDTIME,UPDATEDBY,… --- ## [Why do we need to Install libc6-compat on Alpine variant for logstash 5.0?](https://discuss.elastic.co/t/why-do-we-need-to-install-libc6-compat-on-alpine-variant-for-logstash-5-0/212657)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 11:54am UTC](https://discuss.elastic.co/t/why-do-we-need-to-install-libc6-compat-on-alpine-variant-for-logstash-5-0/212657 "2019-12-20T11:54:15Z")
please let me know the solution. --- ## [Logstash starting is pretty slow](https://discuss.elastic.co/t/logstash-starting-is-pretty-slow/212638)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 9:56am UTC](https://discuss.elastic.co/t/logstash-starting-is-pretty-slow/212638 "2019-12-20T09:56:13Z")
Hi elastic community! I really need some guidiance to speed up the starting of logstash service. Following is logstash restart history. \[2019-12-20T12:20:12,591\]\[INFO \]\[logstash.modules.scaffold\] Initializing module {… --- ## [Convert string field into Date field](https://discuss.elastic.co/t/convert-string-field-into-date-field/211266)
**Author:** [@Ahmed\_Rezgui](https://discuss.elastic.co/u/Ahmed_Rezgui)\ **Replies:** 8\ **Last updated:** [December 20, 2019, 9:37am UTC](https://discuss.elastic.co/t/convert-string-field-into-date-field/211266 "2019-12-20T09:37:41Z")
Hello Everyone I'm trying to index emails into elasticsearch with logstash, the problem is that two fields of the outputs are parsed as String fields but they are supposed to be "date" fields. The format of the fields … --- ## [Syslog\_severity field is always using the default notice rather than the actual severity](https://discuss.elastic.co/t/syslog-severity-field-is-always-using-the-default-notice-rather-than-the-actual-severity/212478)
**Author:** [@oldhamuk](https://discuss.elastic.co/u/oldhamuk)\ **Replies:** 2\ **Last updated:** [December 20, 2019, 8:36am UTC](https://discuss.elastic.co/t/syslog-severity-field-is-always-using-the-default-notice-rather-than-the-actual-severity/212478 "2019-12-20T08:36:52Z")
Good Morning, I'm fairly new to ELK but have managed to build my self solution I can use for a syslog server and I'm happy with it with the exceptions of one thing at the moment. The severity of the syslog events seems… --- ## [Logstash jdbc schedule missing values](https://discuss.elastic.co/t/logstash-jdbc-schedule-missing-values/211748)
**Author:** [@sadeshdrj](https://discuss.elastic.co/u/sadeshdrj)\ **Replies:** 2\ **Last updated:** [December 20, 2019, 7:24am UTC](https://discuss.elastic.co/t/logstash-jdbc-schedule-missing-values/211748 "2019-12-20T07:24:05Z")
Hi There, following i have running input jdbc method in logstash, i have noticed in my discover index the one table column "user name" data is missing or might appear but one few records or sometimes even the table colu… --- ## [SNMP Trap](https://discuss.elastic.co/t/snmp-trap/212610)
**Author:** [@inamatus](https://discuss.elastic.co/u/inamatus)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 6:45am UTC](https://discuss.elastic.co/t/snmp-trap/212610 "2019-12-20T06:45:51Z")
I have SNMP-Trap message. #\\>, #\ **Author:** [@rrue](https://discuss.elastic.co/u/rrue)\ **Replies:** 2\ **Last updated:** [December 19, 2019, 11:09pm UTC](https://discuss.elastic.co/t/date-stamp-in-index-name-is-being-ignored/212532 "2019-12-19T23:09:49Z") I'm upgrading to ELK 7.5 and finding a lot of things have changed since 5.5 (no kidding). In particular I can't seem to make logstash write to an ES index named for the current date. When testing with a very simple logs… --- ## [HTTP\_POLLER Certificate Question](https://discuss.elastic.co/t/http-poller-certificate-question/212557)
**Author:** [@RunningSmurf](https://discuss.elastic.co/u/RunningSmurf)\ **Replies:** 0\ **Last updated:** [December 19, 2019, 8:38pm UTC](https://discuss.elastic.co/t/http-poller-certificate-question/212557 "2019-12-19T20:38:53Z")
Hello, My HTTP\_POLLER makes a call to an API over https. There, I tried to use the following parameters in my .CONF file: truststore =\> "/path/to/downloaded\_truststore.jks" truststore\_password =\> "mypassword" Howeve… --- ## [Compare incoming string/data to keyword list, if in keyword list, flag?](https://discuss.elastic.co/t/compare-incoming-string-data-to-keyword-list-if-in-keyword-list-flag/212527)
**Author:** [@stcdarrell](https://discuss.elastic.co/u/stcdarrell)\ **Replies:** 3\ **Last updated:** [December 19, 2019, 6:46pm UTC](https://discuss.elastic.co/t/compare-incoming-string-data-to-keyword-list-if-in-keyword-list-flag/212527 "2019-12-19T18:46:59Z")
hi, i'm not really sure the best way to describe what i'm trying to do.. but i will have data bring streamed into logstash. i have a growing keyword list, that is a text file. is there a way i can compare the values/da… --- ## [Http filter for each array item](https://discuss.elastic.co/t/http-filter-for-each-array-item/212239)
**Author:** [@mdailey](https://discuss.elastic.co/u/mdailey)\ **Replies:** 1\ **Last updated:** [December 19, 2019, 6:38pm UTC](https://discuss.elastic.co/t/http-filter-for-each-array-item/212239 "2019-12-19T18:38:54Z")
I have a filter that I need to apply to each item of an array that I obtain from an earlier filter. It works fine for a single input like this: http { url =\> "http://my.api.com/api/v1/%{id}?verbose=false" header… --- ## [DLQ 7.1.0 not working](https://discuss.elastic.co/t/dlq-7-1-0-not-working/212543)
**Author:** [@John\_McAfee1](https://discuss.elastic.co/u/John_McAfee1)\ **Replies:** 0\ **Last updated:** [December 19, 2019, 6:24pm UTC](https://discuss.elastic.co/t/dlq-7-1-0-not-working/212543 "2019-12-19T18:24:52Z")
I am using version 7.1.0 Elastic stack to ingest logs into Elasticsearch. When procssing the data, I encounter this ERROR message several times: \[2019-12-19T12:30:10,332\]\[ERROR\]\[logstash.outputs.elasticsearch\] Encounte… --- ## [Doesn't work when trying to match on "source" and "message" in the same grok](https://discuss.elastic.co/t/doesnt-work-when-trying-to-match-on-source-and-message-in-the-same-grok/212353)
**Author:** [@Hamish1](https://discuss.elastic.co/u/Hamish1)\ **Replies:** 1\ **Last updated:** [December 19, 2019, 6:24pm UTC](https://discuss.elastic.co/t/doesnt-work-when-trying-to-match-on-source-and-message-in-the-same-grok/212353 "2019-12-19T18:24:11Z")
Good day everyone! I'm using filebeat and logstash. Here is the part of logstash filter which is not working as I'm expecting. if "app\_name" in \[tags\] { mutate { gsub =\> \["message", "\\x1B\\\[(\[0-9\]{1,2}(;\[0-9\]{1,2}… --- ## [Logstash, filebeat, syslog rfc5424 not working](https://discuss.elastic.co/t/logstash-filebeat-syslog-rfc5424-not-working/212516)
**Author:** [@Willl](https://discuss.elastic.co/u/Willl)\ **Replies:** 0\ **Last updated:** [December 19, 2019, 4:11pm UTC](https://discuss.elastic.co/t/logstash-filebeat-syslog-rfc5424-not-working/212516 "2019-12-19T16:11:00Z")
Hi, I have ELK 7.5.0 setup on one node with a few other systems running auditbeat and filebeat. I have SSL setup for communication between the beat and logstash. I have filebeat specified to look at /var/log/messages.… --- ## [Logstash sync mysql two tables to elasticsearch doesn't work](https://discuss.elastic.co/t/logstash-sync-mysql-two-tables-to-elasticsearch-doesnt-work/212500)
**Author:** [@hel](https://discuss.elastic.co/u/hel)\ **Replies:** 0\ **Last updated:** [December 19, 2019, 1:31pm UTC](https://discuss.elastic.co/t/logstash-sync-mysql-two-tables-to-elasticsearch-doesnt-work/212500 "2019-12-19T13:31:21Z")
I'm tring to use logstash to synchronize mysql tables to elasticsearch. At first there was only one input for synchronizing table 'article' and went well. Then I appended another input for synchronizing table 'user', but… --- ## [AWS ES logstash - Got response code '403' contacting Elasticsearch at URL 'https://search-aws-e\*](https://discuss.elastic.co/t/aws-es-logstash-got-response-code-403-contacting-elasticsearch-at-url-https-search-aws-e/212506)
**Author:** [@Kiran\_Kumar2](https://discuss.elastic.co/u/Kiran_Kumar2)\ **Replies:** 0\ **Last updated:** [December 19, 2019, 2:48pm UTC](https://discuss.elastic.co/t/aws-es-logstash-got-response-code-403-contacting-elasticsearch-at-url-https-search-aws-e/212506 "2019-12-19T14:48:09Z")
I have configured the AWS ES and logstash. and granted the default full AWS policy(AmazonESFullAccess) to the user still I am getting 403 error. my filebeats and logstash are running in same host here are the configura… --- ## [Two or more filters in Logstash](https://discuss.elastic.co/t/two-or-more-filters-in-logstash/212351)
**Author:** [@Paul\_Sternberg](https://discuss.elastic.co/u/Paul_Sternberg)\ **Replies:** 7\ **Last updated:** [December 19, 2019, 1:47pm UTC](https://discuss.elastic.co/t/two-or-more-filters-in-logstash/212351 "2019-12-19T13:47:01Z")
Hi folks! May be somebody know how to use 2 or more filters block or 2 or more grok blocks in logstash configuration? My current configuration looks like this: input { udp { port =\> 5514 type =\> syslog } } input … --- ## [Logstash-\>ES 7.5.0 import fails because of doc\_as\_upsert](https://discuss.elastic.co/t/logstash-es-7-5-0-import-fails-because-of-doc-as-upsert/212499)
**Author:** [@michalz](https://discuss.elastic.co/u/michalz)\ **Replies:** 0\ **Last updated:** [December 19, 2019, 1:28pm UTC](https://discuss.elastic.co/t/logstash-es-7-5-0-import-fails-because-of-doc-as-upsert/212499 "2019-12-19T13:28:17Z")
I try to import data from couchDb using Logstash (7.5.0) to Elasticsearch (7.5.0). When defining a Mapping in ES, I select "dynamic" = "strict" and in my elasticsearch output plugin I have: document\_id =\> "%{\[@metadat… --- ## [Jdbc input plugin cannot load the driver class](https://discuss.elastic.co/t/jdbc-input-plugin-cannot-load-the-driver-class/212482)
**Author:** [@dmytrok](https://discuss.elastic.co/u/dmytrok)\ **Replies:** 0\ **Last updated:** [December 19, 2019, 12:16pm UTC](https://discuss.elastic.co/t/jdbc-input-plugin-cannot-load-the-driver-class/212482 "2019-12-19T12:16:49Z")
I'm trying to fetch information from the internal metrics system of a data virtualization software Tibco TDV. The TDV admins published it as a JDBC-compliant database endpoint and provided me with the jar file and the cl… --- ## [Add request processing time](https://discuss.elastic.co/t/add-request-processing-time/212467)
**Author:** [@saliou](https://discuss.elastic.co/u/saliou)\ **Replies:** 0\ **Last updated:** [December 19, 2019, 10:35am UTC](https://discuss.elastic.co/t/add-request-processing-time/212467 "2019-12-19T10:35:37Z")
Hello everyone I am a beginner in logstash filter and to better understand the patterns I wanted to put a pattern to add the processing time of requests (in millisencodes) in my logs. The problem is that I can't find a… --- ## [Logstash Error: java.lang.NoClassDefFoundError: Could not initialize class org.logstash.config.ir.compiler.ComputeStepSyntaxElement](https://discuss.elastic.co/t/logstash-error-java-lang-noclassdeffounderror-could-not-initialize-class-org-logstash-config-ir-compiler-computestepsyntaxelement/212261)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 1\ **Last updated:** [December 19, 2019, 9:52am UTC](https://discuss.elastic.co/t/logstash-error-java-lang-noclassdeffounderror-could-not-initialize-class-org-logstash-config-ir-compiler-computestepsyntaxelement/212261 "2019-12-19T09:52:57Z")
Hi I just set up a filebeat to reach logstash and im getting the below error: \[2019-12-18T00:14:32,748\]\[INFO \]\[logstash.outputs.elasticsearch\] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :host… --- ## [Can Zeek logs listen on a syslog port](https://discuss.elastic.co/t/can-zeek-logs-listen-on-a-syslog-port/212458)
**Author:** [@tahseen\_fatima](https://discuss.elastic.co/u/tahseen_fatima)\ **Replies:** 0\ **Last updated:** [December 19, 2019, 9:29am UTC](https://discuss.elastic.co/t/can-zeek-logs-listen-on-a-syslog-port/212458 "2019-12-19T09:29:18Z")
Hello, I have a ELK setup of 7.4 in a cluster I have filebeat 7.4 version. I am receiving zeek logs via syslog port 514, and here is the default module configuration of zeek module. \> - module: zeek \> connection: \> … --- ## [Filter by event.code and divide message field](https://discuss.elastic.co/t/filter-by-event-code-and-divide-message-field/211473)
**Author:** [@KOTOXJle6](https://discuss.elastic.co/u/KOTOXJle6)\ **Replies:** 11\ **Last updated:** [December 19, 2019, 9:21am UTC](https://discuss.elastic.co/t/filter-by-event-code-and-divide-message-field/211473 "2019-12-19T09:21:36Z")
Hello! I have logstash 7.1 and winlogbeat 7.1. Im trying to get events 4624 from domain controllers, divide message filed into multiple field and then remove everything i don't need. I have 2 problems there: I've st… --- ## [Logstash could not be started because there is already another instance using the configured data directory](https://discuss.elastic.co/t/logstash-could-not-be-started-because-there-is-already-another-instance-using-the-configured-data-directory/209226)
**Author:** [@sasikumarudu](https://discuss.elastic.co/u/sasikumarudu)\ **Replies:** 1\ **Last updated:** [December 19, 2019, 8:18am UTC](https://discuss.elastic.co/t/logstash-could-not-be-started-because-there-is-already-another-instance-using-the-configured-data-directory/209226 "2019-12-19T08:18:35Z")
Hi Team, I need your help to fix the issue. I used to run Initiate a Job by providing command "/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf -l /var/log/logstash/logstash-plain.log &" in one of… --- ## [Unable to run Logstash : Logstash Error](https://discuss.elastic.co/t/unable-to-run-logstash-logstash-error/212346)
**Author:** [@irobot678](https://discuss.elastic.co/u/irobot678)\ **Replies:** 2\ **Last updated:** [December 19, 2019, 7:18am UTC](https://discuss.elastic.co/t/unable-to-run-logstash-logstash-error/212346 "2019-12-19T07:18:56Z")
HI, I'm using elk 7.3.2 version and im not able to run the logstash due the following error. This is the only error its showing Please help me solve it. \[ERROR\] 2019-12-18 19:46:29.143 \[main\] Logstash - org.jruby.exc… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=428) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=430)