# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=644 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 645 --- ## [Logstash.filters.json throw warn exception:java.lang.ClassCastException: org.jruby.RubyArray cannot be cast to org.jruby.RubyIO](https://discuss.elastic.co/t/logstash-filters-json-throw-warn-exception-java-lang-classcastexception-org-jruby-rubyarray-cannot-be-cast-to-org-jruby-rubyio/157957)
**Author:** [@ocean.pang](https://discuss.elastic.co/u/ocean.pang)\ **Replies:** 0\ **Last updated:** [November 23, 2018, 3:28am UTC](https://discuss.elastic.co/t/logstash-filters-json-throw-warn-exception-java-lang-classcastexception-org-jruby-rubyarray-cannot-be-cast-to-org-jruby-rubyio/157957 "2018-11-23T03:28:06Z")
elasticsearch version : 6.4.3 logstash version : 6.4.3 what i want to do? import relationship data from mysql to elasticsearch issue: the relationship data in mysql like this: rid resumeid eid orid schoolna… --- ## [Multiple GROK filters not applying to log message](https://discuss.elastic.co/t/multiple-grok-filters-not-applying-to-log-message/157020)
**Author:** [@ashok9177](https://discuss.elastic.co/u/ashok9177)\ **Replies:** 13\ **Last updated:** [November 23, 2018, 4:06am UTC](https://discuss.elastic.co/t/multiple-grok-filters-not-applying-to-log-message/157020 "2018-11-23T04:06:01Z")
I have created 2 grok filters to match log messages, but the first filter is only applying the second filter is not applying to all log messages but the 2 filters parsing logs, I have created the first filter for log… --- ## [Output to any output is not working](https://discuss.elastic.co/t/output-to-any-output-is-not-working/157439)
**Author:** [@zozo6015](https://discuss.elastic.co/u/zozo6015)\ **Replies:** 7\ **Last updated:** [November 23, 2018, 1:40am UTC](https://discuss.elastic.co/t/output-to-any-output-is-not-working/157439 "2018-11-23T01:40:32Z")
Hello, I am importing logs via filebeat 6.5.0 into logstash. If in debug mode I can see the data being taken by logstash but it is not sent to any output I have tried stdout output and elasticsearch output but did not g… --- ## [Check ruby code](https://discuss.elastic.co/t/check-ruby-code/157815)
**Author:** [@jmkim](https://discuss.elastic.co/u/jmkim)\ **Replies:** 2\ **Last updated:** [November 23, 2018, 12:41am UTC](https://discuss.elastic.co/t/check-ruby-code/157815 "2018-11-23T00:41:40Z")
Hi, everyone! I'm trying to check an array in Ruby code, but it's not working. Could you check it together? You want to save only if the keys you define are the same. However, all parameters are currently stored. kv… --- ## [Not working exclude in file plugin on logstash-6.5.0](https://discuss.elastic.co/t/not-working-exclude-in-file-plugin-on-logstash-6-5-0/157942)
**Author:** [@BrianVitaliy](https://discuss.elastic.co/u/BrianVitaliy)\ **Replies:** 0\ **Last updated:** [November 22, 2018, 9:07pm UTC](https://discuss.elastic.co/t/not-working-exclude-in-file-plugin-on-logstash-6-5-0/157942 "2018-11-22T21:07:55Z")
Hi. For test I have easy configuration input { file { path =\> "e:/work/\_\_testing/testELK/logs/\*.log" exclude =\> "test2.log" codec =\> "json" start\_position =\> "beginning" } } output { file { path =\> "E:/wor… --- ## [JDBC SQL and Inner and aggregate plug in](https://discuss.elastic.co/t/jdbc-sql-and-inner-and-aggregate-plug-in/157930)
**Author:** [@Dave2](https://discuss.elastic.co/u/Dave2)\ **Replies:** 0\ **Last updated:** [November 22, 2018, 7:26pm UTC](https://discuss.elastic.co/t/jdbc-sql-and-inner-and-aggregate-plug-in/157930 "2018-11-22T19:26:13Z")
Hi, first of all congratulations on your site and documentation I have a problem with my configuration, everything works fine, but there is something I can't accomplish I have this SQL results |clipid|ClientCode|Titl… --- ## [Logstash output plugin index time zone problems](https://discuss.elastic.co/t/logstash-output-plugin-index-time-zone-problems/157917)
**Author:** [@maghibus](https://discuss.elastic.co/u/maghibus)\ **Replies:** 0\ **Last updated:** [November 22, 2018, 4:37pm UTC](https://discuss.elastic.co/t/logstash-output-plugin-index-time-zone-problems/157917 "2018-11-22T16:37:19Z")
Hi, Take for example a log file with the following content: 2018-11-15 00:58:15,417 INFO \[org.alfresco.solr.component.AsyncBuildSuggestComponent\] Building suggester index for: shingleBasedSuggestions 2018-11-15 01:58:… --- ## [No implicit conversion from nil to integer](https://discuss.elastic.co/t/no-implicit-conversion-from-nil-to-integer/157905)
**Author:** [@suishiyou](https://discuss.elastic.co/u/suishiyou)\ **Replies:** 0\ **Last updated:** [November 22, 2018, 3:34pm UTC](https://discuss.elastic.co/t/no-implicit-conversion-from-nil-to-integer/157905 "2018-11-22T15:34:18Z")
My logstash is 6.5.1,and when i running the logstash conf ,i am getting this exception,someone konw why ? \ \[2018-11-22T23:12:39,372\]\[ERROR\]\[logstash.pipeline \] Pipeline aborted due to error {:pipeline\_id=\>"ma… --- ## [Logstash 6.4.1 & 6.5.0 High CPU usage](https://discuss.elastic.co/t/logstash-6-4-1-6-5-0-high-cpu-usage/157744)
**Author:** [@YanekR](https://discuss.elastic.co/u/YanekR)\ **Replies:** 1\ **Last updated:** [November 22, 2018, 11:40am UTC](https://discuss.elastic.co/t/logstash-6-4-1-6-5-0-high-cpu-usage/157744 "2018-11-22T11:40:44Z")
Hi All, I upgraded Elastic Stack from: At first 6.3.0 --\> 6.4.1 (whole stack) Then from 6.4.1 --\> 6.4.2 (only Logstash) then from 6.4.2 --\> 6.5.0 (only Logstash) After first upgrade I have notice high CPU usage chan… --- ## [Regroupe LDAP log lines with same id on the same DataTable](https://discuss.elastic.co/t/regroupe-ldap-log-lines-with-same-id-on-the-same-datatable/157871)
**Author:** [@Noureddine\_Brahmi](https://discuss.elastic.co/u/Noureddine_Brahmi)\ **Replies:** 5\ **Last updated:** [November 22, 2018, 1:09pm UTC](https://discuss.elastic.co/t/regroupe-ldap-log-lines-with-same-id-on-the-same-datatable/157871 "2018-11-22T13:09:28Z")
Hello, I'm currently working on an ElasticSearch project that monitors LDAP logs. In the LDAP logs, related events shares the same id (connection\_number). I was wondering is there is a solution to regroupe the various … --- ## [Trouble parsing xml file in logstash](https://discuss.elastic.co/t/trouble-parsing-xml-file-in-logstash/157874)
**Author:** [@shubh235](https://discuss.elastic.co/u/shubh235)\ **Replies:** 2\ **Last updated:** [November 22, 2018, 1:07pm UTC](https://discuss.elastic.co/t/trouble-parsing-xml-file-in-logstash/157874 "2018-11-22T13:07:03Z")
I am trying to load a basic xml file into logstash but somehow, the file is not loaded. No exceptions are thrown and I am running Logstash through command prompt and not as a service. Could you please check and help me f… --- ## [Index json file into Elasticsearch](https://discuss.elastic.co/t/index-json-file-into-elasticsearch/157715)
**Author:** [@imaad](https://discuss.elastic.co/u/imaad)\ **Replies:** 18\ **Last updated:** [November 22, 2018, 12:37pm UTC](https://discuss.elastic.co/t/index-json-file-into-elasticsearch/157715 "2018-11-22T12:37:30Z")
Hello, I'm using a basic logstash conf to index a json file to Elasticsearch : input { file { path =\> "C:\\Users\\imadd\\OneDrive\\Bureau\\ebusiness.json" sincedb\_path =\> "null" type =\> "json" codec =\> "json" … --- ## [Logstash not able to output to ES error occurred sending bulk request](https://discuss.elastic.co/t/logstash-not-able-to-output-to-es-error-occurred-sending-bulk-request/157000)
**Author:** [@Divit\_Sharma](https://discuss.elastic.co/u/Divit_Sharma)\ **Replies:** 5\ **Last updated:** [November 22, 2018, 12:07pm UTC](https://discuss.elastic.co/t/logstash-not-able-to-output-to-es-error-occurred-sending-bulk-request/157000 "2018-11-22T12:07:30Z")
I am able to see the output in stout but when I am indexing to ES it gives this error The thing is if I am connecting to Hive from jdbc input this error comes. I am able to index data from oracle. An error occurred sen… --- ## [Logstash Exiting: Template loading requested but the Elasticsearch output is not configured/enabled](https://discuss.elastic.co/t/logstash-exiting-template-loading-requested-but-the-elasticsearch-output-is-not-configured-enabled/157866)
**Author:** [@firewall](https://discuss.elastic.co/u/firewall)\ **Replies:** 0\ **Last updated:** [November 22, 2018, 11:27am UTC](https://discuss.elastic.co/t/logstash-exiting-template-loading-requested-but-the-elasticsearch-output-is-not-configured-enabled/157866 "2018-11-22T11:27:26Z")
Hi! I'm new to ELK.... configured a Elasticsearch and Kibana, but my logstash doesn't work.... I'm get this message from client Exiting: Template loading requested but the Elasticsearch output is not configured/enabled … --- ## [Regarding Persistent Queues](https://discuss.elastic.co/t/regarding-persistent-queues/157637)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 1\ **Last updated:** [November 22, 2018, 10:44am UTC](https://discuss.elastic.co/t/regarding-persistent-queues/157637 "2018-11-22T10:44:11Z")
Hi, Can I ask, is there a recommendation regarding the disk size that should be allocated to persistent queues? Or is it simply, the more the better? Regards, D --- ## [How to remove \\ in logstash](https://discuss.elastic.co/t/how-to-remove-in-logstash/157504)
**Author:** [@arun5635](https://discuss.elastic.co/u/arun5635)\ **Replies:** 7\ **Last updated:** [November 22, 2018, 9:01am UTC](https://discuss.elastic.co/t/how-to-remove-in-logstash/157504 "2018-11-22T09:01:15Z")
Hi All I am new to the Logstash, I need to remove \\ from request and Http\_method request": ""GET https://www.docsapp.in:443/dashboardapp/forward/0/content?patientId=6009237 HTTP/2.0"" http\_method": ""GET" I want lik… --- ## [Grok with multiple matches - can i assign type based on match?](https://discuss.elastic.co/t/grok-with-multiple-matches-can-i-assign-type-based-on-match/60397)
**Author:** [@jamesmanning](https://discuss.elastic.co/u/jamesmanning)\ **Replies:** 6\ **Last updated:** [November 22, 2018, 7:53am UTC](https://discuss.elastic.co/t/grok-with-multiple-matches-can-i-assign-type-based-on-match/60397 "2018-11-22T07:53:05Z")
I'm parsing an IRC log and trying to assign different types based on which type of log entry a given line is. Currently I'm using N different matches in a single grok filter, since a given line should match at most one… --- ## [From mysql to aws elasticsearch using logstash](https://discuss.elastic.co/t/from-mysql-to-aws-elasticsearch-using-logstash/157627)
**Author:** [@tripstore](https://discuss.elastic.co/u/tripstore)\ **Replies:** 5\ **Last updated:** [November 22, 2018, 5:12am UTC](https://discuss.elastic.co/t/from-mysql-to-aws-elasticsearch-using-logstash/157627 "2018-11-22T05:12:25Z")
Hi everyone, I'm new to ELK and I have never used AWS Elasticsearch before. I'm trying to migrate data from mysql server to aws elasticsearch using logstash. However, I got an error saying something like this: \[ERROR… --- ## [Send csv to elasticsearch cloud error](https://discuss.elastic.co/t/send-csv-to-elasticsearch-cloud-error/157789)
**Author:** [@mateo](https://discuss.elastic.co/u/mateo)\ **Replies:** 0\ **Last updated:** [November 21, 2018, 11:31pm UTC](https://discuss.elastic.co/t/send-csv-to-elasticsearch-cloud-error/157789 "2018-11-21T23:31:04Z")
Can somebody help me ? I want to send a csv to elasticsearch cloud but I get an error this is my logstash conf and later the error: input { file { path=\>"/home/mateo/Documentos/proyectomis/saldos… --- ## [Logstash-plugins: logstash-input-sqs](https://discuss.elastic.co/t/logstash-plugins-logstash-input-sqs/157786)
**Author:** [@sknayar](https://discuss.elastic.co/u/sknayar)\ **Replies:** 0\ **Last updated:** [November 21, 2018, 10:56pm UTC](https://discuss.elastic.co/t/logstash-plugins-logstash-input-sqs/157786 "2018-11-21T22:56:15Z")
Hello, I'm trying to use sqs plugin to pull messages from AWS sqs queue. My config is as follows input { sqs { access\_key\_id =\> 'xxxxxxxxx' secret\_access\_key =\> 'xxxxxxx' queue =\> "queue-name" region =\> "us-west-… --- ## [\[FATAL\]\[logstash.runner\] Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting](https://discuss.elastic.co/t/fatal-logstash-runner-logstash-could-not-be-started-because-there-is-already-another-instance-using-the-configured-data-directory-if-you-wish-to-run-multiple-instances-you-must-change-the-path-data-setting/156694)
**Author:** [@vnallamothu](https://discuss.elastic.co/u/vnallamothu)\ **Replies:** 0\ **Last updated:** [November 14, 2018, 2:26pm UTC](https://discuss.elastic.co/t/fatal-logstash-runner-logstash-could-not-be-started-because-there-is-already-another-instance-using-the-configured-data-directory-if-you-wish-to-run-multiple-instances-you-must-change-the-path-data-setting/156694 "2018-11-14T14:26:19Z")
Hi, I've successfully configured logstash and was able to run the pipeline and abled to deliver the data from s3 to elasticsearch. But, Logstash is not continuously delivering the data from S3. Data is getting delivered… --- ## [How to debug Logstash filter with dissect?](https://discuss.elastic.co/t/how-to-debug-logstash-filter-with-dissect/157616)
**Author:** [@RussellBateman](https://discuss.elastic.co/u/RussellBateman)\ **Replies:** 6\ **Last updated:** [November 21, 2018, 3:43pm UTC](https://discuss.elastic.co/t/how-to-debug-logstash-filter-with-dissect/157616 "2018-11-21T15:43:20Z")
There's a grok debugger, but how best to debug my dissect filter when none of my log data appear to "get caught" in it? My filter: filter { if( \[ type \] == "doc" ) { dissect { mapping =\> { … --- ## [The logstash-keystore utility produces many, many errors](https://discuss.elastic.co/t/the-logstash-keystore-utility-produces-many-many-errors/157769)
**Author:** [@jbrowe](https://discuss.elastic.co/u/jbrowe)\ **Replies:** 0\ **Last updated:** [November 21, 2018, 7:06pm UTC](https://discuss.elastic.co/t/the-logstash-keystore-utility-produces-many-many-errors/157769 "2018-11-21T19:06:24Z")
I'm working to secure all my Elastic communications channels and config files. I use the elastic and kibana keystore utilities to store all the passwords that are scattered throughout the config files. This now works and… --- ## [Syslog output actually displaying "%{message}" instead of the message from metricbeat](https://discuss.elastic.co/t/syslog-output-actually-displaying-message-instead-of-the-message-from-metricbeat/157740)
**Author:** [@podexperfectussum](https://discuss.elastic.co/u/podexperfectussum)\ **Replies:** 0\ **Last updated:** [November 21, 2018, 4:03pm UTC](https://discuss.elastic.co/t/syslog-output-actually-displaying-message-instead-of-the-message-from-metricbeat/157740 "2018-11-21T16:03:19Z")
I have a need to grab metricbeat output from my logstash nodes and forward it directly to a syslog server (I'll spare you the details). The problem is, the data received looks like this: Nov 13 06:39:12 MONITOREDSERVER… --- ## [Enable to connect Logstash to Kafka](https://discuss.elastic.co/t/enable-to-connect-logstash-to-kafka/157563)
**Author:** [@Youssef\_El](https://discuss.elastic.co/u/Youssef_El)\ **Replies:** 2\ **Last updated:** [November 21, 2018, 3:34pm UTC](https://discuss.elastic.co/t/enable-to-connect-logstash-to-kafka/157563 "2018-11-21T15:34:21Z")
Hello, I'm trying to connect Kafka to logastash by making kafka input in logstash config file ,but i'm confused with kafka bootstrap\_servers who is hosted in hortonworks platform . In ambari UI is mentioned that kafka … --- ## [Add Double quotes for json data in logstash](https://discuss.elastic.co/t/add-double-quotes-for-json-data-in-logstash/157713)
**Author:** [@Sanj](https://discuss.elastic.co/u/Sanj)\ **Replies:** 8\ **Last updated:** [November 21, 2018, 2:48pm UTC](https://discuss.elastic.co/t/add-double-quotes-for-json-data-in-logstash/157713 "2018-11-21T14:48:18Z")
Hi Team, I'm trying to parse a JSON data through logstash. I'm using JSON and split plugins to parse the data. Below is my format of my JSON message. { agentId: "TMS", apiVersion: "v2", entities: \[ { agentId: "S… --- ## [Load balancing support for HTTP plugin](https://discuss.elastic.co/t/load-balancing-support-for-http-plugin/157726)
**Author:** [@bhushan2901](https://discuss.elastic.co/u/bhushan2901)\ **Replies:** 0\ **Last updated:** [November 21, 2018, 2:29pm UTC](https://discuss.elastic.co/t/load-balancing-support-for-http-plugin/157726 "2018-11-21T14:29:43Z")
Hi Team, Do we have a support for load balancing for HTTP plugin ? I can see for elasticsearch plugin the support is there, i.e. if we specify multiple hosts it will load balance , but similar support is there for HTTP … --- ## [Use field value to match custom Grok pattern](https://discuss.elastic.co/t/use-field-value-to-match-custom-grok-pattern/157681)
**Author:** [@apluchart](https://discuss.elastic.co/u/apluchart)\ **Replies:** 10\ **Last updated:** [November 21, 2018, 1:33pm UTC](https://discuss.elastic.co/t/use-field-value-to-match-custom-grok-pattern/157681 "2018-11-21T13:33:07Z")
Hi, I'm trying to parse some log (from cisco ironport) and I want to use a field value to access my custom patterns. To be clear there is my actual filter : dissect { mapping =\> {"message" =\> "\<%{syslog5424pri}\>%{moi… --- ## [Convert epoch milli to datetime](https://discuss.elastic.co/t/convert-epoch-milli-to-datetime/157656)
**Author:** [@sumitbiswas](https://discuss.elastic.co/u/sumitbiswas)\ **Replies:** 5\ **Last updated:** [November 21, 2018, 1:24pm UTC](https://discuss.elastic.co/t/convert-epoch-milli-to-datetime/157656 "2018-11-21T13:24:07Z")
Logstash version: 6.5.0 How do we convert epoch millisecond to datetime? Thanks in advance. Below parses data and in Kibana is seen as below which is wrong. (t) epoch\_time: 1542765548000000000 (t) myTime: 48890264-01… --- ## [Building offline plugin pack for logstash-codec-avro is not working?](https://discuss.elastic.co/t/building-offline-plugin-pack-for-logstash-codec-avro-is-not-working/157709)
**Author:** [@emailshak](https://discuss.elastic.co/u/emailshak)\ **Replies:** 1\ **Last updated:** [November 21, 2018, 1:19pm UTC](https://discuss.elastic.co/t/building-offline-plugin-pack-for-logstash-codec-avro-is-not-working/157709 "2018-11-21T13:19:07Z")
I am trying to build an offline plugin pack for logstash-codec-avro. I keep getting the following message. "ERROR: Cannot create the offline archive, message: cannot find plugin matching: 'logstash-codec-avro'" Not sure… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=643) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=645)