# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=226 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 227 --- ## [Logstash Java plugin development, faster testing w/o rebuilding ruby gem?](https://discuss.elastic.co/t/logstash-java-plugin-development-faster-testing-w-o-rebuilding-ruby-gem/273954)
**Author:** [@BradVido](https://discuss.elastic.co/u/BradVido)\ **Replies:** 0\ **Last updated:** [May 25, 2021, 2:12pm UTC](https://discuss.elastic.co/t/logstash-java-plugin-development-faster-testing-w-o-rebuilding-ruby-gem/273954 "2021-05-25T14:12:10Z")
I'm putting together a Logstash Java plugin, and it's working well, but it takes a bit of time to package to a gem and install in logstash every time we want to test it. Is it possible to use something like path.plugins … --- ## [Integer is not recognized](https://discuss.elastic.co/t/integer-is-not-recognized/273920)
**Author:** [@dinesh5](https://discuss.elastic.co/u/dinesh5)\ **Replies:** 0\ **Last updated:** [May 25, 2021, 10:57am UTC](https://discuss.elastic.co/t/integer-is-not-recognized/273920 "2021-05-25T10:57:12Z")
Hi Here is my sample data Madhya Pradesh;22.9734;78.6569;57766;764338;7558 Maharashtra;19.7515;75.7139;351005;5579897;88620 Manipur;24.6637;93.9063;6534;44089;688 Logstash considered the text and float but it doesn'… --- ## [How get data from snowflake to elastic search in elastic cloud](https://discuss.elastic.co/t/how-get-data-from-snowflake-to-elastic-search-in-elastic-cloud/273944)
**Author:** [@Ahmed\_Sadek](https://discuss.elastic.co/u/Ahmed_Sadek)\ **Replies:** 0\ **Last updated:** [May 25, 2021, 1:30pm UTC](https://discuss.elastic.co/t/how-get-data-from-snowflake-to-elastic-search-in-elastic-cloud/273944 "2021-05-25T13:30:57Z")
Hi, 1- I am using elastic cloud 2- In kibana, under stack management there is option to configure logstash pipeline 3- In documentation there is a plugin to get data from snowflake to elastic using jdbc 4- can someon… --- ## [How get data from google storage to elastic search in elastic cloud](https://discuss.elastic.co/t/how-get-data-from-google-storage-to-elastic-search-in-elastic-cloud/273942)
**Author:** [@Ahmed\_Sadek](https://discuss.elastic.co/u/Ahmed_Sadek)\ **Replies:** 0\ **Last updated:** [May 25, 2021, 1:28pm UTC](https://discuss.elastic.co/t/how-get-data-from-google-storage-to-elastic-search-in-elastic-cloud/273942 "2021-05-25T13:28:37Z")
Hi, 1- I am using elastic cloud 2- Under stack management there is option to configure logstash pipeline 3- in documentation there is a plugin to get data from google storage to elastic 4- can someone provide a usabl… --- ## [Use multiple Elasticsearch Ingest Pipelines with Logstash Elasticsearch Output](https://discuss.elastic.co/t/use-multiple-elasticsearch-ingest-pipelines-with-logstash-elasticsearch-output/273939)
**Author:** [@DougR](https://discuss.elastic.co/u/DougR)\ **Replies:** 0\ **Last updated:** [May 25, 2021, 1:23pm UTC](https://discuss.elastic.co/t/use-multiple-elasticsearch-ingest-pipelines-with-logstash-elasticsearch-output/273939 "2021-05-25T13:23:07Z")
When passing module logs through Logstash to Elasticsearch for processing by ES ingest pipelines, as described here, is it possible to specify multiple pipelines to apply in order? --- ## [Import JSON from SQL Server to ElasticSearch](https://discuss.elastic.co/t/import-json-from-sql-server-to-elasticsearch/273931)
**Author:** [@qttv](https://discuss.elastic.co/u/qttv)\ **Replies:** 0\ **Last updated:** [May 25, 2021, 12:28pm UTC](https://discuss.elastic.co/t/import-json-from-sql-server-to-elasticsearch/273931 "2021-05-25T12:28:49Z")
Hi, I'm importing some data from Microsoft SQL Server in ElasticSearch using Logstash. I'm dealing with a table in wich a column contains JSON logs, and i need to import these logs splitting the fields in order to create… --- ## [Invalid FieldReference: \`\[\]\`](https://discuss.elastic.co/t/invalid-fieldreference/273834)
**Author:** [@Roberto\_B](https://discuss.elastic.co/u/Roberto_B)\ **Replies:** 3\ **Last updated:** [May 25, 2021, 1:02pm UTC](https://discuss.elastic.co/t/invalid-fieldreference/273834 "2021-05-25T13:02:43Z")
Hi all, i receive from logstash error this kind of error: :exception=\>#\ **Author:** [@mirko.spezie](https://discuss.elastic.co/u/mirko.spezie)\ **Replies:** 2\ **Last updated:** [May 25, 2021, 9:53am UTC](https://discuss.elastic.co/t/parsing-json-array-inside/273870 "2021-05-25T09:53:34Z") Hi, I've got a json with an array inside. Based on another thread I've tried something with ruby to change the key names but it doesn't work. "messages": \[ { "message": "XSS Attack Detected via libinjection… --- ## [Parsing xml in logstash \[0\] "\_xmlparsefailure"](https://discuss.elastic.co/t/parsing-xml-in-logstash-0-xmlparsefailure/273857)
**Author:** [@Bugsbee](https://discuss.elastic.co/u/Bugsbee)\ **Replies:** 4\ **Last updated:** [May 25, 2021, 8:42am UTC](https://discuss.elastic.co/t/parsing-xml-in-logstash-0-xmlparsefailure/273857 "2021-05-25T08:42:18Z")
Issue: \[0\] "\_xmlparsefailure" ..Hi all need your help on below issue.. Pleaes refer to step by step explanation of the issue. Step 1 : I have sample input as per below. \ blue … --- ## [Preventing duplicates when reading the same data multiple times](https://discuss.elastic.co/t/preventing-duplicates-when-reading-the-same-data-multiple-times/273576)
**Author:** [@wlbsxnlp22](https://discuss.elastic.co/u/wlbsxnlp22)\ **Replies:** 2\ **Last updated:** [May 25, 2021, 8:34am UTC](https://discuss.elastic.co/t/preventing-duplicates-when-reading-the-same-data-multiple-times/273576 "2021-05-25T08:34:51Z")
Hi, I'm wondering if anyone has a good solution to the following. I have to read in .csv files that are generated every hour. These .csv files contain all the historic data for as far back as the program has run. Every… --- ## [Does anyone know the password for "file1" in zipWithEncryption?](https://discuss.elastic.co/t/does-anyone-know-the-password-for-file1-in-zipwithencryption/273817)
**Author:** [@fox1](https://discuss.elastic.co/u/fox1)\ **Replies:** 3\ **Last updated:** [May 25, 2021, 7:29am UTC](https://discuss.elastic.co/t/does-anyone-know-the-password-for-file1-in-zipwithencryption/273817 "2021-05-25T07:29:52Z")
HI all, I just hope I'm on the right place... Does anyone know by chance what is the password for that little text file called "file1" in the zip file called "zipWithEncryption" ? located somewhere here : ...\\layer\\u… --- ## [Avro to Json](https://discuss.elastic.co/t/avro-to-json/273820)
**Author:** [@thirty2](https://discuss.elastic.co/u/thirty2)\ **Replies:** 2\ **Last updated:** [May 25, 2021, 6:45am UTC](https://discuss.elastic.co/t/avro-to-json/273820 "2021-05-25T06:45:48Z")
Hi, I am consuming data from Kafka with INPUT configuration: input { kafka { codec =\> avro { schema\_uri =\> "/etc/logstash/avro.avsc" } key\_deserializer\_class =\> "org.apach… --- ## [Stuck at Successfully started Logstash API endpoint {:port=\>9601}](https://discuss.elastic.co/t/stuck-at-successfully-started-logstash-api-endpoint-port-9601/273890)
**Author:** [@dinesh5](https://discuss.elastic.co/u/dinesh5)\ **Replies:** 4\ **Last updated:** [May 25, 2021, 6:10am UTC](https://discuss.elastic.co/t/stuck-at-successfully-started-logstash-api-endpoint-port-9601/273890 "2021-05-25T06:10:24Z")
Hello Please help me out I'm stuck at Successfully started Logstash API endpoint {:port=\>9601} my conf file input { file { path =\> "/home/lsdp/Desktop/covid2405.csv" start\_position =\> "beginning" sincedb\_path =\> "… --- ## [How can I match timestamp](https://discuss.elastic.co/t/how-can-i-match-timestamp/273612)
**Author:** [@BAIBAA](https://discuss.elastic.co/u/BAIBAA)\ **Replies:** 3\ **Last updated:** [May 25, 2021, 3:44am UTC](https://discuss.elastic.co/t/how-can-i-match-timestamp/273612 "2021-05-25T03:44:22Z")
--- ## [Logstash configuration error](https://discuss.elastic.co/t/logstash-configuration-error/273652)
**Author:** [@Rizky\_Hudha](https://discuss.elastic.co/u/Rizky_Hudha)\ **Replies:** 1\ **Last updated:** [May 25, 2021, 2:59am UTC](https://discuss.elastic.co/t/logstash-configuration-error/273652 "2021-05-25T02:59:34Z")
hello, I have a problem when I try to configure the logstash here are the errors that appear Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs erro… --- ## [Make idices](https://discuss.elastic.co/t/make-idices/273868)
**Author:** [@majidbarz](https://discuss.elastic.co/u/majidbarz)\ **Replies:** 1\ **Last updated:** [May 25, 2021, 2:07am UTC](https://discuss.elastic.co/t/make-idices/273868 "2021-05-25T02:07:05Z")
HI I use winlogbeats i want to get log from winlogbeats and send to logstash then logstash import to elasticsearch. default in /etc/logstash/conf.d doesnt exist winlogbeat and i use pipelines.yaml and make new file but … --- ## [Error No Available connections logstash a elastic](https://discuss.elastic.co/t/error-no-available-connections-logstash-a-elastic/273568)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 0\ **Last updated:** [May 20, 2021, 4:47pm UTC](https://discuss.elastic.co/t/error-no-available-connections-logstash-a-elastic/273568 "2021-05-20T16:47:47Z")
Good morning, I have encountered the following error, and I do not know how to solve this problem in definitive; we have tried connections and everything is correct, we have tried telnetting to elastic and everything is … --- ## [Logstash data is tripled](https://discuss.elastic.co/t/logstash-data-is-tripled/273671)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 0\ **Last updated:** [May 21, 2021, 2:08pm UTC](https://discuss.elastic.co/t/logstash-data-is-tripled/273671 "2021-05-21T14:08:44Z")
Hello, I am monitoring several computers in different configurations in logstash, they are for 1 minute that I send data from logstash to kibana, but I have seen in discover that the data are appearing tripled and I have… --- ## [Ruby code works in an online IDE but not in logstash pipeline](https://discuss.elastic.co/t/ruby-code-works-in-an-online-ide-but-not-in-logstash-pipeline/273874)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 2\ **Last updated:** [May 24, 2021, 10:47pm UTC](https://discuss.elastic.co/t/ruby-code-works-in-an-online-ide-but-not-in-logstash-pipeline/273874 "2021-05-24T22:47:41Z")
\`Hi, Im using ruby scan method to get the values of some metrics in a line examples: 73% virtual memory used,4442MB virtual memory free,0.10955 pages-sec\` 0% swap space used,6125MB swap space free,0 pages-sec; 88% phys… --- ## [Automatic restart of logstash](https://discuss.elastic.co/t/automatic-restart-of-logstash/273860)
**Author:** [@Falikou1](https://discuss.elastic.co/u/Falikou1)\ **Replies:** 6\ **Last updated:** [May 24, 2021, 9:00pm UTC](https://discuss.elastic.co/t/automatic-restart-of-logstash/273860 "2021-05-24T21:00:31Z")
I would like the logstash service to restart automatically every Tuesday at 8:00 p.m. What type of configuration can you offer me? --- ## [Has anyone tried to load edifact or x12 messages](https://discuss.elastic.co/t/has-anyone-tried-to-load-edifact-or-x12-messages/273689)
**Author:** [@Bugsbee](https://discuss.elastic.co/u/Bugsbee)\ **Replies:** 2\ **Last updated:** [May 24, 2021, 3:56pm UTC](https://discuss.elastic.co/t/has-anyone-tried-to-load-edifact-or-x12-messages/273689 "2021-05-24T15:56:37Z")
hello all , has anyone tried to load x12 or edifact message using logstash/kibana. basically my requirement is simple.. I have the x12 or edifact messaged saved in a directory. i just need to pull that file and load t… --- ## [Joining JSON Array Values](https://discuss.elastic.co/t/joining-json-array-values/273702)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 3\ **Last updated:** [May 24, 2021, 3:16pm UTC](https://discuss.elastic.co/t/joining-json-array-values/273702 "2021-05-24T15:16:14Z")
I have data coming in as JSON structured like below. The JSON is parsed and the data under Charlie is displayed as an array of JSON. I'd like to concatenate all the values in URL to a single field, but can't seem to fi… --- ## [Logstash Error - RuntimeError: entity expansion has grown too large](https://discuss.elastic.co/t/logstash-error-runtimeerror-entity-expansion-has-grown-too-large/273808)
**Author:** [@Rosanna\_Staiano](https://discuss.elastic.co/u/Rosanna_Staiano)\ **Replies:** 2\ **Last updated:** [May 24, 2021, 3:06pm UTC](https://discuss.elastic.co/t/logstash-error-runtimeerror-entity-expansion-has-grown-too-large/273808 "2021-05-24T15:06:07Z")
Hello, I've a lot of xml-file containing log events and some of them are pretty large. I'm trying to parse them with Logstash, but when the xml is too large, I get an xmlparsefailure and I got this error: RuntimeError… --- ## [GROUP BY BASED ON MULTIPLE FIELDS](https://discuss.elastic.co/t/group-by-based-on-multiple-fields/273620)
**Author:** [@Keerthika](https://discuss.elastic.co/u/Keerthika)\ **Replies:** 3\ **Last updated:** [May 24, 2021, 3:05pm UTC](https://discuss.elastic.co/t/group-by-based-on-multiple-fields/273620 "2021-05-24T15:05:17Z")
Hi.. We need to load the data from Kafka to elastic search using logstash configuration. And also need to group by based on multiple fields (Last 1 hour data only(Millions of data) ) in logstash configuration. Kindly … --- ## [Logstash logs are not generating](https://discuss.elastic.co/t/logstash-logs-are-not-generating/273504)
**Author:** [@vikramdayma](https://discuss.elastic.co/u/vikramdayma)\ **Replies:** 1\ **Last updated:** [May 24, 2021, 8:31am UTC](https://discuss.elastic.co/t/logstash-logs-are-not-generating/273504 "2021-05-24T08:31:06Z")
Hi, I setup logstash on centos7. I done some work on it. now logstash logs is not generating. It could not able to find logstash.yml and log4j2.properties. There are warning; sudo /usr/share/logstash/bin/logstash -f… --- ## [What happens when logstash can't access to elasticsearch](https://discuss.elastic.co/t/what-happens-when-logstash-cant-access-to-elasticsearch/273783)
**Author:** [@rknd](https://discuss.elastic.co/u/rknd)\ **Replies:** 1\ **Last updated:** [May 24, 2021, 7:54am UTC](https://discuss.elastic.co/t/what-happens-when-logstash-cant-access-to-elasticsearch/273783 "2021-05-24T07:54:12Z")
I want to ask that, I have logstash pipelines. Output filter only goes elasticsearch. when elasticsearch stops while logstash is running, logstash service stops with an error but what about sincedb ? When i restart ela… --- ## [Logstash Regex](https://discuss.elastic.co/t/logstash-regex/273700)
**Author:** [@pmorenosi](https://discuss.elastic.co/u/pmorenosi)\ **Replies:** 1\ **Last updated:** [May 21, 2021, 10:05pm UTC](https://discuss.elastic.co/t/logstash-regex/273700 "2021-05-21T22:05:36Z")
Hello everybody, in a logstash pipeline I am doing the following "if" if \[message\] = ~ /^postfix.\*/anvil$/ { match =\> \["message", "% {POSTFIX\_ANVIL} $"\] } LOG May 11 05:49:37 smtp postfix / anvil \[27651\]: statistics… --- ## [Stashing your first event assistance](https://discuss.elastic.co/t/stashing-your-first-event-assistance/273769)
**Author:** [@jasieltego](https://discuss.elastic.co/u/jasieltego)\ **Replies:** 2\ **Last updated:** [May 23, 2021, 7:34pm UTC](https://discuss.elastic.co/t/stashing-your-first-event-assistance/273769 "2021-05-23T19:34:22Z")
Hi All, I'm following the guide to stash first event, but I'm getting an error message in the command line when verifying logstash installation. I can CD to logstash directory but it's when I specify configuration from … --- ## [Logstash not working in docker](https://discuss.elastic.co/t/logstash-not-working-in-docker/273748)
**Author:** [@Shubham\_Pachpande](https://discuss.elastic.co/u/Shubham_Pachpande)\ **Replies:** 1\ **Last updated:** [May 23, 2021, 7:15pm UTC](https://discuss.elastic.co/t/logstash-not-working-in-docker/273748 "2021-05-23T19:15:09Z")
Using bundled JDK: /usr/share/logstash/jdk OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. Sending Logstash logs to /usr/share/l… --- ## [What is the difference between fields pass by filebeat and fields in logstash?](https://discuss.elastic.co/t/what-is-the-difference-between-fields-pass-by-filebeat-and-fields-in-logstash/273727)
**Author:** [@stille](https://discuss.elastic.co/u/stille)\ **Replies:** 5\ **Last updated:** [May 23, 2021, 3:37am UTC](https://discuss.elastic.co/t/what-is-the-difference-between-fields-pass-by-filebeat-and-fields-in-logstash/273727 "2021-05-23T03:37:35Z")
I'm a new guy about elk. i'm trying use logstash filter instead of filebeat processors. There are some fields passed by filebeat in logstash , for example: i had set some fields in filebeat: "fields.app"="nginx" or "do… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=225) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=227)