# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=728 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 729 --- ## [Logstash output not sending data to Influxdb](https://discuss.elastic.co/t/logstash-output-not-sending-data-to-influxdb/138344)
**Author:** [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Replies:** 5\ **Last updated:** [July 5, 2018, 5:22am UTC](https://discuss.elastic.co/t/logstash-output-not-sending-data-to-influxdb/138344 "2018-07-05T05:22:51Z")
Hi All, I have installed metricbeat (Version 6.3.0) in my ubuntu machine and sending data to logstash output and I am running logstash (5.6.10 Version) to send metricbeat data to influxdb and below is my logstash confi… --- ## [Multiple CSV files(100types) with different formats,how to write a logstash config for the same](https://discuss.elastic.co/t/multiple-csv-files-100types-with-different-formats-how-to-write-a-logstash-config-for-the-same/138621)
**Author:** [@roity](https://discuss.elastic.co/u/roity)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 3:40am UTC](https://discuss.elastic.co/t/multiple-csv-files-100types-with-different-formats-how-to-write-a-logstash-config-for-the-same/138621 "2018-07-05T03:40:08Z")
Hi All, How are you?? I am a begineer and I have given this project for the first time and I am very hopeful.So sorry,If this question is silly. Actually I have so many types of files ,approx 100 or greater.I have to … --- ## [Logstash aggregated error](https://discuss.elastic.co/t/logstash-aggregated-error/138031)
**Author:** [@andradedb](https://discuss.elastic.co/u/andradedb)\ **Replies:** 2\ **Last updated:** [July 4, 2018, 11:20pm UTC](https://discuss.elastic.co/t/logstash-aggregated-error/138031 "2018-07-04T23:20:43Z")
I'm trying to aggregated some results from my statement and i was following this example over here: https://www.javacodegeeks.com/2017/10/aggregate-index-data-elasticsearch-using-logstash-jdbc.html But I'm getting some … --- ## [Pipeline - Error registering plugin](https://discuss.elastic.co/t/pipeline-error-registering-plugin/138525)
**Author:** [@qiuxing](https://discuss.elastic.co/u/qiuxing)\ **Replies:** 1\ **Last updated:** [July 5, 2018, 1:01am UTC](https://discuss.elastic.co/t/pipeline-error-registering-plugin/138525 "2018-07-05T01:01:15Z")
\[2018-07-04T17:52:25,940\]\[WARN \]\[logstash.outputs.elasticsearch\] You are using a deprecated config setting "document\_type" set in elasticsearch. Deprecated settings will continue to work, but are scheduled for removal fr… --- ## [I want to upload two csv files of different format simultaneously](https://discuss.elastic.co/t/i-want-to-upload-two-csv-files-of-different-format-simultaneously/138612)
**Author:** [@Krishna2](https://discuss.elastic.co/u/Krishna2)\ **Replies:** 0\ **Last updated:** [July 5, 2018, 12:48am UTC](https://discuss.elastic.co/t/i-want-to-upload-two-csv-files-of-different-format-simultaneously/138612 "2018-07-05T00:48:33Z")
Hi I want to upload two different csv files simultaneously into Logstash. Two files have different format etc. so in the configuration csv {} part I have different columns. Both have different index. I cannot use any … --- ## [Logstash outputs elasticsearch: Marking url as dead](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead/138204)
**Author:** [@anton-xqz](https://discuss.elastic.co/u/anton-xqz)\ **Replies:** 1\ **Last updated:** [July 4, 2018, 6:36pm UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-marking-url-as-dead/138204 "2018-07-04T18:36:55Z")
Hello, We are running ELK cluster on ec2 instance. Elasticsearch cluster in docker on ec2 instances in 3 regions. 33 nodes at total (i3.2xlarge for hot nodes and m5.4xlarge for warm nodes, 25TB data at total, ~2TB data… --- ## [Is it possible to include regex in dissect filter?](https://discuss.elastic.co/t/is-it-possible-to-include-regex-in-dissect-filter/138553)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 4\ **Last updated:** [July 4, 2018, 3:46pm UTC](https://discuss.elastic.co/t/is-it-possible-to-include-regex-in-dissect-filter/138553 "2018-07-04T15:46:03Z")
Hi, I am using ELK GA 6.3.0. I am using Logstash to read data from Kafka. I have messages like; 1 0 1707820 142124 198704 511288 0 0 0 144 There is a white space at the beginning of message. There are sp… --- ## [Ruby exception - Logstash](https://discuss.elastic.co/t/ruby-exception-logstash/138587)
**Author:** [@He\_Pamela](https://discuss.elastic.co/u/He_Pamela)\ **Replies:** 0\ **Last updated:** [July 4, 2018, 3:09pm UTC](https://discuss.elastic.co/t/ruby-exception-logstash/138587 "2018-07-04T15:09:37Z")
I'm getting the ERROR \[ERROR\]\[logstash.filters.ruby \] Ruby exception occurred: End of file reached this is my ruby. ruby { init =\> " require 'net/http' require 'uri' " code =\> " direccion = event.get('DIRECC… --- ## ["Ruby exception occurred:" ( sometimes occurred and sometimes not)](https://discuss.elastic.co/t/ruby-exception-occurred-sometimes-occurred-and-sometimes-not/138575)
**Author:** [@pedrotcm](https://discuss.elastic.co/u/pedrotcm)\ **Replies:** 0\ **Last updated:** [July 4, 2018, 2:00pm UTC](https://discuss.elastic.co/t/ruby-exception-occurred-sometimes-occurred-and-sometimes-not/138575 "2018-07-04T14:00:56Z")
Hello, I'm using logstash 6.2.4 with jdbc input from mongodb. I can not figure out what the problem might be. Sometimes it runs without giving errors and sometimes it happens to give errors, being that it is the same pip… --- ## [Date filter plugin issue](https://discuss.elastic.co/t/date-filter-plugin-issue/138208)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 12\ **Last updated:** [July 4, 2018, 1:19pm UTC](https://discuss.elastic.co/t/date-filter-plugin-issue/138208 "2018-07-04T13:19:55Z")
Hey, I try to parse a duration field thanks to "date" filter plugin but i'm unable to do this correctly \>\< My log : 0h 41' 47" 175ms I tried : H'h' MM\\' ss'"' SSS'ms' But it doesn't work..I am a little confused with… --- ## [Logstash error arcsight module](https://discuss.elastic.co/t/logstash-error-arcsight-module/138559)
**Author:** [@firru](https://discuss.elastic.co/u/firru)\ **Replies:** 0\ **Last updated:** [July 4, 2018, 12:41pm UTC](https://discuss.elastic.co/t/logstash-error-arcsight-module/138559 "2018-07-04T12:41:52Z")
Dear Team, I am facing issue while installing arcsight module with elk. When trying to arcsight module command in the bin directory of logstash came across below error. \[2018-07-04T15:35:24,025\]\[ERROR\]\[logstash.license… --- ## [Failed to install template.,host unreachable](https://discuss.elastic.co/t/failed-to-install-template-host-unreachable/138554)
**Author:** [@Gokul\_15](https://discuss.elastic.co/u/Gokul_15)\ **Replies:** 2\ **Last updated:** [July 4, 2018, 12:02pm UTC](https://discuss.elastic.co/t/failed-to-install-template-host-unreachable/138554 "2018-07-04T12:02:47Z")
Gokul Kathirvel 1 second ago \[ERROR\]\[logstash.outputs.elasticsearch\] Failed to install template. {:message=\>"Elasticsearch Unreachable: \[http://localhost:9200/\]\[Manticore::SocketTimeout\] Read timed out", :class=\>"LogSt… --- ## [Forwarding cloudtrail and cloudwatch logs to Logstash](https://discuss.elastic.co/t/forwarding-cloudtrail-and-cloudwatch-logs-to-logstash/138542)
**Author:** [@MrThecrackerx](https://discuss.elastic.co/u/MrThecrackerx)\ **Replies:** 0\ **Last updated:** [July 4, 2018, 11:07am UTC](https://discuss.elastic.co/t/forwarding-cloudtrail-and-cloudwatch-logs-to-logstash/138542 "2018-07-04T11:07:54Z")
Hello all, I am experimenting something. I have a aws instances where i have to forward cloudtrail and cloudwatch to a logstash instance which is not located in AWS. Can this be achieved ? https://www.elastic.co/guide/… --- ## [AD - Winlogbeat - Consecutive failures to logon to specific set of computers](https://discuss.elastic.co/t/ad-winlogbeat-consecutive-failures-to-logon-to-specific-set-of-computers/138105)
**Author:** [@spravveen](https://discuss.elastic.co/u/spravveen)\ **Replies:** 3\ **Last updated:** [July 4, 2018, 11:06am UTC](https://discuss.elastic.co/t/ad-winlogbeat-consecutive-failures-to-logon-to-specific-set-of-computers/138105 "2018-07-04T11:06:18Z")
Hi, I have an reporting / search requirement on Active Directory events. I am using winlogbeat to read events out of windows event log. Streaming these events to LogStash and further on to a log file and elasticsearch. … --- ## [Removing spaces in long log text file using grok](https://discuss.elastic.co/t/removing-spaces-in-long-log-text-file-using-grok/138515)
**Author:** [@serozhka](https://discuss.elastic.co/u/serozhka)\ **Replies:** 1\ **Last updated:** [July 4, 2018, 11:04am UTC](https://discuss.elastic.co/t/removing-spaces-in-long-log-text-file-using-grok/138515 "2018-07-04T11:04:15Z")
I have long entry in my text file (just a string), which cannot be controlled, and there are a lot of spaces. So is there a possibility to remove them before pushing to Kafka? I need just simple grok filter with Greedyda… --- ## [message=\>"Expected one of #](https://discuss.elastic.co/t/message-expected-one-of/138496)
**Author:** [@Gokul\_15](https://discuss.elastic.co/u/Gokul_15)\ **Replies:** 1\ **Last updated:** [July 4, 2018, 10:54am UTC](https://discuss.elastic.co/t/message-expected-one-of/138496 "2018-07-04T10:54:10Z")
Sending Logstash's logs to C:/logstash-6.3.0/logs which is now configured via log4j2.properties \[2018-07-04T12:49:04,756\]\[WARN \]\[logstash.config.source.multilocal\] Ignoring the 'pipelines.yml' file because modules or co… --- ## [Can't Get Custom Grok Filter To Work](https://discuss.elastic.co/t/cant-get-custom-grok-filter-to-work/138505)
**Author:** [@Alper\_Mehmet\_Ozdemir](https://discuss.elastic.co/u/Alper_Mehmet_Ozdemir)\ **Replies:** 1\ **Last updated:** [July 4, 2018, 10:28am UTC](https://discuss.elastic.co/t/cant-get-custom-grok-filter-to-work/138505 "2018-07-04T10:28:04Z")
Hi, I am new to elastic search and started using logstash to parse unstructured logs. I am using a grok filter to split the logs as much as ı understood from the documentation but it doesn't work. It just uploads every l… --- ## [Logstash 6.2.4 high CPU usage](https://discuss.elastic.co/t/logstash-6-2-4-high-cpu-usage/138347)
**Author:** [@vzh](https://discuss.elastic.co/u/vzh)\ **Replies:** 3\ **Last updated:** [July 4, 2018, 10:23am UTC](https://discuss.elastic.co/t/logstash-6-2-4-high-cpu-usage/138347 "2018-07-04T10:23:47Z")
Hello. Since we've updated logstash from 5.4.2 to 6.2.4 version we have problem with high using of CPU. As you can see on screenshots this high usage is not depend on message rate: All config files are the same. We… --- ## [Logstash-multiple inputs one output problem](https://discuss.elastic.co/t/logstash-multiple-inputs-one-output-problem/138417)
**Author:** [@dnastala](https://discuss.elastic.co/u/dnastala)\ **Replies:** 3\ **Last updated:** [July 4, 2018, 9:59am UTC](https://discuss.elastic.co/t/logstash-multiple-inputs-one-output-problem/138417 "2018-07-04T09:59:39Z")
Hi all, I have a configuration misery to solve. In my current setup I have one logstash server version 5.5.1 that takes two inputs(has to be only one logstash server). One is from lumberjack and the other one is from … --- ## [Removing nested fields with empty field names](https://discuss.elastic.co/t/removing-nested-fields-with-empty-field-names/138406)
**Author:** [@Mugen](https://discuss.elastic.co/u/Mugen)\ **Replies:** 3\ **Last updated:** [July 4, 2018, 7:41am UTC](https://discuss.elastic.co/t/removing-nested-fields-with-empty-field-names/138406 "2018-07-04T07:41:48Z")
Hi guys, I've got the following problem: Some of my events somehow get sent to logstash with a field consisiting of an empty name and an empty value (\[records\]\[conditions\]\[""\]). Of course Elasticsearch doesn't like thi… --- ## [Sending text log files using Logstash to Kafka](https://discuss.elastic.co/t/sending-text-log-files-using-logstash-to-kafka/138333)
**Author:** [@serozhka](https://discuss.elastic.co/u/serozhka)\ **Replies:** 7\ **Last updated:** [July 4, 2018, 8:31am UTC](https://discuss.elastic.co/t/sending-text-log-files-using-logstash-to-kafka/138333 "2018-07-04T08:31:32Z")
Hi! I have a question about correct setup of logstash in order to be able to send plain text log files in json format to Kafka clusters. In my logstash.conf I have: input { file { path =\> "/app/server/default/logs/a… --- ## [Unable to find JRuby in centos server](https://discuss.elastic.co/t/unable-to-find-jruby-in-centos-server/138487)
**Author:** [@wailoont](https://discuss.elastic.co/u/wailoont)\ **Replies:** 0\ **Last updated:** [July 4, 2018, 6:22am UTC](https://discuss.elastic.co/t/unable-to-find-jruby-in-centos-server/138487 "2018-07-04T06:22:37Z")
Hi Elastic, I downloaded tar.gz for logstash from https://www.elastic.co/downloads/logstash. When i copy the tar to my server, i extract it, created a conf file. I try to run using ./logstash -f logstash.conf to start… --- ## [Query\_Template Format in Elastic Search filter plugin](https://discuss.elastic.co/t/query-template-format-in-elastic-search-filter-plugin/138482)
**Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 0\ **Last updated:** [July 4, 2018, 6:02am UTC](https://discuss.elastic.co/t/query-template-format-in-elastic-search-filter-plugin/138482 "2018-07-04T06:02:31Z")
Hi - I just trying to run a GET request with the dsl script using the query\_template config in ES filter plugin on logstash. I believe something is wrong with the JSON request . Could someone please help me out. templa… --- ## [How to manually commit in kafka logstash input plugin](https://discuss.elastic.co/t/how-to-manually-commit-in-kafka-logstash-input-plugin/138481)
**Author:** [@pankhuri](https://discuss.elastic.co/u/pankhuri)\ **Replies:** 0\ **Last updated:** [July 4, 2018, 5:58am UTC](https://discuss.elastic.co/t/how-to-manually-commit-in-kafka-logstash-input-plugin/138481 "2018-07-04T05:58:23Z")
I am a new user for this plugin and I cannot find how and when i can manually commit for a message received by this input plugin. All i can find is to set enable\_auto\_commit as False but where should i manually commit? --- ## [JSON parse error, original data now in message field - Unexpected character (',' (code 44))](https://discuss.elastic.co/t/json-parse-error-original-data-now-in-message-field-unexpected-character-code-44/138340)
**Author:** [@jbowker](https://discuss.elastic.co/u/jbowker)\ **Replies:** 5\ **Last updated:** [July 4, 2018, 5:30am UTC](https://discuss.elastic.co/t/json-parse-error-original-data-now-in-message-field-unexpected-character-code-44/138340 "2018-07-04T05:30:57Z")
I am trying to deploy this pipeline on Kibana/Logstash pipelines: input { rabbitmq { host =\> "xxxxxxxxxx" vhost =\> "/" queue =\> "outrights" port =\> 5672 heartbeat =\> 30 durable =\> true password =\> "xxxx… --- ## [Logstash exception while parsing Java Log file](https://discuss.elastic.co/t/logstash-exception-while-parsing-java-log-file/138369)
**Author:** [@satendr](https://discuss.elastic.co/u/satendr)\ **Replies:** 2\ **Last updated:** [July 4, 2018, 5:13am UTC](https://discuss.elastic.co/t/logstash-exception-while-parsing-java-log-file/138369 "2018-07-04T05:13:17Z")
HI , I am new to ELK and trying to implement parsing for our Log file , but when i run config file , its giving Parsing Exception. My logstash config file is: input { beats { port =\> "5044" } } filter { grok { … --- ## [Trouble with \\n in dissect filter mapping](https://discuss.elastic.co/t/trouble-with-n-in-dissect-filter-mapping/138353)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 2\ **Last updated:** [July 4, 2018, 5:03am UTC](https://discuss.elastic.co/t/trouble-with-n-in-dissect-filter-mapping/138353 "2018-07-04T05:03:45Z")
Hi, I am using ELK GA 6.3.0. I use Logstash to read messages from Kafka as JSON. I have a message like; \ \\\n \ I am using the dissect filter like; … --- ## [Tranlate filter for bind logs but multiple fields](https://discuss.elastic.co/t/tranlate-filter-for-bind-logs-but-multiple-fields/138474)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 0\ **Last updated:** [July 4, 2018, 4:28am UTC](https://discuss.elastic.co/t/tranlate-filter-for-bind-logs-but-multiple-fields/138474 "2018-07-04T04:28:58Z")
Hi, I am collecting bind9 logs through logstash; now I am thinking of using translation plugin but the my destination field varies and I want that to be appear in logs or should get indexed. Can someone please give me a… --- ## [Jdbc log parsing](https://discuss.elastic.co/t/jdbc-log-parsing/136232)
**Author:** [@hburnswell](https://discuss.elastic.co/u/hburnswell)\ **Replies:** 2\ **Last updated:** [July 3, 2018, 11:08pm UTC](https://discuss.elastic.co/t/jdbc-log-parsing/136232 "2018-07-03T23:08:53Z")
Hi All, I would like to parse a jdbc log and I'm just learning the process. The log looks like this: 40703|master|180615-09:36:22:315|7 ms|statement|UPDATE tasks SET p\_runningonclusternode = ? WHERE p\_runningoncluste… --- ## [Geoip database format](https://discuss.elastic.co/t/geoip-database-format/135838)
**Author:** [@Vitalijs\_Grinbergs](https://discuss.elastic.co/u/Vitalijs_Grinbergs)\ **Replies:** 0\ **Last updated:** [June 14, 2018, 6:56am UTC](https://discuss.elastic.co/t/geoip-database-format/135838 "2018-06-14T06:56:19Z")
Is there a way to use updated geoip database in .dat format? I've used mmdb tools to convert .mmdb ti .csv and updated it with private IP subnets and geolocation according to https://github.com/threatstream/mhn/wiki/Cus… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=727) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=729)