# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=599 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 600 --- ## [Can not convert from long to float logstash](https://discuss.elastic.co/t/can-not-convert-from-long-to-float-logstash/168577)
**Author:** [@KashiPashoria\_Tonny](https://discuss.elastic.co/u/KashiPashoria_Tonny)\ **Replies:** 2\ **Last updated:** [February 18, 2019, 12:23pm UTC](https://discuss.elastic.co/t/can-not-convert-from-long-to-float-logstash/168577 "2019-02-18T12:23:28Z")
I'm taking data from sql server to elasticsearch 6.6 using logstash and jdbc. Some of my records are skipped by logstash printing these warning logs: \[2019-02-15T16:48:52,630\]\[WARN \]\[logstash.outputs.elasticsearch\] Coul… --- ## [Duplication of rows in jdbc logstash input - Using stored procedure](https://discuss.elastic.co/t/duplication-of-rows-in-jdbc-logstash-input-using-stored-procedure/168803)
**Author:** [@Giripratap\_Singh](https://discuss.elastic.co/u/Giripratap_Singh)\ **Replies:** 1\ **Last updated:** [February 18, 2019, 12:19pm UTC](https://discuss.elastic.co/t/duplication-of-rows-in-jdbc-logstash-input-using-stored-procedure/168803 "2019-02-18T12:19:51Z")
I'm fetching data from jdbc input plugin. After running through the whole procedure logstash stops. Problem is whenever the new rows are added, I run the .conf file and it duplicates the previous data. I have read only … --- ## [Logstash.yml confilct when configuring modules in it. Using the official elastic logstash docker image](https://discuss.elastic.co/t/logstash-yml-confilct-when-configuring-modules-in-it-using-the-official-elastic-logstash-docker-image/168455)
**Author:** [@solstarz](https://discuss.elastic.co/u/solstarz)\ **Replies:** 6\ **Last updated:** [February 15, 2019, 9:50am UTC](https://discuss.elastic.co/t/logstash-yml-confilct-when-configuring-modules-in-it-using-the-official-elastic-logstash-docker-image/168455 "2019-02-15T09:50:13Z")
Hi, I have configured the azure module in logstash.yml, but after the container starts im receiving this error: ERROR: Settings 'path.config' (-f) or 'config.string' (-e) can't be used in conjunction with (--modules) o… --- ## [How to convert String to Timestamp](https://discuss.elastic.co/t/how-to-convert-string-to-timestamp/168793)
**Author:** [@priya08](https://discuss.elastic.co/u/priya08)\ **Replies:** 1\ **Last updated:** [February 18, 2019, 9:29am UTC](https://discuss.elastic.co/t/how-to-convert-string-to-timestamp/168793 "2019-02-18T09:29:57Z")
Hi , This is sample log statement \[INFO\]||datetime:20190218142929801+0530|hostname:sampleHostName|threadId:sampleThread|userId:null|className:SampleClass|methodName:samplemethod I want to convert datetime(String) fiel… --- ## [How to use Beat with Amazon S3 logs as input?](https://discuss.elastic.co/t/how-to-use-beat-with-amazon-s3-logs-as-input/168792)
**Author:** [@Pradip\_Sakhavala](https://discuss.elastic.co/u/Pradip_Sakhavala)\ **Replies:** 0\ **Last updated:** [February 18, 2019, 9:25am UTC](https://discuss.elastic.co/t/how-to-use-beat-with-amazon-s3-logs-as-input/168792 "2019-02-18T09:25:53Z")
Hi, I have logs stored on AWS s3. I am using s3-input-plugin in logstash as input to read that logs. Now, I want to setup another logstash with same logs. I want to read logs twice from s3 because of network-out charges… --- ## [Logstash ruby hash](https://discuss.elastic.co/t/logstash-ruby-hash/168788)
**Author:** [@fedwe](https://discuss.elastic.co/u/fedwe)\ **Replies:** 0\ **Last updated:** [February 18, 2019, 9:00am UTC](https://discuss.elastic.co/t/logstash-ruby-hash/168788 "2019-02-18T09:00:08Z")
How do i make 2 hash keys for the same value for a csv reference in ruby filter. like i have an id field and a name field and a description field in a csv file and i have same ids but different names that have different… --- ## [Logstash ruby conditions and referrence](https://discuss.elastic.co/t/logstash-ruby-conditions-and-referrence/168784)
**Author:** [@fedwe](https://discuss.elastic.co/u/fedwe)\ **Replies:** 0\ **Last updated:** [February 18, 2019, 8:40am UTC](https://discuss.elastic.co/t/logstash-ruby-conditions-and-referrence/168784 "2019-02-18T08:40:48Z")
hello i am still new in ruby coding and i want to ask how can i apply these conditions in ruby and my case is that i receive different numbers from different countries like +962123456 or 00962123456 or 962123456.i have a… --- ## [Logstash + unknown setting seperator for csv](https://discuss.elastic.co/t/logstash-unknown-setting-seperator-for-csv/168692)
**Author:** [@bharathi1](https://discuss.elastic.co/u/bharathi1)\ **Replies:** 3\ **Last updated:** [February 18, 2019, 8:36am UTC](https://discuss.elastic.co/t/logstash-unknown-setting-seperator-for-csv/168692 "2019-02-18T08:36:11Z")
2019-02-16T20:07:07,894\]\[WARN \]\[logstash.config.source.multilocal\] Ignoring the 'pipelines.yml' file because modules or command line options are specified \[2019-02-16T20:07:07,910\]\[INFO \]\[logstash.runner \] Star… --- ## [Logstash 6.5.4 does not output to output file](https://discuss.elastic.co/t/logstash-6-5-4-does-not-output-to-output-file/167084)
**Author:** [@Alexas](https://discuss.elastic.co/u/Alexas)\ **Replies:** 5\ **Last updated:** [February 18, 2019, 5:39am UTC](https://discuss.elastic.co/t/logstash-6-5-4-does-not-output-to-output-file/167084 "2019-02-18T05:39:03Z")
Hello! I have an installed version of logstash.version "=\>" 6.5.4 " I start logstash: sudo initctl start logstash I want to execute the script recorded in the file 777.conf input {file {path =\> "/ home / sonic / log … --- ## [How do I split the json string into tab characters?](https://discuss.elastic.co/t/how-do-i-split-the-json-string-into-tab-characters/168759)
**Author:** [@Jin\_Pyo\_Ahn](https://discuss.elastic.co/u/Jin_Pyo_Ahn)\ **Replies:** 1\ **Last updated:** [February 18, 2019, 12:35am UTC](https://discuss.elastic.co/t/how-do-i-split-the-json-string-into-tab-characters/168759 "2019-02-18T00:35:55Z")
{"doc":""2019-02-18 09:00:37.740\\tma/1.0\\timpression\\tapp:\\tkr.co.auction.auctionbrowser.6.7.0\\t/page/home/homemain\\t-\\tlink\\t100000307\\tASN=2&txtMD=\\t1895ef\\tm\\t41\\tfalse\\tMobileApp/1.2 (iOS; 6.7.0; kr.co.auction.Auctio… --- ## [What does "auth" =\> "-" mean?](https://discuss.elastic.co/t/what-does-auth-mean/168723)
**Author:** [@li\_jessen2016](https://discuss.elastic.co/u/li_jessen2016)\ **Replies:** 3\ **Last updated:** [February 17, 2019, 10:44pm UTC](https://discuss.elastic.co/t/what-does-auth-mean/168723 "2019-02-17T22:44:56Z")
Does it mean the value of the field "auth" is missing, or in other words, not presented in the source message? --- ## [How to Parse Json log format by ConfigMap in Logstash](https://discuss.elastic.co/t/how-to-parse-json-log-format-by-configmap-in-logstash/168438)
**Author:** [@aleksei.saiko](https://discuss.elastic.co/u/aleksei.saiko)\ **Replies:** 4\ **Last updated:** [February 17, 2019, 2:15pm UTC](https://discuss.elastic.co/t/how-to-parse-json-log-format-by-configmap-in-logstash/168438 "2019-02-17T14:15:41Z")
Hi, The logs that I'm receiving for message, looking like that in Kibana - message {"time\_date": "2019-02-14T14:00:39+00:00","client": "10.xxx.xxx.xxx", "host": "xxx.com", "scheme": "https", "request\_method": "… --- ## [Parse content type](https://discuss.elastic.co/t/parse-content-type/168690)
**Author:** [@Babak](https://discuss.elastic.co/u/Babak)\ **Replies:** 2\ **Last updated:** [February 17, 2019, 7:33am UTC](https://discuss.elastic.co/t/parse-content-type/168690 "2019-02-17T07:33:28Z")
I'm parsing a log message with arbitrary fields. In my message we have field within two quotations in these two formats "text/html" "text/html; charset=UTF-8" I can handle each one separately. For first one I use "%{W… --- ## [Logstash to Elastic search, Could not index event to Elasticsearch , "reason"=\>"mapper \[\] of different type, current\_type \[long\], merged\_type \[text\]](https://discuss.elastic.co/t/logstash-to-elastic-search-could-not-index-event-to-elasticsearch-reason-mapper-of-different-type-current-type-long-merged-type-text/168279)
**Author:** [@SuperTomcat1](https://discuss.elastic.co/u/SuperTomcat1)\ **Replies:** 2\ **Last updated:** [February 17, 2019, 1:07am UTC](https://discuss.elastic.co/t/logstash-to-elastic-search-could-not-index-event-to-elasticsearch-reason-mapper-of-different-type-current-type-long-merged-type-text/168279 "2019-02-17T01:07:06Z")
error i see in logstash logs: Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"arrayx1", :\_type=\>"doc", :routing=\>nil}, #\\], :response=\> {"index"… --- ## [Logstash filter mutate remove\_field inside of an array](https://discuss.elastic.co/t/logstash-filter-mutate-remove-field-inside-of-an-array/168503)
**Author:** [@SuperTomcat1](https://discuss.elastic.co/u/SuperTomcat1)\ **Replies:** 10\ **Last updated:** [February 17, 2019, 1:05am UTC](https://discuss.elastic.co/t/logstash-filter-mutate-remove-field-inside-of-an-array/168503 "2019-02-17T01:05:17Z")
so i was trying to filter out all references to "volumes.lun-mapping-list" in my json http\_poller input. so this field repeats multiple times but the data is not valuable and its an array inside of the an array so ES do… --- ## [Need help updating syslog conf and grok filter](https://discuss.elastic.co/t/need-help-updating-syslog-conf-and-grok-filter/168702)
**Author:** [@johnnyd](https://discuss.elastic.co/u/johnnyd)\ **Replies:** 5\ **Last updated:** [February 16, 2019, 10:00pm UTC](https://discuss.elastic.co/t/need-help-updating-syslog-conf-and-grok-filter/168702 "2019-02-16T22:00:55Z")
I am setting up my first ELK server. My first goal is to ingest firewall syslogs from a Mikrotik router. Second is to add other syslog sources. I have gotten to a part where I am stuck and am going in circles. I have … --- ## [Decode JWT token with logstash](https://discuss.elastic.co/t/decode-jwt-token-with-logstash/168696)
**Author:** [@yossiv](https://discuss.elastic.co/u/yossiv)\ **Replies:** 3\ **Last updated:** [February 16, 2019, 8:23pm UTC](https://discuss.elastic.co/t/decode-jwt-token-with-logstash/168696 "2019-02-16T20:23:16Z")
Hi Guys, i was wondering if there is any way to decode JWT token with logstash? i didnt find anything about this. --- ## [Logstash unable to read file in 6.6.0](https://discuss.elastic.co/t/logstash-unable-to-read-file-in-6-6-0/168672)
**Author:** [@shilpa.ks](https://discuss.elastic.co/u/shilpa.ks)\ **Replies:** 6\ **Last updated:** [February 16, 2019, 8:19pm UTC](https://discuss.elastic.co/t/logstash-unable-to-read-file-in-6-6-0/168672 "2019-02-16T20:19:10Z")
Hello Team, I have a .conf file which works perfectly fine with Logstash 6.3.2 but the same conf file is not working in 6.6.0 Please see below the conf file; input { #stdin { } file { path =\> \["C:\\ELK\\Test.txt"… --- ## [Differentiate between sections within a single log file](https://discuss.elastic.co/t/differentiate-between-sections-within-a-single-log-file/168478)
**Author:** [@Abhilash\_B](https://discuss.elastic.co/u/Abhilash_B)\ **Replies:** 7\ **Last updated:** [February 16, 2019, 7:34pm UTC](https://discuss.elastic.co/t/differentiate-between-sections-within-a-single-log-file/168478 "2019-02-16T19:34:42Z")
Hey, I have a log file which looks like this. Its basically a tab separated text file with a few lines of metadata text. I would want to extract "TIME" only as part of metadata field. For other tab separated lines, I wo… --- ## [How to match multi ip and port](https://discuss.elastic.co/t/how-to-match-multi-ip-and-port/168703)
**Author:** [@nurhambali](https://discuss.elastic.co/u/nurhambali)\ **Replies:** 6\ **Last updated:** [February 16, 2019, 7:22pm UTC](https://discuss.elastic.co/t/how-to-match-multi-ip-and-port/168703 "2019-02-16T19:22:05Z")
hi, I try to separate ip and port: src="10.20.20.131:0 125.167.15.161:0 147.75.111.32:0 223.202.110.11:0 36.74.149.118:0 36.86.63.182:0 203.190.242.244:0 " in this way (?%{IP}( %{IP})\*) but the results are only one i… --- ## [Router syslog input, how to grok](https://discuss.elastic.co/t/router-syslog-input-how-to-grok/168687)
**Author:** [@johnnyd](https://discuss.elastic.co/u/johnnyd)\ **Replies:** 0\ **Last updated:** [February 16, 2019, 2:23pm UTC](https://discuss.elastic.co/t/router-syslog-input-how-to-grok/168687 "2019-02-16T14:23:24Z")
Ignore this post --- ## [Percentage Calculation in Ruby](https://discuss.elastic.co/t/percentage-calculation-in-ruby/168700)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 1\ **Last updated:** [February 16, 2019, 5:28pm UTC](https://discuss.elastic.co/t/percentage-calculation-in-ruby/168700 "2019-02-16T17:28:07Z")
I am doing calculation in Ruby but the fields are empty in Kibana. The ruby code: event.set('BusinessTimeUnavailabilityinSeconds', ((event.get('BusinessUnavailabilityHours').to\_i \* 3600 ) + (event.get… --- ## [CSV with fields in Portuguese](https://discuss.elastic.co/t/csv-with-fields-in-portuguese/168683)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 2\ **Last updated:** [February 16, 2019, 4:13pm UTC](https://discuss.elastic.co/t/csv-with-fields-in-portuguese/168683 "2019-02-16T16:13:12Z")
I have a csv file with some of the fields in Portuguese. In the input of logstash I wrote: codec =\> plain { charset=\>"UTF-8" } I am getting the following warnings: \[2019-02-16T14:38:25,876\]\[WARN \]\[logstash.codecs.p… --- ## [Monitor TomCat Logs](https://discuss.elastic.co/t/monitor-tomcat-logs/168693)
**Author:** [@dfoley84](https://discuss.elastic.co/u/dfoley84)\ **Replies:** 1\ **Last updated:** [February 16, 2019, 3:38pm UTC](https://discuss.elastic.co/t/monitor-tomcat-logs/168693 "2019-02-16T15:38:07Z")
I am trying to Monitor our Tomcat logs with Elastic Search / LogStash when i try and create a mapping: curl -XPUT "http://:9200/logs/" -d '{ "mappings" : { "tomcat\_access":{ "discover":".\*"}}}' I am getting the follow… --- ## [Import csv file to ES but date isn't correct](https://discuss.elastic.co/t/import-csv-file-to-es-but-date-isnt-correct/168689)
**Author:** [@Thodoris\_Alexopoulos](https://discuss.elastic.co/u/Thodoris_Alexopoulos)\ **Replies:** 1\ **Last updated:** [February 16, 2019, 2:44pm UTC](https://discuss.elastic.co/t/import-csv-file-to-es-but-date-isnt-correct/168689 "2019-02-16T14:44:08Z")
Hello, I am trying to import a CSV file to the elasticsearch using logstash but i am facing a problem with the date. csv file 2019-02-06 00 ,1.48,5.17,3.43,737.13,2049.12,1752.04 2019-02-06 01 ,1.80,7.14,4.41,762.62,2… --- ## [Grok pars failure](https://discuss.elastic.co/t/grok-pars-failure/168682)
**Author:** [@Hamed\_khosravi](https://discuss.elastic.co/u/Hamed_khosravi)\ **Replies:** 1\ **Last updated:** [February 16, 2019, 1:18pm UTC](https://discuss.elastic.co/t/grok-pars-failure/168682 "2019-02-16T13:18:05Z")
hi dears, i want to analyze log files like this: """""""" date=2018-11-07 time=12:49:48 bid=32679715 dvid=1085 itime=1541575641 log\_id="20000018" type="attack" subtype="N/A" pri="None" msg\_id=560216191 timezone="None"… --- ## [Ruby SyntaxError - unexpected tCONSTANT](https://discuss.elastic.co/t/ruby-syntaxerror-unexpected-tconstant/168676)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 1\ **Last updated:** [February 16, 2019, 12:26pm UTC](https://discuss.elastic.co/t/ruby-syntaxerror-unexpected-tconstant/168676 "2019-02-16T12:26:53Z")
Hi, That is my error I am getting when Logstash start: error: (SyntaxError) (ruby filter code):15: syntax error, unexpected tCONSTANT event.set('NonPrimeTimeUnavailabilityinPrecentage, ((event.get('NonPrimeTimeUnavail… --- ## [CSV filter - How can I parse unquoted CSV?](https://discuss.elastic.co/t/csv-filter-how-can-i-parse-unquoted-csv/168566)
**Author:** [@Skeeve](https://discuss.elastic.co/u/Skeeve)\ **Replies:** 7\ **Last updated:** [February 15, 2019, 10:05pm UTC](https://discuss.elastic.co/t/csv-filter-how-can-i-parse-unquoted-csv/168566 "2019-02-15T22:05:55Z")
My Data looks similar to this: 2019-02-15T10:56:00.000+000\\t{"test":"value","more":{"property":"value"}} but with a lot more fields. Issue here is that one field is a JSON structure which is unquoted. When I do not set… --- ## [Logstash pipeline and agent errors in docker logs](https://discuss.elastic.co/t/logstash-pipeline-and-agent-errors-in-docker-logs/168651)
**Author:** [@jwalls](https://discuss.elastic.co/u/jwalls)\ **Replies:** 3\ **Last updated:** [February 15, 2019, 9:15pm UTC](https://discuss.elastic.co/t/logstash-pipeline-and-agent-errors-in-docker-logs/168651 "2019-02-15T21:15:30Z")
I'm running logstash 6.5.4 in a docker container with the following conf file: input { redis { host =\> "${REDIS\_0\_HOST}" port =\> "${REDIS\_0\_PORT}" data\_type =\> "list" key =\> "logstash" } } input { … --- ## [Could not index event to Elasticsearch error](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error/168629)
**Author:** [@jwalls](https://discuss.elastic.co/u/jwalls)\ **Replies:** 3\ **Last updated:** [February 15, 2019, 8:42pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-error/168629 "2019-02-15T20:42:30Z")
I'm not getting any logs stored, and when checking the docker logs i have this repeated over and over \[2019-02-15T18:07:03,579\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>40… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=598) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=600)