# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=509 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 510 --- ## [Logstash : failed to parse field \[@timestamp\] of type \[date\]](https://discuss.elastic.co/t/logstash-failed-to-parse-field-timestamp-of-type-date/190789)
**Author:** [@Sandeep\_Bind](https://discuss.elastic.co/u/Sandeep_Bind)\ **Replies:** 1\ **Last updated:** [July 16, 2019, 4:37pm UTC](https://discuss.elastic.co/t/logstash-failed-to-parse-field-timestamp-of-type-date/190789 "2019-07-16T16:37:25Z")
This is my logstash.conf file. I am trying to read the data from mysql database and send it to elastic search. I am getting below error. This is my index creation command : curl -X POST -H "Content-Type: application/js… --- ## [Logstash configuration returns different results](https://discuss.elastic.co/t/logstash-configuration-returns-different-results/190471)
**Author:** [@GitsBdr](https://discuss.elastic.co/u/GitsBdr)\ **Replies:** 14\ **Last updated:** [July 16, 2019, 4:33pm UTC](https://discuss.elastic.co/t/logstash-configuration-returns-different-results/190471 "2019-07-16T16:33:05Z")
Hi guys. Below my logstash configuration : input { file { path =\> "/tmp/file.csv" type =\> "myindex" start\_position =\> "beginning" } } filter { if (\[type\] == "myindex") { csv { skip\_header =\>… --- ## [\[geoip\]\[country\_name\] long value](https://discuss.elastic.co/t/geoip-country-name-long-value/190786)
**Author:** [@hardwarehacker](https://discuss.elastic.co/u/hardwarehacker)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 3:55pm UTC](https://discuss.elastic.co/t/geoip-country-name-long-value/190786 "2019-07-16T15:55:12Z")
Hi, The field \[geoip\]\[country\_name\] writes a long value in memcached instead of only the string: " DMZ1:ET instead of: DMZ1 I guess I am addressing it incorrectly. Is there any suggestion out there? Thanks! \ **Author:** [@GuillaumeYnov](https://discuss.elastic.co/u/GuillaumeYnov)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 2:03pm UTC](https://discuss.elastic.co/t/duplicate-row-csv/190750 "2019-07-16T14:03:50Z") I need a litle a bit of help, i am a total newbie on ELK, I want to add log to the same index every week, but i will get duplicate row in my output file, but i want only one occurence of each log. What i am suppose to … --- ## [How to get all thread names with there utilization using logstash-jmx plugin](https://discuss.elastic.co/t/how-to-get-all-thread-names-with-there-utilization-using-logstash-jmx-plugin/190740)
**Author:** [@amritesh\_mishra](https://discuss.elastic.co/u/amritesh_mishra)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 1:25pm UTC](https://discuss.elastic.co/t/how-to-get-all-thread-names-with-there-utilization-using-logstash-jmx-plugin/190740 "2019-07-16T13:25:31Z")
We want to get the data per thread utilization for a java process using logstash-jmx plugin as well as the name of the thread which is currently running for that java process. Can you help on this so that we can do the … --- ## [Logstash Error with JDBC plugin](https://discuss.elastic.co/t/logstash-error-with-jdbc-plugin/189429)
**Author:** [@LouisP](https://discuss.elastic.co/u/LouisP)\ **Replies:** 3\ **Last updated:** [July 16, 2019, 1:23pm UTC](https://discuss.elastic.co/t/logstash-error-with-jdbc-plugin/189429 "2019-07-16T13:23:46Z")
Hi, I used logstash with elasticsearch for a few days now, everything was working fine: with logstash, I was querying a MySQL table and indexing it in elasticsearch to visualize it in Kibana. Then, logstash stopped worki… --- ## [Grok handling logs with less information](https://discuss.elastic.co/t/grok-handling-logs-with-less-information/188814)
**Author:** [@John\_Lim](https://discuss.elastic.co/u/John_Lim)\ **Replies:** 2\ **Last updated:** [July 16, 2019, 12:49pm UTC](https://discuss.elastic.co/t/grok-handling-logs-with-less-information/188814 "2019-07-16T12:49:21Z")
Hello again, I need an assistance on how could I extract logs with less information. For instance regular logs have full information while some logs have shorter information (see below), I tried adding two formats in gro… --- ## [Configure two types of data in logstash filter](https://discuss.elastic.co/t/configure-two-types-of-data-in-logstash-filter/190686)
**Author:** [@Ahmet\_Kartal](https://discuss.elastic.co/u/Ahmet_Kartal)\ **Replies:** 2\ **Last updated:** [July 16, 2019, 12:27pm UTC](https://discuss.elastic.co/t/configure-two-types-of-data-in-logstash-filter/190686 "2019-07-16T12:27:17Z")
This is my first data IP Adress - - \[01/Jul/2019:15:36:10 +0300\] "POST /search?page=1&page\_size=25 HTTP/1.1" 200 78 "https://try.com/search/?contains=pflichtteilauskunft" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:67… --- ## [Failed to send event to MongoDB, retrying in 3 seconds {:event=\>#\, :exception=\>#\ **Author:** [@Ricky\_Tokdis](https://discuss.elastic.co/u/Ricky_Tokdis)\ **Replies:** 3\ **Last updated:** [July 16, 2019, 12:06pm UTC](https://discuss.elastic.co/t/failed-to-send-event-to-mongodb-retrying-in-3-seconds-event-logstash-0x2545634-exception-argumenterror-wrong-number-of-arguments/190424 "2019-07-16T12:06:38Z") How to fix this problem? input { jdbc { jdbc\_driver\_library =\> "/etc/mysql-connector/mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar" jdbc\_driver\_class =\> "com.mysql.jdbc.Driver" jdbc\_connection\_stri… --- ## [Logstash not working with https elastic search output](https://discuss.elastic.co/t/logstash-not-working-with-https-elastic-search-output/190494)
**Author:** [@Sharma3007](https://discuss.elastic.co/u/Sharma3007)\ **Replies:** 4\ **Last updated:** [July 16, 2019, 11:54am UTC](https://discuss.elastic.co/t/logstash-not-working-with-https-elastic-search-output/190494 "2019-07-16T11:54:47Z")
Hi Team, I am getting below error while run log stash with https elastic search output: Here are the error details: \[2019-07-15T12:08:17,978\]\[ERROR\]\[logstash.licensechecker.licensereader\] Unable to retrieve license i… --- ## [Remote connection port](https://discuss.elastic.co/t/remote-connection-port/190719)
**Author:** [@amitavmohanty01](https://discuss.elastic.co/u/amitavmohanty01)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 11:25am UTC](https://discuss.elastic.co/t/remote-connection-port/190719 "2019-07-16T11:25:51Z")
As per the profiling blog, I was trying to connect remote Logstash process to VisualVM. However, I could not find the port setting. Where can I specify that? --- ## [Threads split among categories](https://discuss.elastic.co/t/threads-split-among-categories/190711)
**Author:** [@amitavmohanty01](https://discuss.elastic.co/u/amitavmohanty01)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 10:52am UTC](https://discuss.elastic.co/t/threads-split-among-categories/190711 "2019-07-16T10:52:20Z")
In Logstash, I am seeing 320 threads as per the stats API. As per my pipelines, 126 threads are Kafka input threads. The number of pipeline workers is default so it would be 4. I want to check what are all the threads th… --- ## [2 different logs match the filter, but only one is not parsed](https://discuss.elastic.co/t/2-different-logs-match-the-filter-but-only-one-is-not-parsed/190499)
**Author:** [@Kazesu](https://discuss.elastic.co/u/Kazesu)\ **Replies:** 4\ **Last updated:** [July 16, 2019, 10:50am UTC](https://discuss.elastic.co/t/2-different-logs-match-the-filter-but-only-one-is-not-parsed/190499 "2019-07-16T10:50:31Z")
Hello, I have a standalone setup, and recently, one of my machines is reaching the server with several different logs, mostly differing in the "syslog\_pri". One of them is not being parsed, and getting grokparsefailure, … --- ## [Parsing Windows Events Log using Logstash](https://discuss.elastic.co/t/parsing-windows-events-log-using-logstash/190693)
**Author:** [@gracia](https://discuss.elastic.co/u/gracia)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 9:15am UTC](https://discuss.elastic.co/t/parsing-windows-events-log-using-logstash/190693 "2019-07-16T09:15:42Z")
Is there a plugin to parse windows events logs using logstash? I have past, non-live logs in a Linux environment which I want to parse. Thanks! --- ## [Json forwarding (multiple fields)](https://discuss.elastic.co/t/json-forwarding-multiple-fields/190677)
**Author:** [@JensVanDeynse](https://discuss.elastic.co/u/JensVanDeynse)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 8:22am UTC](https://discuss.elastic.co/t/json-forwarding-multiple-fields/190677 "2019-07-16T08:22:11Z")
Hello I'm trying to set up json forwarding from our filebeats to our logstash which will eventually forward it to the correct index in ElasticSearch. For several logs this works but one specific one fails for us and it … --- ## [How to extract JSON content in message to top level](https://discuss.elastic.co/t/how-to-extract-json-content-in-message-to-top-level/190422)
**Author:** [@justAbeginner](https://discuss.elastic.co/u/justAbeginner)\ **Replies:** 5\ **Last updated:** [July 16, 2019, 7:50am UTC](https://discuss.elastic.co/t/how-to-extract-json-content-in-message-to-top-level/190422 "2019-07-16T07:50:17Z")
Sorry if it is a duplicated post. I have been searching google and the forum for answer, however I still could not find it. It would be nice if someone can direct me to the answer if it is duplicated. I am currently u… --- ## [Multiple logs are getting dumped because inode value of same file is different on each server(server that have auto scalling)](https://discuss.elastic.co/t/multiple-logs-are-getting-dumped-because-inode-value-of-same-file-is-different-on-each-server-server-that-have-auto-scalling/190668)
**Author:** [@Arpita\_Awasthi](https://discuss.elastic.co/u/Arpita_Awasthi)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 7:47am UTC](https://discuss.elastic.co/t/multiple-logs-are-getting-dumped-because-inode-value-of-same-file-is-different-on-each-server-server-that-have-auto-scalling/190668 "2019-07-16T07:47:47Z")
We have Linux servers in the cloud having "auto-scaling". The logs of the servers are created in the shared path(named /logs).PFA the format of the log path: Syntax : /logs///server.log Example: /logs/HOSTNAME0/2019-0… --- ## [Error while facing logstash startup - No Implicit conversion of nil into String](https://discuss.elastic.co/t/error-while-facing-logstash-startup-no-implicit-conversion-of-nil-into-string/190461)
**Author:** [@Mushtaqb](https://discuss.elastic.co/u/Mushtaqb)\ **Replies:** 2\ **Last updated:** [July 16, 2019, 7:40am UTC](https://discuss.elastic.co/t/error-while-facing-logstash-startup-no-implicit-conversion-of-nil-into-string/190461 "2019-07-16T07:40:12Z")
\[main\] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (TypeError) no implicit conversion of nil into String. Logstash went shutdown suddenly due to VM shutdown . When we try… --- ## [Scheduling logstash pipeline from postgresql to elasticsearch. ERROR: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/cronline.rb:77: warning: constant ::Fixnum is deprecated](https://discuss.elastic.co/t/scheduling-logstash-pipeline-from-postgresql-to-elasticsearch-error-usr-share-logstash-vendor-bundle-jruby-2-5-0-gems-rufus-scheduler-3-0-9-lib-rufus-scheduler-cronline-rb-warning-constant-fixnum-is-deprecated/186878)
**Author:** [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Replies:** 1\ **Last updated:** [July 16, 2019, 7:08am UTC](https://discuss.elastic.co/t/scheduling-logstash-pipeline-from-postgresql-to-elasticsearch-error-usr-share-logstash-vendor-bundle-jruby-2-5-0-gems-rufus-scheduler-3-0-9-lib-rufus-scheduler-cronline-rb-warning-constant-fixnum-is-deprecated/186878 "2019-07-16T07:08:29Z")
Hi, I'm trying to keep postsgresql synchronized with elasticsearch using logstash. To do this, I tried to insert the scheduling parameter into the input configuration pipeline: file.conf input { jdbc { # Postgres jd… --- ## [Filebeat publish and stop service in Idle scenarios](https://discuss.elastic.co/t/filebeat-publish-and-stop-service-in-idle-scenarios/190657)
**Author:** [@Subramanian\_Chellapp](https://discuss.elastic.co/u/Subramanian_Chellapp)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 7:02am UTC](https://discuss.elastic.co/t/filebeat-publish-and-stop-service-in-idle-scenarios/190657 "2019-07-16T07:02:13Z")
Folks, We have mapped the log path in filebeat.yml then logstash created new Index with given format and running on everyday successfully. We are using the below syntax to publish the filebeat manually to upload log ev… --- ## [Incorrect feed in current logstash index](https://discuss.elastic.co/t/incorrect-feed-in-current-logstash-index/189296)
**Author:** [@Subramanian\_Chellapp](https://discuss.elastic.co/u/Subramanian_Chellapp)\ **Replies:** 6\ **Last updated:** [July 16, 2019, 6:40am UTC](https://discuss.elastic.co/t/incorrect-feed-in-current-logstash-index/189296 "2019-07-16T06:40:57Z")
Folks, We have one master node and 2 cluster node which configured on AWS EFS volume end(data1, data2 and data3) . The current version of ELK 6.8. We have configured the below Filebeat for feeding IIS server log, file… --- ## [Filebeat -\> Logstash Connection ISSUE](https://discuss.elastic.co/t/filebeat-logstash-connection-issue/190181)
**Author:** [@Sundaramoorthy\_Anand](https://discuss.elastic.co/u/Sundaramoorthy_Anand)\ **Replies:** 8\ **Last updated:** [July 16, 2019, 6:16am UTC](https://discuss.elastic.co/t/filebeat-logstash-connection-issue/190181 "2019-07-16T06:16:03Z")
My ELK is in a server of xyzcorp.com of IP 16.xxx.xxx.xxx and filebeat is in xyz.com of IP 16.aaa.aaa.aaa Both are in different domains but in same intranet umbrella. In filebeat.yml, I'm enabling the output.logstas… --- ## [Logstash configuration: for journalbeat from multiple servers](https://discuss.elastic.co/t/logstash-configuration-for-journalbeat-from-multiple-servers/190475)
**Author:** [@K\_B1](https://discuss.elastic.co/u/K_B1)\ **Replies:** 4\ **Last updated:** [July 16, 2019, 6:00am UTC](https://discuss.elastic.co/t/logstash-configuration-for-journalbeat-from-multiple-servers/190475 "2019-07-16T06:00:46Z")
Hi there, I've been trying to collect logs, using journalbeat from multiple servers onto a single logstash server. I already have winlogbeat on my windows server which is using 5044, hence other servers with journalbeat … --- ## [Getting failed to flush outgoing items:webhdfs IOError. Does anyone know what happened and how to fix it?](https://discuss.elastic.co/t/getting-failed-to-flush-outgoing-items-webhdfs-ioerror-does-anyone-know-what-happened-and-how-to-fix-it/190645)
**Author:** [@Himanshu\_Vishwakarma](https://discuss.elastic.co/u/Himanshu_Vishwakarma)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 5:32am UTC](https://discuss.elastic.co/t/getting-failed-to-flush-outgoing-items-webhdfs-ioerror-does-anyone-know-what-happened-and-how-to-fix-it/190645 "2019-07-16T05:32:22Z")
logstash.outputs.webhdfs \] Failed to flush outgoing items {:outgoing\_count=\>1000, :exception=\>"WebHDFS::IOError", :backtrace=\>\["/var/logstash-6.2.4/vendor/bundle/jruby/2.3.0/gems/webhdfs-0.8.0/lib/webhdfs/client\_v1.rb:40… --- ## [Identifying events that cause logstash to hang](https://discuss.elastic.co/t/identifying-events-that-cause-logstash-to-hang/190641)
**Author:** [@gparks](https://discuss.elastic.co/u/gparks)\ **Replies:** 1\ **Last updated:** [July 16, 2019, 4:55am UTC](https://discuss.elastic.co/t/identifying-events-that-cause-logstash-to-hang/190641 "2019-07-16T04:55:34Z")
I have a pipeline for java logs configured with grok filter as follows grok { id =\> 'grok' match =\> {"message" =\> \["(?m)\\\[%{DATA:time}\\\]\\\[%{DATA:traceid}\\\] \\\[%{DATA:level}\\\] %{JAVACLASS:class} - (((?\.\*… --- ## [Logstash pipeline 한 개에 여러 Config 설정을 하고 싶습니다](https://discuss.elastic.co/t/logstash-pipeline-config/190632)
**Author:** [@pgw5540](https://discuss.elastic.co/u/pgw5540)\ **Replies:** 0\ **Last updated:** [July 16, 2019, 2:34am UTC](https://discuss.elastic.co/t/logstash-pipeline-config/190632 "2019-07-16T02:34:57Z")
안녕하세요. Logstash 한 개의 pipeline에 여러 config 설정을 하고 싶습니다. 수십개의 csv file을 읽어와서 동일한 개수의 각기 다른 index로 데이터를 보내고자 합니다. 따라서, 해당 개수만큼 config파일을 만들게 되었는데, 특정 index 데이터들은 하나의 파이프라인으로 묶어서 입력하고자 합니다. 하지만 한 개의 파이프라인 설정으로 여러개의 conf… --- ## [How to add nested log data or replace string values with nested log data on logstash.conf?](https://discuss.elastic.co/t/how-to-add-nested-log-data-or-replace-string-values-with-nested-log-data-on-logstash-conf/190222)
**Author:** [@kensaku-okada](https://discuss.elastic.co/u/kensaku-okada)\ **Replies:** 2\ **Last updated:** [July 16, 2019, 1:19am UTC](https://discuss.elastic.co/t/how-to-add-nested-log-data-or-replace-string-values-with-nested-log-data-on-logstash-conf/190222 "2019-07-16T01:19:59Z")
I want to do the data modification with logstash.conf like (see "host" key): From: { "host": "192.168.1.1", "CompanyNo": "9999999999998", "@timestamp": "2019-07-12T05:19:40.046Z", } To: { "host": {"name": "192.1… --- ## [Logstash file output truncate on startup](https://discuss.elastic.co/t/logstash-file-output-truncate-on-startup/190616)
**Author:** [@Sous\_Lesquels](https://discuss.elastic.co/u/Sous_Lesquels)\ **Replies:** 1\ **Last updated:** [July 15, 2019, 9:47pm UTC](https://discuss.elastic.co/t/logstash-file-output-truncate-on-startup/190616 "2019-07-15T21:47:31Z")
Is there a way to tell logstash file output plugin to truncate files on startup only? I would find this useful when testing. I.e. test the config, see the error, save an update to config, logstash re-reads the config an… --- ## [Logstash aggregate plugin with the same key](https://discuss.elastic.co/t/logstash-aggregate-plugin-with-the-same-key/190612)
**Author:** [@Sous\_Lesquels](https://discuss.elastic.co/u/Sous_Lesquels)\ **Replies:** 4\ **Last updated:** [July 15, 2019, 9:27pm UTC](https://discuss.elastic.co/t/logstash-aggregate-plugin-with-the-same-key/190612 "2019-07-15T21:27:16Z")
Assuming I have a file like this: start k1 v11 other lines end k1 i12 other lines other lines start k1 v21 anything else end k1 i22 anything else anything else anything else start k1 v31 anything else anything else … --- ## [Snmp grok filter failure](https://discuss.elastic.co/t/snmp-grok-filter-failure/190532)
**Author:** [@liqinnetgain](https://discuss.elastic.co/u/liqinnetgain)\ **Replies:** 6\ **Last updated:** [July 15, 2019, 8:36pm UTC](https://discuss.elastic.co/t/snmp-grok-filter-failure/190532 "2019-07-15T20:36:40Z")
Here's my line iso.3.6.1.4.1.2440.1.11.2.4.8.0 = Counter64: 0 And the filter input { stdin {} } input { snmp { get =\> \["1.3.6.1.4.1.2440.1.11.2.4.8.0"\] hosts =\> \[{host =\> "udp:10.0.10.108/… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=508) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=510)