# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=506 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 507 --- ## [\-e option not working in windows](https://discuss.elastic.co/t/e-option-not-working-in-windows/191791)
**Author:** [@varunjithtk](https://discuss.elastic.co/u/varunjithtk)\ **Replies:** 0\ **Last updated:** [July 23, 2019, 10:55am UTC](https://discuss.elastic.co/t/e-option-not-working-in-windows/191791 "2019-07-23T10:55:27Z")
New to elastic. Noted that following the steps mentioned in , command " bin/logstash -e 'input { stdin { } } output { stdout {} }'" doesnt work in windows. When I created a conf files with 'input { stdin { } } output {… --- ## [Logstash excute path](https://discuss.elastic.co/t/logstash-excute-path/191785)
**Author:** [@Abdelmadjid](https://discuss.elastic.co/u/Abdelmadjid)\ **Replies:** 0\ **Last updated:** [July 23, 2019, 10:44am UTC](https://discuss.elastic.co/t/logstash-excute-path/191785 "2019-07-23T10:44:14Z")
when i'm trying to excute data path in logstash with windows system i configure these error Could not find or load main class Files\\logstash\\logstash-core\\lib\\jars\\animal-sniffer-annotations-1.14.jar;C:Program --- ## [Logstash can't establish pipeline with ssl](https://discuss.elastic.co/t/logstash-cant-establish-pipeline-with-ssl/191434)
**Author:** [@johnbchron](https://discuss.elastic.co/u/johnbchron)\ **Replies:** 8\ **Last updated:** [July 23, 2019, 8:02am UTC](https://discuss.elastic.co/t/logstash-cant-establish-pipeline-with-ssl/191434 "2019-07-23T08:02:29Z")
So I have been trying to implement ssl security across my single-node stack, and I got kibana-elasticsearch ssl working (didn't enable https), but I'm having a hard time getting logstash to communicate with elasticsearch… --- ## [Removing substring from key and result](https://discuss.elastic.co/t/removing-substring-from-key-and-result/191730)
**Author:** [@shagun](https://discuss.elastic.co/u/shagun)\ **Replies:** 1\ **Last updated:** [July 23, 2019, 7:13am UTC](https://discuss.elastic.co/t/removing-substring-from-key-and-result/191730 "2019-07-23T07:13:37Z")
{ "x'property'" : "x'true'", "x'type'" : "x'user'" } I need to parse it and want to remove the x' and single quotes from both key and value pair like: { "property" : "true", … --- ## [Logstash file input plugin no output response](https://discuss.elastic.co/t/logstash-file-input-plugin-no-output-response/191559)
**Author:** [@dslee](https://discuss.elastic.co/u/dslee)\ **Replies:** 2\ **Last updated:** [July 23, 2019, 6:49am UTC](https://discuss.elastic.co/t/logstash-file-input-plugin-no-output-response/191559 "2019-07-23T06:49:37Z")
I am new to ELK so any comment is welcome. I am now using logstash to parse logs and I want to combine different lines of into one line by using codec =\> multiline plugin. Here is my config. input { #beats { # por… --- ## [Logstash is not accepting logs from rsyslog](https://discuss.elastic.co/t/logstash-is-not-accepting-logs-from-rsyslog/191293)
**Author:** [@cschelin](https://discuss.elastic.co/u/cschelin)\ **Replies:** 2\ **Last updated:** [July 22, 2019, 8:42pm UTC](https://discuss.elastic.co/t/logstash-is-not-accepting-logs-from-rsyslog/191293 "2019-07-22T20:42:55Z")
Environment: 50 CentOS 7.6 servers 192.168.1.145 - Elastic Search master server 192.168.1.146 - Elastic Search master server 192.168.1.147 - Elastic Search master server 192.168.1.148 - Elastic Search data server 192.1… --- ## [Logstash 7.2 exception when enabling TLS and X-pack (error with elastic output)](https://discuss.elastic.co/t/logstash-7-2-exception-when-enabling-tls-and-x-pack-error-with-elastic-output/191532)
**Author:** [@LogBabel](https://discuss.elastic.co/u/LogBabel)\ **Replies:** 6\ **Last updated:** [July 22, 2019, 5:17pm UTC](https://discuss.elastic.co/t/logstash-7-2-exception-when-enabling-tls-and-x-pack-error-with-elastic-output/191532 "2019-07-22T17:17:11Z")
Hello. After an upgrade to Logstash 7.2 my config using Ruby code fails to start. The Ruby code is basic inline using "ruby { code =\> '' } " Error message below. Thank you for the help. Here is the giant error, it i… --- ## [Why I need add "delete" privilege for logstash user?](https://discuss.elastic.co/t/why-i-need-add-delete-privilege-for-logstash-user/190907)
**Author:** [@azhurbilo](https://discuss.elastic.co/u/azhurbilo)\ **Replies:** 1\ **Last updated:** [July 22, 2019, 4:54pm UTC](https://discuss.elastic.co/t/why-i-need-add-delete-privilege-for-logstash-user/190907 "2019-07-22T16:54:09Z")
According https://www.elastic.co/guide/en/logstash/6.8/ls-security.html why I need add "delete" privilege for logstash user? --- ## [What is meant by the line "Non-zero metrics in the last 30s" in WinLogBeat logs](https://discuss.elastic.co/t/what-is-meant-by-the-line-non-zero-metrics-in-the-last-30s-in-winlogbeat-logs/191225)
**Author:** [@akhilsharma.in](https://discuss.elastic.co/u/akhilsharma.in)\ **Replies:** 3\ **Last updated:** [July 22, 2019, 3:16pm UTC](https://discuss.elastic.co/t/what-is-meant-by-the-line-non-zero-metrics-in-the-last-30s-in-winlogbeat-logs/191225 "2019-07-22T15:16:02Z")
I usually see the line "Non-zero metrics in the last 30s" in WinLogBeat logs and it does not load any events. This ends with the line "Failed to publish events caused by: write tcp 127.0.0.1:63448-\>127.0.0.1:5044" and th… --- ## [Using Prune Filter To Remove Null Value Fields](https://discuss.elastic.co/t/using-prune-filter-to-remove-null-value-fields/191659)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 3\ **Last updated:** [July 22, 2019, 3:14pm UTC](https://discuss.elastic.co/t/using-prune-filter-to-remove-null-value-fields/191659 "2019-07-22T15:14:18Z")
I am using the JSON filter to parse out events. However, many fields are frequently blank. I'd like to remove those fields without having to manually specify if blank then use mutate to remove field dozens of times. I… --- ## [How to map and index the specific json object into elastic search](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204)
**Author:** [@manjudr](https://discuss.elastic.co/u/manjudr)\ **Replies:** 13\ **Last updated:** [July 22, 2019, 1:11pm UTC](https://discuss.elastic.co/t/how-to-map-and-index-the-specific-json-object-into-elastic-search/191204 "2019-07-22T13:11:02Z")
Let's I'm reading data from the kafka topic and indexing specific data into specific elastic search index. Example : Logstash reading message from the kafka topic i has two messages(1. LOG, 2.ERROR) Message1. //{ "e… --- ## [Parsing dictionary into json file](https://discuss.elastic.co/t/parsing-dictionary-into-json-file/191386)
**Author:** [@saisimo02](https://discuss.elastic.co/u/saisimo02)\ **Replies:** 8\ **Last updated:** [July 22, 2019, 12:35pm UTC](https://discuss.elastic.co/t/parsing-dictionary-into-json-file/191386 "2019-07-22T12:35:08Z")
Hello, I would like to parse a json file, my file looks like that: file={"id":"\*\*\*\*\*\*\*","name":"test","alarm":{"error":2,"warning":3}} using this filter: json { source =\> "message" } I get only thes… --- ## [Can I config logstash to read config other than the path /etc/logstash/conf.d/](https://discuss.elastic.co/t/can-i-config-logstash-to-read-config-other-than-the-path-etc-logstash-conf-d/191561)
**Author:** [@mmk1995](https://discuss.elastic.co/u/mmk1995)\ **Replies:** 1\ **Last updated:** [July 22, 2019, 12:13pm UTC](https://discuss.elastic.co/t/can-i-config-logstash-to-read-config-other-than-the-path-etc-logstash-conf-d/191561 "2019-07-22T12:13:48Z")
I am trying to set the path for different config as below, pipeline.yml: - pipeline.id: main path.config: "/etc/logstash/conf.d/\*.conf" - pipeline.id: dhcp-test path.config: "/home/elk2/dhcp\_log.conf" - pipeline.id… --- ## [Reading multiple CSV file with multiple header](https://discuss.elastic.co/t/reading-multiple-csv-file-with-multiple-header/191582)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 1\ **Last updated:** [July 22, 2019, 12:04pm UTC](https://discuss.elastic.co/t/reading-multiple-csv-file-with-multiple-header/191582 "2019-07-22T12:04:57Z")
Hi I am trying to read multiple CSV file(bulk) those contains multiple headers. The no of column and their names are dynamic. While parsing the CSV files, I don't want to loose mapping between column header and the cor… --- ## [Parsing Inner JSON - Array of JSON Objects](https://discuss.elastic.co/t/parsing-inner-json-array-of-json-objects/191601)
**Author:** [@shivkumar](https://discuss.elastic.co/u/shivkumar)\ **Replies:** 2\ **Last updated:** [July 22, 2019, 11:57am UTC](https://discuss.elastic.co/t/parsing-inner-json-array-of-json-objects/191601 "2019-07-22T11:57:07Z")
The following is the content of my JSON. I am trying to parse the nested JSON. I tried to use the Split filter for the same but I am facing the error. { "School": { "Details": \[{ "Name": "a", "Class": "1" }, { … --- ## [Grok multiple date formats](https://discuss.elastic.co/t/grok-multiple-date-formats/190930)
**Author:** [@Regul](https://discuss.elastic.co/u/Regul)\ **Replies:** 2\ **Last updated:** [July 22, 2019, 10:15am UTC](https://discuss.elastic.co/t/grok-multiple-date-formats/190930 "2019-07-22T10:15:00Z")
I'm trying to parse syslog messages from 2 sources at the moment, but they have different date formats: \<12\>Jul 16 14:37:33 \<30\>2019:07:16-14:23:13 Right now I got the following: \<%{POSINT:syslog\_pri}\>%{YEAR}:%{MONT… --- ## [ID generation in sequential logs try to link MAJOR EVENT and MINOR EVENTS by an ID share using filter elasticsearch error \_elasticsearch\_lookup\_failure or ruby filter](https://discuss.elastic.co/t/id-generation-in-sequential-logs-try-to-link-major-event-and-minor-events-by-an-id-share-using-filter-elasticsearch-error-elasticsearch-lookup-failure-or-ruby-filter/190700)
**Author:** [@BlackBag98](https://discuss.elastic.co/u/BlackBag98)\ **Replies:** 2\ **Last updated:** [July 16, 2019, 11:50am UTC](https://discuss.elastic.co/t/id-generation-in-sequential-logs-try-to-link-major-event-and-minor-events-by-an-id-share-using-filter-elasticsearch-error-elasticsearch-lookup-failure-or-ruby-filter/190700 "2019-07-16T11:50:28Z")
Here is my log anonymized : \[DEBUG\] 12-04-2019 07:13:28 \[Import\] - Columns= TABLE\_DESCRIPTION(\_\_\_\_COL1\_\_\_COL2\_\_\_COL3\_\_\_COL4\_\_\_5\_\_\_6\_\_\_7\_\_\_8\_\_\_9\_\_\_10\_\_...\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_) \[ INFO\] 12-04-2019 07:13:28 \[Im… --- ## [My Logstash is not receiving any data from Filebeat, its just listening to the port](https://discuss.elastic.co/t/my-logstash-is-not-receiving-any-data-from-filebeat-its-just-listening-to-the-port/191370)
**Author:** [@irobot678](https://discuss.elastic.co/u/irobot678)\ **Replies:** 5\ **Last updated:** [July 22, 2019, 9:26am UTC](https://discuss.elastic.co/t/my-logstash-is-not-receiving-any-data-from-filebeat-its-just-listening-to-the-port/191370 "2019-07-22T09:26:28Z")
HI, I am using elk stack (elastic , logstash , kibana) version : 6.8.0 in ubuntu 14.04. when i am trying to send logs from filebeat to logstash there is no error but my logstash is just listening to the port and its n… --- ## [Processing order of \* .conf files](https://discuss.elastic.co/t/processing-order-of-conf-files/191374)
**Author:** [@vld53](https://discuss.elastic.co/u/vld53)\ **Replies:** 2\ **Last updated:** [July 22, 2019, 7:36am UTC](https://discuss.elastic.co/t/processing-order-of-conf-files/191374 "2019-07-22T07:36:11Z")
Hello everybody. Well, sorry for my English. How can i manage processing order of \*.conf files? So in other words i want to perform \*.conf files in my order. Now i cant understand how logstash handle it. For example I… --- ## [NetFow plugin](https://discuss.elastic.co/t/netfow-plugin/191587)
**Author:** [@Tariq\_Emad](https://discuss.elastic.co/u/Tariq_Emad)\ **Replies:** 0\ **Last updated:** [July 22, 2019, 6:49am UTC](https://discuss.elastic.co/t/netfow-plugin/191587 "2019-07-22T06:49:28Z")
Hi all, I looking to work on ELK ,, i already installed on windows server 2016 .. when i trying to install Netflow plugin i get error PS C:\\ELK\\logstash-7.2.0\\logstash-7.2.0\\bin\> ./logstash --modules netflow --setup … --- ## [Stored Query from Database as JDBC input to Logstash](https://discuss.elastic.co/t/stored-query-from-database-as-jdbc-input-to-logstash/191208)
**Author:** [@Sammeta\_David\_Raju](https://discuss.elastic.co/u/Sammeta_David_Raju)\ **Replies:** 2\ **Last updated:** [July 22, 2019, 5:51am UTC](https://discuss.elastic.co/t/stored-query-from-database-as-jdbc-input-to-logstash/191208 "2019-07-22T05:51:10Z")
input { jdbc { jdbc\_driver\_library =\> "mysql-connector-java-5.1.36-bin.jar" jdbc\_driver\_class =\> "com.mysql.jdbc.Driver" jdbc\_connection\_string =\> "jdbc:mysql://localhost:3306/MYDB" … --- ## [Mysql input is showing error](https://discuss.elastic.co/t/mysql-input-is-showing-error/191568)
**Author:** [@pathri](https://discuss.elastic.co/u/pathri)\ **Replies:** 1\ **Last updated:** [July 22, 2019, 5:04am UTC](https://discuss.elastic.co/t/mysql-input-is-showing-error/191568 "2019-07-22T05:04:32Z")
i want to fetch data from MySQL, below is the config file input { jdbc { jdbc\_driver\_library =\> "mysql-connector-java-5.1.36-bin.jar" jdbc\_driver\_class =\> "com.mysql.jdbc.Driver" jdbc\_connection\_string =\> "jdbc:mysq… --- ## [Logstash to collect data rom whatssap](https://discuss.elastic.co/t/logstash-to-collect-data-rom-whatssap/191552)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 0\ **Last updated:** [July 22, 2019, 12:44am UTC](https://discuss.elastic.co/t/logstash-to-collect-data-rom-whatssap/191552 "2019-07-22T00:44:55Z")
has someone already configured logstash to collect data from whatssap, can you give some help or tutorial thank you --- ## [Whtasapp group analysis using elk,](https://discuss.elastic.co/t/whtasapp-group-analysis-using-elk/191551)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 0\ **Last updated:** [July 22, 2019, 12:40am UTC](https://discuss.elastic.co/t/whtasapp-group-analysis-using-elk/191551 "2019-07-22T00:40:25Z")
yesterday I found an innovative thing, it is a tutorial about whtasapp group analysis using elk, and i want to know if someone have tried that and if it is possible to do that with detailled configuration ( i have alrea… --- ## [Distinguish json pattern and send it to specific index](https://discuss.elastic.co/t/distinguish-json-pattern-and-send-it-to-specific-index/191542)
**Author:** [@hjsjar](https://discuss.elastic.co/u/hjsjar)\ **Replies:** 1\ **Last updated:** [July 21, 2019, 10:40pm UTC](https://discuss.elastic.co/t/distinguish-json-pattern-and-send-it-to-specific-index/191542 "2019-07-21T22:40:25Z")
hi everyone I have a logstash container querying data from a mysql table called events. The mysql table contains a json field that can contain different data structures, i need to identify when an event contains certain… --- ## [Logstash - how to overwrite document instead of creating new ones](https://discuss.elastic.co/t/logstash-how-to-overwrite-document-instead-of-creating-new-ones/191456)
**Author:** [@mickey\_lubs](https://discuss.elastic.co/u/mickey_lubs)\ **Replies:** 5\ **Last updated:** [July 21, 2019, 10:39pm UTC](https://discuss.elastic.co/t/logstash-how-to-overwrite-document-instead-of-creating-new-ones/191456 "2019-07-21T22:39:50Z")
Hi Guys, I need help. I have got db2 logstash pipeline which runs a query at intervals. The problem is everytime it runs the query, it creates a new document with same data which fills up my index with duplicate data. H… --- ## [Conf configuration](https://discuss.elastic.co/t/conf-configuration/190066)
**Author:** [@Krishna\_Pagar](https://discuss.elastic.co/u/Krishna_Pagar)\ **Replies:** 5\ **Last updated:** [July 21, 2019, 12:05pm UTC](https://discuss.elastic.co/t/conf-configuration/190066 "2019-07-21T12:05:44Z")
Hi, I have created conf file with below details, but whenever I am searching in Kibana with type - auth\_inbound\_web I can see all logs with only 7 days. I need that for 30 days. There are 2 Index namely - logstash-tes… --- ## [Is it possible to collect data from website instead of logs](https://discuss.elastic.co/t/is-it-possible-to-collect-data-from-website-instead-of-logs/191501)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 0\ **Last updated:** [July 21, 2019, 3:28am UTC](https://discuss.elastic.co/t/is-it-possible-to-collect-data-from-website-instead-of-logs/191501 "2019-07-21T03:28:20Z")
hi, is it possible to collect data from website instead of logs? i have a web site for news publicly hosted and i want to know all the published event using logstash and visulize them in kibana, is there any example of c… --- ## [Debugging logstash with jruby](https://discuss.elastic.co/t/debugging-logstash-with-jruby/191493)
**Author:** [@Ram\_Sharan\_Mittal](https://discuss.elastic.co/u/Ram_Sharan_Mittal)\ **Replies:** 0\ **Last updated:** [July 20, 2019, 8:59pm UTC](https://discuss.elastic.co/t/debugging-logstash-with-jruby/191493 "2019-07-20T20:59:27Z")
Can debug JRuby code for logstash? We need to see some variable value at run time. --- ## [Logstash-how to read gzip logs stored in Azure Blob using](https://discuss.elastic.co/t/logstash-how-to-read-gzip-logs-stored-in-azure-blob-using/191480)
**Author:** [@Narendra\_Jadhav](https://discuss.elastic.co/u/Narendra_Jadhav)\ **Replies:** 0\ **Last updated:** [July 20, 2019, 12:11pm UTC](https://discuss.elastic.co/t/logstash-how-to-read-gzip-logs-stored-in-azure-blob-using/191480 "2019-07-20T12:11:43Z")
Hello Everyone, I have requirement to read the logs stored in Azure which is in \*.log.gz format. I have installed logstash-input-azureblob, logstash-codec-gzip\_lines plugins with logstash In the conf file I have belo… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=505) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=507)