# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=437 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 438 --- ## [RSS Feeds Using logstash](https://discuss.elastic.co/t/rss-feeds-using-logstash/210291)
**Author:** [@raihan](https://discuss.elastic.co/u/raihan)\ **Replies:** 0\ **Last updated:** [December 3, 2019, 5:40am UTC](https://discuss.elastic.co/t/rss-feeds-using-logstash/210291 "2019-12-03T05:40:43Z")
I am fetching RSS Feeds using logstash-input-rss plugin. I am mainly facing three problems as below. It will be great if someone can give me insgihts. How to avoid repetition of items? i.e. same item is indexed after … --- ## [Logstash Keystore creation in Windows 10 not working](https://discuss.elastic.co/t/logstash-keystore-creation-in-windows-10-not-working/210283)
**Author:** [@joshMG](https://discuss.elastic.co/u/joshMG)\ **Replies:** 0\ **Last updated:** [December 3, 2019, 4:57am UTC](https://discuss.elastic.co/t/logstash-keystore-creation-in-windows-10-not-working/210283 "2019-12-03T04:57:33Z")
Is there any additional configuration needed to create a keystore for logstash in Windows? I installed 7.4.2 via the zip file in Windows 10 but when I try to run the logstash create keystore command it just hangs there. … --- ## [Query about logstash](https://discuss.elastic.co/t/query-about-logstash/210277)
**Author:** [@kasim123](https://discuss.elastic.co/u/kasim123)\ **Replies:** 0\ **Last updated:** [December 3, 2019, 4:21am UTC](https://discuss.elastic.co/t/query-about-logstash/210277 "2019-12-03T04:21:40Z")
Hi Team, We have EFK stack running in out k8s cluster. All components are part of cluster. Now we want to implement logstash to create parsing rules for logs. How do I install it in current environment? What should I u… --- ## [Logstash Email - Constantly sending](https://discuss.elastic.co/t/logstash-email-constantly-sending/210262)
**Author:** [@lquin1978](https://discuss.elastic.co/u/lquin1978)\ **Replies:** 0\ **Last updated:** [December 3, 2019, 12:23am UTC](https://discuss.elastic.co/t/logstash-email-constantly-sending/210262 "2019-12-03T00:23:10Z")
I am using an email output within logstash to send me a copy of an xml file that has been generated by an NMAP scan. The issue is Logstash is repeatedly sending me a copy of the same file every minute. Is there a way t… --- ## [Drop logs that does not contains type syslog](https://discuss.elastic.co/t/drop-logs-that-does-not-contains-type-syslog/207094)
**Author:** [@Jiri\_Safar](https://discuss.elastic.co/u/Jiri_Safar)\ **Replies:** 2\ **Last updated:** [December 2, 2019, 9:44pm UTC](https://discuss.elastic.co/t/drop-logs-that-does-not-contains-type-syslog/207094 "2019-12-02T21:44:37Z")
Hi, I have a multiple input and output: input is gelf and output for that is log that in to elastic. input is a syslog and output is in to elastic. But what hapens is that some gelf are send in to my syslog out… --- ## [Mapping of WKT POINT to GEOPOINT](https://discuss.elastic.co/t/mapping-of-wkt-point-to-geopoint/210246)
**Author:** [@silvermax](https://discuss.elastic.co/u/silvermax)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 9:13pm UTC](https://discuss.elastic.co/t/mapping-of-wkt-point-to-geopoint/210246 "2019-12-02T21:13:57Z")
Dear All, I saw that WKT POINT is one of the primitive described in the GEO-POINT datatype documentation. I'm fetching this data from SQL DB using the JDBL connector. I saw a lot of thread but still not clear how to ma… --- ## [Filebeat - logstash new indexs are created without template](https://discuss.elastic.co/t/filebeat-logstash-new-indexs-are-created-without-template/210241)
**Author:** [@Sjap](https://discuss.elastic.co/u/Sjap)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 8:35pm UTC](https://discuss.elastic.co/t/filebeat-logstash-new-indexs-are-created-without-template/210241 "2019-12-02T20:35:29Z")
Hi Everybody, I'm trying to use Logstash for all my filebeat nodes with ILM enabled and with ingest-pipelines. New indexes are created as 'filebeat-7.4.2' but where is no ILM enabled. - It looks like the indexes not ar… --- ## [Need help - logstash pipeline isn't working anymore ...?](https://discuss.elastic.co/t/need-help-logstash-pipeline-isnt-working-anymore/210006)
**Author:** [@Cdnvballer](https://discuss.elastic.co/u/Cdnvballer)\ **Replies:** 3\ **Last updated:** [December 2, 2019, 7:08pm UTC](https://discuss.elastic.co/t/need-help-logstash-pipeline-isnt-working-anymore/210006 "2019-12-02T19:08:21Z")
Kibana -- Discover console is showing events coming in, but the indexing is not working right. used to be able to see the fields \[system\]\[syslog\]\[hostname\] and related but now they're gone?? ==02-beats-input.conf == i… --- ## [SNMPTRAP input behind F5 load balancer - Service Monitor generating lots of errors in log](https://discuss.elastic.co/t/snmptrap-input-behind-f5-load-balancer-service-monitor-generating-lots-of-errors-in-log/210207)
**Author:** [@marc1](https://discuss.elastic.co/u/marc1)\ **Replies:** 3\ **Last updated:** [December 2, 2019, 4:57pm UTC](https://discuss.elastic.co/t/snmptrap-input-behind-f5-load-balancer-service-monitor-generating-lots-of-errors-in-log/210207 "2019-12-02T16:57:48Z")
Hello, We have a cluster of logstash instances behind an F5 load balancer. Part of the F5 implementation includes a health check that is sent every couple of seconds to make sure the UDP port on the logstash is still up… --- ## [Logstash failing for csv load with double quotes](https://discuss.elastic.co/t/logstash-failing-for-csv-load-with-double-quotes/210213)
**Author:** [@krishna\_bhargav](https://discuss.elastic.co/u/krishna_bhargav)\ **Replies:** 1\ **Last updated:** [December 2, 2019, 4:49pm UTC](https://discuss.elastic.co/t/logstash-failing-for-csv-load-with-double-quotes/210213 "2019-12-02T16:49:06Z")
Am using csv filter pluggin with separator =\> '|', it works if we remove the quotes for "QA". Ay work around to make this work or is this limitation with CSV filter. Here is the sample load data: QA8\_COMMON|1044|AC|101… --- ## [Logstash 7.4.0 support Excel sheet filter plugin?](https://discuss.elastic.co/t/logstash-7-4-0-support-excel-sheet-filter-plugin/210209)
**Author:** [@sanurani](https://discuss.elastic.co/u/sanurani)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 3:35pm UTC](https://discuss.elastic.co/t/logstash-7-4-0-support-excel-sheet-filter-plugin/210209 "2019-12-02T15:35:09Z")
Am looking for a input filter plugin to read data from excel sheet and am working on Logstash 7.4.0 version. I see the online resources for CSV filter plugin but not for excel sheet. Appreciate any help, thank you:) … --- ## [Avoid print of "original data now in message field" when "JSON parse error"](https://discuss.elastic.co/t/avoid-print-of-original-data-now-in-message-field-when-json-parse-error/210146)
**Author:** [@gghdr](https://discuss.elastic.co/u/gghdr)\ **Replies:** 1\ **Last updated:** [December 2, 2019, 3:31pm UTC](https://discuss.elastic.co/t/avoid-print-of-original-data-now-in-message-field-when-json-parse-error/210146 "2019-12-02T15:31:00Z")
Hi all, as the title suggests, i want to get the "JSON parse error" as usual, but without the print of the message (that in my situation can be really big, MBs) on the log, to avoid the saturation of the log itself. Fr… --- ## [Time difference in log times](https://discuss.elastic.co/t/time-difference-in-log-times/210179)
**Author:** [@rj23495](https://discuss.elastic.co/u/rj23495)\ **Replies:** 1\ **Last updated:** [December 2, 2019, 3:21pm UTC](https://discuss.elastic.co/t/time-difference-in-log-times/210179 "2019-12-02T15:21:17Z")
Hi, i have added a new time field for my log parser and there is a time difference of 2:30 in time that is recorded in ES and the log time. PFA. grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:time} \\\[%{DATA:pool}… --- ## [Different field types from same grok statement](https://discuss.elastic.co/t/different-field-types-from-same-grok-statement/209966)
**Author:** [@probson](https://discuss.elastic.co/u/probson)\ **Replies:** 1\ **Last updated:** [December 2, 2019, 3:11pm UTC](https://discuss.elastic.co/t/different-field-types-from-same-grok-statement/209966 "2019-12-02T15:11:05Z")
Hi, I use the below grok to convert the source.address to source.ip and convert from string to IP as part of the ECS. I have firewall and IPS logs running through this. Prior to the grok i convert source.address to st… --- ## [How can i Update-by query and index events to elastic in the same pipeline?](https://discuss.elastic.co/t/how-can-i-update-by-query-and-index-events-to-elastic-in-the-same-pipeline/210200)
**Author:** [@evalufran](https://discuss.elastic.co/u/evalufran)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 2:32pm UTC](https://discuss.elastic.co/t/how-can-i-update-by-query-and-index-events-to-elastic-in-the-same-pipeline/210200 "2019-12-02T14:32:54Z")
Hi! I have an index which represents a transaction of states, so i might have more than one doc with different states and with the same id. I thought that the best way to retrive the last state of an id was to add a new … --- ## [Nested Data](https://discuss.elastic.co/t/nested-data/210192)
**Author:** [@MSef](https://discuss.elastic.co/u/MSef)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 1:42pm UTC](https://discuss.elastic.co/t/nested-data/210192 "2019-12-02T13:42:53Z")
Hello there, I have a CSV file and I am trying to filter the data using logstash. What i am having trouble with is that I have a column with data inside of it like this: Column name: endPlaces Data inside of column e… --- ## [Logstash 7.2 takes about atleast 5 mins to update ES 7.2](https://discuss.elastic.co/t/logstash-7-2-takes-about-atleast-5-mins-to-update-es-7-2/208812)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 9\ **Last updated:** [December 2, 2019, 11:16am UTC](https://discuss.elastic.co/t/logstash-7-2-takes-about-atleast-5-mins-to-update-es-7-2/208812 "2019-12-02T11:16:49Z")
Hi, I have implemented a ELK setup of 7.2 version. Below is my logstash conf input { file { path =\> "/etc/logstash/conf.d/mytest.txt" start\_position =\> "beginning" sincedb\_path =\> "/d… --- ## [Encrypt / secure Database password in logstash.conf file](https://discuss.elastic.co/t/encrypt-secure-database-password-in-logstash-conf-file/210137)
**Author:** [@sampath1](https://discuss.elastic.co/u/sampath1)\ **Replies:** 5\ **Last updated:** [December 2, 2019, 11:02am UTC](https://discuss.elastic.co/t/encrypt-secure-database-password-in-logstash-conf-file/210137 "2019-12-02T11:02:39Z")
Hi, I am new to this Logstash. I don't want to specify plain text "username" and "password" in logstash.conf for the Database input plugin. Is there any way to do that? Is there any possible way to encrypt or hide the… --- ## [JDBC\_password config for logstash?](https://discuss.elastic.co/t/jdbc-password-config-for-logstash/210153)
**Author:** [@sampath1](https://discuss.elastic.co/u/sampath1)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 10:19am UTC](https://discuss.elastic.co/t/jdbc-password-config-for-logstash/210153 "2019-12-02T10:19:15Z")
Continuing the discussion from Jdbc\_password configuration for logstash: Any update on this? --- ## [Each time logstash instance dies, the number of data stored in elasticsearch increases](https://discuss.elastic.co/t/each-time-logstash-instance-dies-the-number-of-data-stored-in-elasticsearch-increases/209320)
**Author:** [@securecode99](https://discuss.elastic.co/u/securecode99)\ **Replies:** 2\ **Last updated:** [December 2, 2019, 9:38am UTC](https://discuss.elastic.co/t/each-time-logstash-instance-dies-the-number-of-data-stored-in-elasticsearch-increases/209320 "2019-12-02T09:38:34Z")
I'm using ELK(Elasticsearch, Logstash, Kibana, Beat) with a Kafka in front of Logstash. However, I noticed something unusual during the failover test of Logstash. The test was carried out by suddenly killing the one lo… --- ## [Logstash - Elastic Search Input plugin](https://discuss.elastic.co/t/logstash-elastic-search-input-plugin/210145)
**Author:** [@rsumit](https://discuss.elastic.co/u/rsumit)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 9:30am UTC](https://discuss.elastic.co/t/logstash-elastic-search-input-plugin/210145 "2019-12-02T09:30:55Z")
Hi Team I am creating a query using elastic search input plugin. when I ran it on developer tool on kibana before running on LS I am getting the following error: POST /sbl\_nmon2019.12.02/\_search?size=0 {"query": {"… --- ## [Error when trying to use snmp trap with Logstash](https://discuss.elastic.co/t/error-when-trying-to-use-snmp-trap-with-logstash/210141)
**Author:** [@peterthief](https://discuss.elastic.co/u/peterthief)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 9:08am UTC](https://discuss.elastic.co/t/error-when-trying-to-use-snmp-trap-with-logstash/210141 "2019-12-02T09:08:55Z")
Hi all, I try to use this config below to start logstash: input { snmp { hosts =\> \[{host =\> "udp:192.27.18.351/5000" version =\> "3"}\] security\_name =\> "admin" auth\_protocol =\> "sha2" auth\_pass =\> "Loc… --- ## [Grok vs real life - kibana](https://discuss.elastic.co/t/grok-vs-real-life-kibana/210136)
**Author:** [@Marta\_Zagrajek](https://discuss.elastic.co/u/Marta_Zagrajek)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 8:26am UTC](https://discuss.elastic.co/t/grok-vs-real-life-kibana/210136 "2019-12-02T08:26:39Z")
I finally made good grok pattern with my log and in grok debugger all was fine. When I implemented it in my stack, I do not have proper fields, my "exception" field is empty or one char. Log sample: 21:35:13.242 p… --- ## [How can I Filter Error logs](https://discuss.elastic.co/t/how-can-i-filter-error-logs/210130)
**Author:** [@kishara](https://discuss.elastic.co/u/kishara)\ **Replies:** 0\ **Last updated:** [December 2, 2019, 7:48am UTC](https://discuss.elastic.co/t/how-can-i-filter-error-logs/210130 "2019-12-02T07:48:27Z")
How can I filter Error logs through the other logs input { beats { port =\> 5044 } } filter{ grok{ #I have no idea about GROK filter } } output { elasticsearch { hosts =\> "localhost:9200" manage\_template =\> … --- ## [RDS to Elasticsearch - Setting up Logstash in a highly available/redundant architecture](https://discuss.elastic.co/t/rds-to-elasticsearch-setting-up-logstash-in-a-highly-available-redundant-architecture/210070)
**Author:** [@dshenoy](https://discuss.elastic.co/u/dshenoy)\ **Replies:** 0\ **Last updated:** [December 1, 2019, 4:12pm UTC](https://discuss.elastic.co/t/rds-to-elasticsearch-setting-up-logstash-in-a-highly-available-redundant-architecture/210070 "2019-12-01T16:12:41Z")
I am looking into a highly available solution to feed in AWS RDS transactions into my Elasticsearch repository. There are some articles on this topic. For example: But this is not a flexible and agile in the long-t… --- ## [Generating geo\_point based on a couple float](https://discuss.elastic.co/t/generating-geo-point-based-on-a-couple-float/210045)
**Author:** [@Claudio\_Grasso](https://discuss.elastic.co/u/Claudio_Grasso)\ **Replies:** 0\ **Last updated:** [December 1, 2019, 4:27am UTC](https://discuss.elastic.co/t/generating-geo-point-based-on-a-couple-float/210045 "2019-12-01T04:27:17Z")
Hi guys, I need to import a MySQL table into ES where the table has one longitud and one longitud rows. It is pretty much the same use case as the one described in: so I followed the same steps and I obtained a field… --- ## [Logstash Aggregate - pushing same data multiple times in nested array json](https://discuss.elastic.co/t/logstash-aggregate-pushing-same-data-multiple-times-in-nested-array-json/210029)
**Author:** [@sunnysetia](https://discuss.elastic.co/u/sunnysetia)\ **Replies:** 0\ **Last updated:** [November 30, 2019, 12:56pm UTC](https://discuss.elastic.co/t/logstash-aggregate-pushing-same-data-multiple-times-in-nested-array-json/210029 "2019-11-30T12:56:46Z")
Hi I have been trying to index data from Postgres to Elasticsearch using logstash. Scenario: There is a parent medicine which can have multiple child/variant medicines. I need to index data for each parent medicine w… --- ## [Is it mandatory to use web scrapper or just logstash to get data from public site](https://discuss.elastic.co/t/is-it-mandatory-to-use-web-scrapper-or-just-logstash-to-get-data-from-public-site/210019)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 2\ **Last updated:** [November 30, 2019, 12:31pm UTC](https://discuss.elastic.co/t/is-it-mandatory-to-use-web-scrapper-or-just-logstash-to-get-data-from-public-site/210019 "2019-11-30T12:31:13Z")
a question asked by many people, in fact is it possible to ingest data from an internet sales website, is it mandatory to use web scrapper or just logstash can be configured to retrieve articles posted on a site in real … --- ## [Input plugin development : How to run ruby file standalone before generating gem](https://discuss.elastic.co/t/input-plugin-development-how-to-run-ruby-file-standalone-before-generating-gem/209991)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 1\ **Last updated:** [November 30, 2019, 11:33am UTC](https://discuss.elastic.co/t/input-plugin-development-how-to-run-ruby-file-standalone-before-generating-gem/209991 "2019-11-30T11:33:20Z")
Hi All i am developing a input plugin, i am following the below steps Generated the full file tree structure from online input plugin generation as mentioned here Installed rbenv Installed jruby gem install bundler cd… --- ## [It is really that http\_poller allow to get data from instagram using api?](https://discuss.elastic.co/t/it-is-really-that-http-poller-allow-to-get-data-from-instagram-using-api/210022)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 0\ **Last updated:** [November 30, 2019, 8:30am UTC](https://discuss.elastic.co/t/it-is-really-that-http-poller-allow-to-get-data-from-instagram-using-api/210022 "2019-11-30T08:30:15Z")
i found a post showing that we can use http\_poller with logstash and instagram apito read data from instagram this the stackoverflow (the second post in the forum) thank you [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=436) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=438)