# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=857 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 858 --- ## [Logstash Index](https://discuss.elastic.co/t/logstash-index/111474)
**Author:** [@ChrisChua](https://discuss.elastic.co/u/ChrisChua)\ **Replies:** 3\ **Last updated:** [December 13, 2017, 7:15pm UTC](https://discuss.elastic.co/t/logstash-index/111474 "2017-12-13T19:15:22Z")
Hi guys, I have a problem with the logstash output, it work fine when I didn't put index, but when I put my custom index name then it won't store in elasticsearch. elasticsearch { hosts =\> \["localhost:9200"\] … --- ## [Logstash XML Parser Fail - File Does Not Exist?](https://discuss.elastic.co/t/logstash-xml-parser-fail-file-does-not-exist/111386)
**Author:** [@ben-10](https://discuss.elastic.co/u/ben-10)\ **Replies:** 1\ **Last updated:** [December 13, 2017, 4:39pm UTC](https://discuss.elastic.co/t/logstash-xml-parser-fail-file-does-not-exist/111386 "2017-12-13T16:39:34Z")
After upgrading logstash to 5.6.5 I am know getting the following error en masse and don't understand what the error is trying to tell me. Snip of logstash config for xml section: xml { sour… --- ## [Logstash failing with "Encountered a retryable error."](https://discuss.elastic.co/t/logstash-failing-with-encountered-a-retryable-error/110879)
**Author:** [@gudladona](https://discuss.elastic.co/u/gudladona)\ **Replies:** 2\ **Last updated:** [December 13, 2017, 4:39pm UTC](https://discuss.elastic.co/t/logstash-failing-with-encountered-a-retryable-error/110879 "2017-12-13T16:39:15Z")
Environment: Logstash 5.6.3 Elastic 5.5 We have been seeeing lot of error messages in logstash logs lately. Not sure what the issue is. Please provide some pointers. Dec 8 11:59:13 ls01 journal: {"level":"ERROR","l… --- ## [Logstash high cpu usage when idle](https://discuss.elastic.co/t/logstash-high-cpu-usage-when-idle/111600)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 1\ **Last updated:** [December 13, 2017, 4:31pm UTC](https://discuss.elastic.co/t/logstash-high-cpu-usage-when-idle/111600 "2017-12-13T16:31:24Z")
I'm running ELK stack: elk & beats: # rpm -q elasticsearch logstash kibana elasticsearch-6.0.1-1.noarch logstash-6.0.1-1.noarch kibana-6.0.1-1.x86\_64 # rpm -q filebeat metricbeat filebeat-6.0.1-1.x86\_64 metricbeat-6.0.… --- ## [Problem loading CSV file using logstash](https://discuss.elastic.co/t/problem-loading-csv-file-using-logstash/111455)
**Author:** [@aperez](https://discuss.elastic.co/u/aperez)\ **Replies:** 4\ **Last updated:** [December 13, 2017, 4:16pm UTC](https://discuss.elastic.co/t/problem-loading-csv-file-using-logstash/111455 "2017-12-13T16:16:58Z")
I am new into Logstash/elasticsearch/kibana I have created the following logstash config in windows input { file { path =\> "C:\\Users\\aperez.SEAPUB\\Downloads\\logstash-6.0.1\\data\\avrs\_syslog" start\_position =\> "beginn… --- ## [JMX input Plugin for Logstash and jmxmp](https://discuss.elastic.co/t/jmx-input-plugin-for-logstash-and-jmxmp/111601)
**Author:** [@Andy\_Latham](https://discuss.elastic.co/u/Andy_Latham)\ **Replies:** 0\ **Last updated:** [December 13, 2017, 4:10pm UTC](https://discuss.elastic.co/t/jmx-input-plugin-for-logstash-and-jmxmp/111601 "2017-12-13T16:10:22Z")
Hi All, Does anyone know if the JMX input Plugin for Logstash supports jmxmp, I've hunted the docs and online and can't see any specific mention so thought I'd post here. Cheers -Andy --- ## [DLQ Support for Non-elasticsearch plugins and keeping up with Logstash as a plugin developer](https://discuss.elastic.co/t/dlq-support-for-non-elasticsearch-plugins-and-keeping-up-with-logstash-as-a-plugin-developer/111599)
**Author:** [@the\_angry\_angel](https://discuss.elastic.co/u/the_angry_angel)\ **Replies:** 0\ **Last updated:** [December 13, 2017, 4:07pm UTC](https://discuss.elastic.co/t/dlq-support-for-non-elasticsearch-plugins-and-keeping-up-with-logstash-as-a-plugin-developer/111599 "2017-12-13T16:07:18Z")
2 questions. I didn't feel like these were particularly github issues, but I suspect the forum isn't necessarily the right place to ask either. DLQ support for output plugins other than logstash-output-elasticsearch; … --- ## [Using sql server queries in elasticsearch 5.6.3](https://discuss.elastic.co/t/using-sql-server-queries-in-elasticsearch-5-6-3/104027)
**Author:** [@parsa](https://discuss.elastic.co/u/parsa)\ **Replies:** 43\ **Last updated:** [December 13, 2017, 3:03pm UTC](https://discuss.elastic.co/t/using-sql-server-queries-in-elasticsearch-5-6-3/104027 "2017-12-13T15:03:23Z")
How can I use sql server queries in elastic search 5.6.3? I don't know anything about this process. Just I could install elastic search on windows successfully. please help for more details. --- ## [Filter data after a slash](https://discuss.elastic.co/t/filter-data-after-a-slash/111163)
**Author:** [@scch](https://discuss.elastic.co/u/scch)\ **Replies:** 2\ **Last updated:** [December 13, 2017, 2:56pm UTC](https://discuss.elastic.co/t/filter-data-after-a-slash/111163 "2017-12-13T14:56:59Z")
hi, I am trying to filter out the value which is after slash through Logstash from "DMZ\\L8978899" ERROR: No output in ruby codec ERROR in logs : Cannot create pipeline {:reason=\>"Expected one of #, { at line 98, colum… --- ## [Error creating: pods "logstash-" is forbidden: service account was not found, retry after the service account is created](https://discuss.elastic.co/t/error-creating-pods-logstash-is-forbidden-service-account-was-not-found-retry-after-the-service-account-is-created/111573)
**Author:** [@marouane](https://discuss.elastic.co/u/marouane)\ **Replies:** 1\ **Last updated:** [December 13, 2017, 2:10pm UTC](https://discuss.elastic.co/t/error-creating-pods-logstash-is-forbidden-service-account-was-not-found-retry-after-the-service-account-is-created/111573 "2017-12-13T14:10:24Z")
Hello evey one, i'm working on logstash deployment using kubernetes: This is the error that i have : Error creating: pods "logstash-" is forbidden: service account was not found, retry after the service account is crea… --- ## [Logstash XPack Password](https://discuss.elastic.co/t/logstash-xpack-password/111149)
**Author:** [@MultiplierMultiplier](https://discuss.elastic.co/u/MultiplierMultiplier)\ **Replies:** 4\ **Last updated:** [December 13, 2017, 2:03pm UTC](https://discuss.elastic.co/t/logstash-xpack-password/111149 "2017-12-13T14:03:13Z")
So I have installed X Pack on my ELK stack, I am just wondering to put the below (I have set the new username and password this is just the example): xpack.monitoring.elasticsearch.username: logstash\_system xpack.monit… --- ## [Date parsing](https://discuss.elastic.co/t/date-parsing/111557)
**Author:** [@cilzzz](https://discuss.elastic.co/u/cilzzz)\ **Replies:** 2\ **Last updated:** [December 13, 2017, 1:44pm UTC](https://discuss.elastic.co/t/date-parsing/111557 "2017-12-13T13:44:55Z")
Hello, i have a logfile generated by machine and i need to parse the date using logstash in the date field the date format is for example 12/05/2017 11:49:05 and 4/Feb/2017 10:05:33 for the first format my parser wo… --- ## [\[@metadata\]\[field\] not behaving as expected](https://discuss.elastic.co/t/metadata-field-not-behaving-as-expected/111554)
**Author:** [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)\ **Replies:** 3\ **Last updated:** [December 13, 2017, 1:41pm UTC](https://discuss.elastic.co/t/metadata-field-not-behaving-as-expected/111554 "2017-12-13T13:41:05Z")
I am using \[@metadata\] fields for filtering and outputs. I have an issue with how the field is behaving when there are two @metadata fields with different names. Below is a stripped down config I am using for testing: … --- ## [How to start Logstash in Azure?](https://discuss.elastic.co/t/how-to-start-logstash-in-azure/111560)
**Author:** [@Joff](https://discuss.elastic.co/u/Joff)\ **Replies:** 0\ **Last updated:** [December 13, 2017, 1:07pm UTC](https://discuss.elastic.co/t/how-to-start-logstash-in-azure/111560 "2017-12-13T13:07:18Z")
Hi, We are trying to develop a PoC for Elastic on Azure Cloud We installed the Elastic template from Azure Marketplace, selected 3 nodes and now we have following configuration: 1 Kibana VM 3 Data VM 3 Master VM Wh… --- ## [Date filter is not overwriting to @timestamp](https://discuss.elastic.co/t/date-filter-is-not-overwriting-to-timestamp/111495)
**Author:** [@Balamurali](https://discuss.elastic.co/u/Balamurali)\ **Replies:** 1\ **Last updated:** [December 13, 2017, 12:50pm UTC](https://discuss.elastic.co/t/date-filter-is-not-overwriting-to-timestamp/111495 "2017-12-13T12:50:40Z")
Hi all, I 'm using logstash 6.0 version, unable to overwrite @timestamp using date filter. Below sample logs are 2017-12-11 01:04:56,195 INFO XXXYXYYXYXYXY 2017-12-11 01:04:56,239 ERR XYEUUSSI Configurations are c… --- ## [Logstash says that Elasticsearch Unreachable](https://discuss.elastic.co/t/logstash-says-that-elasticsearch-unreachable/111551)
**Author:** [@ggdagg](https://discuss.elastic.co/u/ggdagg)\ **Replies:** 1\ **Last updated:** [December 13, 2017, 11:36am UTC](https://discuss.elastic.co/t/logstash-says-that-elasticsearch-unreachable/111551 "2017-12-13T11:36:45Z")
Hi all, I encounter a problem with Logstash that says my Elasticsearch is unreachable. However, the below command proves that my Elasticsearch works correctly: curl localhost:9200 { "name" : "D34P3jP", "cluster\_nam… --- ## [Problem is with logstash input snmptrap plugin....yamlmibdir option](https://discuss.elastic.co/t/problem-is-with-logstash-input-snmptrap-plugin-yamlmibdir-option/111546)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 0\ **Last updated:** [December 13, 2017, 11:21am UTC](https://discuss.elastic.co/t/problem-is-with-logstash-input-snmptrap-plugin-yamlmibdir-option/111546 "2017-12-13T11:21:33Z")
Hi All.... Here i have an issue with an option yamlmibdir in input snmptrap plugin input { snmptrap { type =\> "snmptrap" community =\> "public" port =\> 162 yamlmibdir =\> "/opt/logstash-5.4.0/vendor/bundle/jruby/1.9/gem… --- ## [How do I separate the kafka topics from logstash?](https://discuss.elastic.co/t/how-do-i-separate-the-kafka-topics-from-logstash/111528)
**Author:** [@david\_k](https://discuss.elastic.co/u/david_k)\ **Replies:** 0\ **Last updated:** [December 13, 2017, 10:01am UTC](https://discuss.elastic.co/t/how-do-i-separate-the-kafka-topics-from-logstash/111528 "2017-12-13T10:01:11Z")
Hi all This question is related to kafka and logstash input, output. I try to separate the topics through the following conf file but it doesn't work. Thanks. input { kafka { bootstrap\_servers =\> "localhost:9092" … --- ## [No Compatible Fields: The "logstash-\*" index pattern does not contain any of the following field types: geo\_point](https://discuss.elastic.co/t/no-compatible-fields-the-logstash-index-pattern-does-not-contain-any-of-the-following-field-types-geo-point/110473)
**Author:** [@bianchi74](https://discuss.elastic.co/u/bianchi74)\ **Replies:** 2\ **Last updated:** [December 13, 2017, 9:31am UTC](https://discuss.elastic.co/t/no-compatible-fields-the-logstash-index-pattern-does-not-contain-any-of-the-following-field-types-geo-point/110473 "2017-12-13T09:31:52Z")
Hi all, when I try to create a "Coordinate Map" I receive the following error message: No Compatible Fields: The "logstash-\*" index pattern does not contain any of the following field types: geo\_point My coordinates… --- ## [Set index depending on hostname/custom field](https://discuss.elastic.co/t/set-index-depending-on-hostname-custom-field/110591)
**Author:** [@abd.wsu](https://discuss.elastic.co/u/abd.wsu)\ **Replies:** 2\ **Last updated:** [December 13, 2017, 9:16am UTC](https://discuss.elastic.co/t/set-index-depending-on-hostname-custom-field/110591 "2017-12-13T09:16:35Z")
Hello, I am setting a custom field in filebeat for environment depending upon the server. I am then trying to use that field to create a seperate index depending on the field value. But am getting an error every time. … --- ## [Logstash not reading .conf-file when running as a service](https://discuss.elastic.co/t/logstash-not-reading-conf-file-when-running-as-a-service/110636)
**Author:** [@rvarttinen](https://discuss.elastic.co/u/rvarttinen)\ **Replies:** 1\ **Last updated:** [December 13, 2017, 9:13am UTC](https://discuss.elastic.co/t/logstash-not-reading-conf-file-when-running-as-a-service/110636 "2017-12-13T09:13:36Z")
My apologies if this issue already has been answerd, although I've done some searching and haven't found any other thread with the problem I am experiencing. Similar ones, but not really what I'm looking for ... So, her… --- ## [Logstash - how to get yesterday's date](https://discuss.elastic.co/t/logstash-how-to-get-yesterdays-date/109054)
**Author:** [@Pawel\_Piorkowski](https://discuss.elastic.co/u/Pawel_Piorkowski)\ **Replies:** 4\ **Last updated:** [December 13, 2017, 8:56am UTC](https://discuss.elastic.co/t/logstash-how-to-get-yesterdays-date/109054 "2017-12-13T08:56:23Z")
Hello, I have ELK stack installed and would like to store csv data there. The only problem I am facing is that my data doesn't contain date information (just time). Example message looks something like: aaa;bbb;ccc;23:… --- ## [Elapsed filter: where is "elapsed\_time" stored?](https://discuss.elastic.co/t/elapsed-filter-where-is-elapsed-time-stored/111497)
**Author:** [@Ganesh\_Venkataraman](https://discuss.elastic.co/u/Ganesh_Venkataraman)\ **Replies:** 1\ **Last updated:** [December 13, 2017, 7:23am UTC](https://discuss.elastic.co/t/elapsed-filter-where-is-elapsed-time-stored/111497 "2017-12-13T07:23:21Z")
I have written elapsed filter and putting logstash in debug I am able to see it calculates correctly. How do i store it to a new field or does it get automatically stored somewhere? add\_field line in my code does not wo… --- ## [Run "---setup" netflow module when LS is a service (was Install netflow module error)](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 20\ **Last updated:** [December 13, 2017, 7:06am UTC](https://discuss.elastic.co/t/run-setup-netflow-module-when-ls-is-a-service-was-install-netflow-module-error/109166 "2017-12-13T07:06:13Z")
ENV: Cluster 6.0, LS 6.0, x-pack trial, Cent0s 7 When i run install module netflow i got error /usr/share/logstash/bin/logstash --path.settings=/etc/logstash --modules netflow --setup -M "netflow.var.kibana.host=http… --- ## [Parsing json file](https://discuss.elastic.co/t/parsing-json-file/111224)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 2\ **Last updated:** [December 13, 2017, 3:17am UTC](https://discuss.elastic.co/t/parsing-json-file/111224 "2017-12-13T03:17:44Z")
Hi All, I am trying to parse the .json file its perfectly getting parsed when I do stdin i.e. per entry but .json being a file with comma separated entries how do I parse using file input parameter? here is my file and… --- ## [Mapping one field from csv file to two fields in elasticsearch template file](https://discuss.elastic.co/t/mapping-one-field-from-csv-file-to-two-fields-in-elasticsearch-template-file/111462)
**Author:** [@Arjun\_subramanyam\_va](https://discuss.elastic.co/u/Arjun_subramanyam_va)\ **Replies:** 1\ **Last updated:** [December 13, 2017, 1:26am UTC](https://discuss.elastic.co/t/mapping-one-field-from-csv-file-to-two-fields-in-elasticsearch-template-file/111462 "2017-12-13T01:26:15Z")
I have a title column in my csv file in the elastic search schema we defined two entities namely student and teacher both share this title field like student.title and teacher.title how can i specify this in logstash --- ## [Good query doesn't work in logstash elastic input?](https://discuss.elastic.co/t/good-query-doesnt-work-in-logstash-elastic-input/108490)
**Author:** [@ra.girard](https://discuss.elastic.co/u/ra.girard)\ **Replies:** 4\ **Last updated:** [December 12, 2017, 11:51pm UTC](https://discuss.elastic.co/t/good-query-doesnt-work-in-logstash-elastic-input/108490 "2017-12-12T23:51:28Z")
This query run perfectly directly to ElasticSearch but doesn't work in logstash. Someone could explain to me why ? Thanks My query GET /my.logs/\_search?filter\_path=hits.total,hits.hits.\_source&size=9999 { "query" :… --- ## [Timestamp timezon date/time issue](https://discuss.elastic.co/t/timestamp-timezon-date-time-issue/111449)
**Author:** [@mcguacon](https://discuss.elastic.co/u/mcguacon)\ **Replies:** 1\ **Last updated:** [December 12, 2017, 11:43pm UTC](https://discuss.elastic.co/t/timestamp-timezon-date-time-issue/111449 "2017-12-12T23:43:10Z")
I am having an issue of date/times being skewed by 6 hours. Within my config file I have had it set to "UTC", "US/Central", and left it off completely to try and remedy my problem. My logstash input is a json file, whe… --- ## [Elasticsearch filter - fields array](https://discuss.elastic.co/t/elasticsearch-filter-fields-array/111435)
**Author:** [@chetan\_vartak](https://discuss.elastic.co/u/chetan_vartak)\ **Replies:** 1\ **Last updated:** [December 12, 2017, 11:09pm UTC](https://discuss.elastic.co/t/elasticsearch-filter-fields-array/111435 "2017-12-12T23:09:31Z")
Hi, I've this script... It works fine with one field in elasticsearch filter. But starts giving error with multiple fields. How to pass fields array elasticsearch { hosts =\> \["localhost:3602"\] sort =\> "eid:desc" ind… --- ## [Pipeline not working with logstash-output-elasticsearch plugin when using doc\_as\_upsert =\> true](https://discuss.elastic.co/t/pipeline-not-working-with-logstash-output-elasticsearch-plugin-when-using-doc-as-upsert-true/111430)
**Author:** [@Dineshs91](https://discuss.elastic.co/u/Dineshs91)\ **Replies:** 0\ **Last updated:** [December 12, 2017, 8:13pm UTC](https://discuss.elastic.co/t/pipeline-not-working-with-logstash-output-elasticsearch-plugin-when-using-doc-as-upsert-true/111430 "2017-12-12T20:13:18Z")
Pipeline has no effect while using action =\> update and doc\_as\_upsert =\> true. When I change the action to index, it works. Here is my logstash configuration (output section) output { elasticsearch { action =\> … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=856) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=858)