# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=555 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 556 --- ## [Regarding Logstash Version](https://discuss.elastic.co/t/regarding-logstash-version/179292)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [May 2, 2019, 7:07am UTC](https://discuss.elastic.co/t/regarding-logstash-version/179292 "2019-05-02T07:07:26Z")
Hi, Is there a way that I can infer the version of logstash at the filter level? Regards, D --- ## [Unable to use if condition in logstash output section](https://discuss.elastic.co/t/unable-to-use-if-condition-in-logstash-output-section/179289)
**Author:** [@Nithani25](https://discuss.elastic.co/u/Nithani25)\ **Replies:** 2\ **Last updated:** [May 2, 2019, 7:05am UTC](https://discuss.elastic.co/t/unable-to-use-if-condition-in-logstash-output-section/179289 "2019-05-02T07:05:39Z")
Hi Team, I am using file to inject two types of logs available on the same server via tcp port communication, since the log source is different i want two different indices to be created from logstash, which i am not ab… --- ## [Logstash ES Output Plugin](https://discuss.elastic.co/t/logstash-es-output-plugin/179285)
**Author:** [@Swapnil\_Bagadia](https://discuss.elastic.co/u/Swapnil_Bagadia)\ **Replies:** 0\ **Last updated:** [May 2, 2019, 6:15am UTC](https://discuss.elastic.co/t/logstash-es-output-plugin/179285 "2019-05-02T06:15:47Z")
Should not the default elasticsearch-document\_type for the current logstash ES Output Plugin be \_doc instead of doc ? https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-e… --- ## [\[Resolved\]Logstash - data is not aggregrated in chronological order of a httpsession and output file looks like in Json format](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835)
**Author:** [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Replies:** 6\ **Last updated:** [May 2, 2019, 6:02am UTC](https://discuss.elastic.co/t/resolved-logstash-data-is-not-aggregrated-in-chronological-order-of-a-httpsession-and-output-file-looks-like-in-json-format/178835 "2019-05-02T06:02:12Z")
Hi Team, Logstash configuraiton run as below input { file{ path =\> "C:/elkstack/elasticsearch-6.5.1/logs/userscenario.csv" start\_position =\> "beginning" sincedb\_path =\> "C:/elkstack/elasticsearch-6.5.1/sincedb/sinc… --- ## [DNS Filter Lookup Failures since 6.7 upgrade](https://discuss.elastic.co/t/dns-filter-lookup-failures-since-6-7-upgrade/174577)
**Author:** [@millap](https://discuss.elastic.co/u/millap)\ **Replies:** 13\ **Last updated:** [May 2, 2019, 5:57am UTC](https://discuss.elastic.co/t/dns-filter-lookup-failures-since-6-7-upgrade/174577 "2019-05-02T05:57:44Z")
Hi Guys, Hoping someone can offer some guidance. Since our LS upgraded to 6.7.0, we've consistently seen issues with LS resolving DNS entries which appears to build over time. E.g - \[logstash.filters.dns \] DNS: timeout… --- ## [Input / filter plugin for Google bigquery](https://discuss.elastic.co/t/input-filter-plugin-for-google-bigquery/179277)
**Author:** [@Praneet\_Khandelwal](https://discuss.elastic.co/u/Praneet_Khandelwal)\ **Replies:** 0\ **Last updated:** [May 2, 2019, 5:42am UTC](https://discuss.elastic.co/t/input-filter-plugin-for-google-bigquery/179277 "2019-05-02T05:42:02Z")
I am running a logstash instance which is supposed to receive data from Google BigQuery. The issue I am facing is that I need to somehow map the data coming with the schema and create a document by combining them. Is the… --- ## [How to parse and add filter in nested json](https://discuss.elastic.co/t/how-to-parse-and-add-filter-in-nested-json/179017)
**Author:** [@stour](https://discuss.elastic.co/u/stour)\ **Replies:** 2\ **Last updated:** [May 2, 2019, 5:39am UTC](https://discuss.elastic.co/t/how-to-parse-and-add-filter-in-nested-json/179017 "2019-05-02T05:39:19Z")
Hi, I am trying to parse below json. { "s\_no":1011, "uuid":"1f86c6ce84678e9bd5d67bcd49a86d3a99129d80", "data":{ "time":"2019-01-01T13:32:10.8287355Z", "system":{ "processors":\[ { "name":"Intel(R) Core(TM) i5-35… --- ## [ES output using ILM settings "Malformed escape pair at index"](https://discuss.elastic.co/t/es-output-using-ilm-settings-malformed-escape-pair-at-index/178117)
**Author:** [@justinw](https://discuss.elastic.co/u/justinw)\ **Replies:** 1\ **Last updated:** [May 1, 2019, 11:42pm UTC](https://discuss.elastic.co/t/es-output-using-ilm-settings-malformed-escape-pair-at-index/178117 "2019-05-01T23:42:09Z")
Hi, Using logstash 6.7, I'm switching from daily rotated indices to ILM, and am seeing an error when trying to template the ILM settings. Error java.net.URISyntaxException: Malformed escape pair at index 0: %{dest}-6.… --- ## [Logstash doesn't Import CSV columns correctly](https://discuss.elastic.co/t/logstash-doesnt-import-csv-columns-correctly/179253)
**Author:** [@Ac3](https://discuss.elastic.co/u/Ac3)\ **Replies:** 2\ **Last updated:** [May 1, 2019, 10:49pm UTC](https://discuss.elastic.co/t/logstash-doesnt-import-csv-columns-correctly/179253 "2019-05-01T22:49:00Z")
Hi I am importing some IP data stored in CSV into ELK using Logstash. When I view the data in Kibana however the columns are all over the place and they are formatted fine in the CSV? Kibana Screenshot Does ELK stru… --- ## [Automatic CSV generation with Logstash](https://discuss.elastic.co/t/automatic-csv-generation-with-logstash/179264)
**Author:** [@matheusribeirogarcia](https://discuss.elastic.co/u/matheusribeirogarcia)\ **Replies:** 0\ **Last updated:** [May 1, 2019, 10:39pm UTC](https://discuss.elastic.co/t/automatic-csv-generation-with-logstash/179264 "2019-05-01T22:39:08Z")
I know there's an option to generate CSV files manually, but is there a way to generate this CSV file automatically? --- ## [Getting Long/Lat From Physical Address](https://discuss.elastic.co/t/getting-long-lat-from-physical-address/179251)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 3\ **Last updated:** [May 1, 2019, 8:56pm UTC](https://discuss.elastic.co/t/getting-long-lat-from-physical-address/179251 "2019-05-01T20:56:32Z")
Just as the title says, I have a field that is the complete physical address of locations. Is there a way to lookup long/lat for each address so that I can map these events to a visual in Kibana? --- ## [The \[default\] mapping cannot be updated on index](https://discuss.elastic.co/t/the-default-mapping-cannot-be-updated-on-index/179255)
**Author:** [@DanSheps](https://discuss.elastic.co/u/DanSheps)\ **Replies:** 0\ **Last updated:** [May 1, 2019, 8:48pm UTC](https://discuss.elastic.co/t/the-default-mapping-cannot-be-updated-on-index/179255 "2019-05-01T20:48:58Z")
Hello all, I recently upgraded to ELK 7.0 and I am having problems with a pipeline. Previously, I did not have any template set, so it was using the default logstash template. After the update, errors were being throw… --- ## [Reading gzipped files and shutting down Logstash once done](https://discuss.elastic.co/t/reading-gzipped-files-and-shutting-down-logstash-once-done/179241)
**Author:** [@emilg02](https://discuss.elastic.co/u/emilg02)\ **Replies:** 1\ **Last updated:** [May 1, 2019, 7:22pm UTC](https://discuss.elastic.co/t/reading-gzipped-files-and-shutting-down-logstash-once-done/179241 "2019-05-01T19:22:05Z")
Hi all, My use case is as follows - I need to process gzipped files by demand and I cannot afford having an instance of Logstash continuously running - therefore I must launch an instance, have it process the file, and … --- ## [Read a particular file from a folder in S3 bucket using logstash](https://discuss.elastic.co/t/read-a-particular-file-from-a-folder-in-s3-bucket-using-logstash/179239)
**Author:** [@behers10](https://discuss.elastic.co/u/behers10)\ **Replies:** 0\ **Last updated:** [May 1, 2019, 7:01pm UTC](https://discuss.elastic.co/t/read-a-particular-file-from-a-folder-in-s3-bucket-using-logstash/179239 "2019-05-01T19:01:57Z")
Hi , I have been trying to pull a particular file from a particular folder in s3 bucket . Could any one suggest any approach . i tried to use prefix utility in logstash plugin , but that not seems to be working. Than… --- ## [Logstash plugin to read from HDFS or HIVE](https://discuss.elastic.co/t/logstash-plugin-to-read-from-hdfs-or-hive/179238)
**Author:** [@behers10](https://discuss.elastic.co/u/behers10)\ **Replies:** 0\ **Last updated:** [May 1, 2019, 6:58pm UTC](https://discuss.elastic.co/t/logstash-plugin-to-read-from-hdfs-or-hive/179238 "2019-05-01T18:58:43Z")
Hi , I am trying to figure out a way to read data from HDFS or HIVE table using logstash plugin. Request anybody to let me know any approach as there are no correct logstash input plugin to do that . Thanks in Advance … --- ## [Configuring Netflow Module](https://discuss.elastic.co/t/configuring-netflow-module/179230)
**Author:** [@etsai7](https://discuss.elastic.co/u/etsai7)\ **Replies:** 0\ **Last updated:** [May 1, 2019, 4:39pm UTC](https://discuss.elastic.co/t/configuring-netflow-module/179230 "2019-05-01T16:39:04Z")
Hello, How/Where can I configure the IP for the netflow module in Logstash to listen, through UDP for netflow packets Thanks, Eric --- ## [Accessing Redis cache for dynamic keys](https://discuss.elastic.co/t/accessing-redis-cache-for-dynamic-keys/179209)
**Author:** [@Dipal\_Modi](https://discuss.elastic.co/u/Dipal_Modi)\ **Replies:** 3\ **Last updated:** [May 1, 2019, 3:14pm UTC](https://discuss.elastic.co/t/accessing-redis-cache-for-dynamic-keys/179209 "2019-05-01T15:14:23Z")
While using redis input plugin, it requires a key to be specified in the configuration only. Is there any way to build key using another data source for which we can use redis cache? The same functionality is available… --- ## [Error when installing logstash plugin](https://discuss.elastic.co/t/error-when-installing-logstash-plugin/179210)
**Author:** [@scorpioy1](https://discuss.elastic.co/u/scorpioy1)\ **Replies:** 0\ **Last updated:** [May 1, 2019, 1:49pm UTC](https://discuss.elastic.co/t/error-when-installing-logstash-plugin/179210 "2019-05-01T13:49:24Z")
Hi, I'm trying to install some logstash plugin but got errors. logstash-6.6.0\> bin/logstash-plugin install logstash-output-exec-3.1.4.gem LoadError: load error: jopenssl/load -- java.lang.VerifyError: class org.bouncyc… --- ## [Stdin in logstach](https://discuss.elastic.co/t/stdin-in-logstach/179206)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 1\ **Last updated:** [May 1, 2019, 1:44pm UTC](https://discuss.elastic.co/t/stdin-in-logstach/179206 "2019-05-01T13:44:02Z")
hi my friend, i want to dipslay data wrotten from keyboard in kibana, what should i do in logstash and what is the config and commande to get the data from keyboard thank you --- ## [Remove unwanted fields](https://discuss.elastic.co/t/remove-unwanted-fields/178795)
**Author:** [@Jan\_Kaspar](https://discuss.elastic.co/u/Jan_Kaspar)\ **Replies:** 8\ **Last updated:** [May 1, 2019, 1:30pm UTC](https://discuss.elastic.co/t/remove-unwanted-fields/178795 "2019-05-01T13:30:45Z")
Hi all!, I just have a question. I am using filebeat for parsing XML file generated by NPS. It seems like it working fine. There are some things i would like to adjust. This is message: 04/28/2019 09:44:02.897W2012-AR… --- ## [Looking for an Elasticsearch mapping example](https://discuss.elastic.co/t/looking-for-an-elasticsearch-mapping-example/179155)
**Author:** [@databloom](https://discuss.elastic.co/u/databloom)\ **Replies:** 1\ **Last updated:** [April 30, 2019, 10:26pm UTC](https://discuss.elastic.co/t/looking-for-an-elasticsearch-mapping-example/179155 "2019-04-30T22:26:18Z")
I spent some time using Packetbeat, great tool, but I need to dig into nfs filehandles and metadata in the ops that I just can't get in the packetbeat dump. I found this post, which is everything I could hope for except… --- ## [Losgstash error starting with Expected one of #, {, ,, \] at line XX](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072)
**Author:** [@franco.federico](https://discuss.elastic.co/u/franco.federico)\ **Replies:** 9\ **Last updated:** [April 30, 2019, 10:24pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072 "2019-04-30T22:24:52Z")
I create a grok filter in order to capture this information with logstash from a syslog message The message of Syslog is like this "Logon Attempt";"4624";"Audit Success";"NetApp-Security-Auditing";"CIFS";"Security";"IT… --- ## [Logstash support for netflow as output](https://discuss.elastic.co/t/logstash-support-for-netflow-as-output/179098)
**Author:** [@Dipal\_Modi](https://discuss.elastic.co/u/Dipal_Modi)\ **Replies:** 1\ **Last updated:** [April 30, 2019, 10:16pm UTC](https://discuss.elastic.co/t/logstash-support-for-netflow-as-output/179098 "2019-04-30T22:16:17Z")
Is logstash support netflow as output? In current structure, we can specify netflow input using input { udp { host =\> "127.0.0.1" port =\> 5555 codec =\> netflow { versions =\> \[5, 9, 10\] } } } Does same thing av… --- ## [Extraxt matching words in text file](https://discuss.elastic.co/t/extraxt-matching-words-in-text-file/179156)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 1\ **Last updated:** [April 30, 2019, 10:05pm UTC](https://discuss.elastic.co/t/extraxt-matching-words-in-text-file/179156 "2019-04-30T22:05:05Z")
hi dears, i have a text file (or log) file and i want to know if some words such as "animal" "birds" ... if they exist in my file; can you give me examples of filter input and output in logstach thank you very much --- ## [Snort Logs with FileBeat](https://discuss.elastic.co/t/snort-logs-with-filebeat/177879)
**Author:** [@johndowe](https://discuss.elastic.co/u/johndowe)\ **Replies:** 4\ **Last updated:** [April 30, 2019, 9:20pm UTC](https://discuss.elastic.co/t/snort-logs-with-filebeat/177879 "2019-04-30T21:20:29Z")
Hi, I have setup filebeat on a pi running Snort sending logs to a cloud ELK stack. I am trying to figure out how to arrange logs and doing the following process: on the beats side i have this in the filebeat.yml: path… --- ## [Logstash Module Netflow](https://discuss.elastic.co/t/logstash-module-netflow/175711)
**Author:** [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Replies:** 10\ **Last updated:** [April 30, 2019, 8:30pm UTC](https://discuss.elastic.co/t/logstash-module-netflow/175711 "2019-04-30T20:30:05Z")
Hi, I want to load the netflow module on Centos 7 with Java 8 and ELK V.7.0 RC 2. All services are up and everything looks fine. I had a little bit trouble with Java because I wanted to use Java 11. Now when I want to… --- ## [CSV file into ES through logstash](https://discuss.elastic.co/t/csv-file-into-es-through-logstash/179106)
**Author:** [@userelastic](https://discuss.elastic.co/u/userelastic)\ **Replies:** 2\ **Last updated:** [April 30, 2019, 5:01pm UTC](https://discuss.elastic.co/t/csv-file-into-es-through-logstash/179106 "2019-04-30T17:01:33Z")
Newbie here; I am trying to get a CSV file from a windows server and send it to ES through logstash; I am hoping to get the columns in ES so we can do proper searches; but it seems all the CSV output goes into message fi… --- ## [Split data and convert to json](https://discuss.elastic.co/t/split-data-and-convert-to-json/179083)
**Author:** [@rahulkothanath](https://discuss.elastic.co/u/rahulkothanath)\ **Replies:** 5\ **Last updated:** [April 30, 2019, 3:19pm UTC](https://discuss.elastic.co/t/split-data-and-convert-to-json/179083 "2019-04-30T15:19:20Z")
I want to split a text ,based on a pattern and parse each split separate and combine last to make final json. I am new to log-stash and wants to find the best solution for my problem. Input: @CM003352018053013591473247… --- ## [How to convert string to geoip datatype while ingesting data from logstash to elastic search](https://discuss.elastic.co/t/how-to-convert-string-to-geoip-datatype-while-ingesting-data-from-logstash-to-elastic-search/179095)
**Author:** [@Swati123](https://discuss.elastic.co/u/Swati123)\ **Replies:** 1\ **Last updated:** [April 30, 2019, 3:01pm UTC](https://discuss.elastic.co/t/how-to-convert-string-to-geoip-datatype-while-ingesting-data-from-logstash-to-elastic-search/179095 "2019-04-30T15:01:26Z")
Hi all, I have a log field by default as string src\_ip = 10.14.114.18 I am able to have it as ip datatype in kibana by running default mappings API. But for visualization, I need this as geoip datatype so that it can b… --- ## [Can I store entire grok pattern in a variable](https://discuss.elastic.co/t/can-i-store-entire-grok-pattern-in-a-variable/179101)
**Author:** [@rahulkothanath](https://discuss.elastic.co/u/rahulkothanath)\ **Replies:** 3\ **Last updated:** [April 30, 2019, 2:57pm UTC](https://discuss.elastic.co/t/can-i-store-entire-grok-pattern-in-a-variable/179101 "2019-04-30T14:57:31Z")
I have filter like below: filter { grok { match =\> { "message" =\> "(?\^.{3}) %{GREEDYDATA:body}" } } can I put "(?\^.{3}) %{GREEDYDATA:body}" inside a variable and use it a… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=554) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=556)