# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=463 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 464 --- ## [Forwarding logs](https://discuss.elastic.co/t/forwarding-logs/203332)
**Author:** [@trajano](https://discuss.elastic.co/u/trajano)\ **Replies:** 3\ **Last updated:** [October 13, 2019, 5:43am UTC](https://discuss.elastic.co/t/forwarding-logs/203332 "2019-10-13T05:43:13Z")
I'm trying to set up a log spooling system where application logs sent by applications from one region is spooled and sent to another region. Since logstash-forwarder is deprecated, I was thinking of how to do this with… --- ## [Problem with parsing the CSV file correctly](https://discuss.elastic.co/t/problem-with-parsing-the-csv-file-correctly/203320)
**Author:** [@Rysiu](https://discuss.elastic.co/u/Rysiu)\ **Replies:** 3\ **Last updated:** [October 12, 2019, 4:59pm UTC](https://discuss.elastic.co/t/problem-with-parsing-the-csv-file-correctly/203320 "2019-10-12T16:59:34Z")
Hello, I have a problem with parsing the CSV file in Logstash. The structure of my template in index looks like this: { "mappings": { "doc": { "properties": { "ip\_src": { "type": "ip" }, "ip\_dst": { "ty… --- ## [How to use logstash-input-http-poller from zabbix api method?](https://discuss.elastic.co/t/how-to-use-logstash-input-http-poller-from-zabbix-api-method/203231)
**Author:** [@uf012366](https://discuss.elastic.co/u/uf012366)\ **Replies:** 2\ **Last updated:** [October 12, 2019, 1:42am UTC](https://discuss.elastic.co/t/how-to-use-logstash-input-http-poller-from-zabbix-api-method/203231 "2019-10-12T01:42:52Z")
i try use Http\_poller input plugin loading Zabbix data, but cannot use ZABBIX api method loading data. how to use zabbix jsonrpc api method in the logstash Http\_poller input plugin ???? Are there any cases in this reg… --- ## [Empty variable vs. non-existent one](https://discuss.elastic.co/t/empty-variable-vs-non-existent-one/203095)
**Author:** [@pgf](https://discuss.elastic.co/u/pgf)\ **Replies:** 4\ **Last updated:** [October 11, 2019, 6:18pm UTC](https://discuss.elastic.co/t/empty-variable-vs-non-existent-one/203095 "2019-10-11T18:18:02Z")
Is there any way to tell the difference between an empty variable and a non-existent one? The code if \[v1\] == "" { do something } acts exactly the same whether the variable exists and is empty or does not exist at al… --- ## [How do we read the filename when the file is uploaded in the azure storage blob and then extract this name and use grok filter command to save it in elastic search](https://discuss.elastic.co/t/how-do-we-read-the-filename-when-the-file-is-uploaded-in-the-azure-storage-blob-and-then-extract-this-name-and-use-grok-filter-command-to-save-it-in-elastic-search/203271)
**Author:** [@divyapoojary](https://discuss.elastic.co/u/divyapoojary)\ **Replies:** 0\ **Last updated:** [October 11, 2019, 4:20pm UTC](https://discuss.elastic.co/t/how-do-we-read-the-filename-when-the-file-is-uploaded-in-the-azure-storage-blob-and-then-extract-this-name-and-use-grok-filter-command-to-save-it-in-elastic-search/203271 "2019-10-11T16:20:32Z")
Hi Team, Requirement: To read the filename of the file getting processed by logstash and put it in elastic search. The input is azure storage blob. So need help in writing the configuration such that when the logstash … --- ## [Logstash is not generating log file](https://discuss.elastic.co/t/logstash-is-not-generating-log-file/202827)
**Author:** [@Suresh\_Pal](https://discuss.elastic.co/u/Suresh_Pal)\ **Replies:** 4\ **Last updated:** [October 11, 2019, 4:11pm UTC](https://discuss.elastic.co/t/logstash-is-not-generating-log-file/202827 "2019-10-11T16:11:51Z")
Hi team, My logstash is not generating log file. /var/log/logstash Config file: Settings file in YAML Settings can be specified either in hierarchical form, e.g.: pipeline: batch: size: 125 delay: 5 Or as flat ke… --- ## [Logstash not able to send Event to Splunk HTTP Event Collector](https://discuss.elastic.co/t/logstash-not-able-to-send-event-to-splunk-http-event-collector/203149)
**Author:** [@as50790](https://discuss.elastic.co/u/as50790)\ **Replies:** 4\ **Last updated:** [October 11, 2019, 3:24pm UTC](https://discuss.elastic.co/t/logstash-not-able-to-send-event-to-splunk-http-event-collector/203149 "2019-10-11T15:24:03Z")
I'm trying to use logstash to send data directly to an http event collector (HEC). The HEC collector accepts the following correctly. curl -k https://10.8.71.164:8088/services/collector/raw -H "Authorization: Splunk a3… --- ## [Timestamp not match](https://discuss.elastic.co/t/timestamp-not-match/203212)
**Author:** [@DorianL](https://discuss.elastic.co/u/DorianL)\ **Replies:** 1\ **Last updated:** [October 11, 2019, 3:15pm UTC](https://discuss.elastic.co/t/timestamp-not-match/203212 "2019-10-11T15:15:28Z")
Hello, I have a problem with my date filter. I have 2 hours of shift. Here is my filter date { match =\> \[ "log\_date", "MMM dd HH:mm:ss.SSS", "MMM d HH:mm:ss.SSS", "ISO8601" \] timezone =\> "Europe/Paris" ta… --- ## [Warnings when installing logstash?](https://discuss.elastic.co/t/warnings-when-installing-logstash/203201)
**Author:** [@hanserik](https://discuss.elastic.co/u/hanserik)\ **Replies:** 1\ **Last updated:** [October 11, 2019, 3:12pm UTC](https://discuss.elastic.co/t/warnings-when-installing-logstash/203201 "2019-10-11T15:12:16Z")
When installling release 7.4 of logstash, I get below warnings. Do I need to be concerned? What needs to be done? OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely… --- ## [What value contains logstash\_last\_run file(default)](https://discuss.elastic.co/t/what-value-contains-logstash-last-run-file-default/203244)
**Author:** [@rameshkr1994](https://discuss.elastic.co/u/rameshkr1994)\ **Replies:** 1\ **Last updated:** [October 11, 2019, 2:55pm UTC](https://discuss.elastic.co/t/what-value-contains-logstash-last-run-file-default/203244 "2019-10-11T14:55:17Z")
Dear All. i having trouble while running simple sql file from logstash. i am getting metadata issue from logstash\_last\_run file. could you please tell me for default configuration for this file(logstash\_last\_run ). … --- ## [Mib import: Issue converting hpux mib using smidump for snmp input filter](https://discuss.elastic.co/t/mib-import-issue-converting-hpux-mib-using-smidump-for-snmp-input-filter/203119)
**Author:** [@vshankara](https://discuss.elastic.co/u/vshankara)\ **Replies:** 0\ **Last updated:** [October 10, 2019, 10:14pm UTC](https://discuss.elastic.co/t/mib-import-issue-converting-hpux-mib-using-smidump-for-snmp-input-filter/203119 "2019-10-10T22:14:59Z")
Trying to convert hpux mib for use in snmp input filter. using smidump gives the below error smidump: module 'hpux.mib' contains errors, expect flawed output --- ## [How to set secret file logstash](https://discuss.elastic.co/t/how-to-set-secret-file-logstash/203213)
**Author:** [@kiran\_kumar1](https://discuss.elastic.co/u/kiran_kumar1)\ **Replies:** 0\ **Last updated:** [October 11, 2019, 10:44am UTC](https://discuss.elastic.co/t/how-to-set-secret-file-logstash/203213 "2019-10-11T10:44:30Z")
Hi Team, I am newer to ELK, how can I set the secretName for logstash. here am getting this issue: Successfully assigned 2304670352/logstash-0 to acs-node58.usrno1.applecloud.io Normal SuccessfulAttachVolume 3m4s … --- ## [Use JSON structure from input in HTTP request](https://discuss.elastic.co/t/use-json-structure-from-input-in-http-request/203044)
**Author:** [@MMH](https://discuss.elastic.co/u/MMH)\ **Replies:** 2\ **Last updated:** [October 11, 2019, 9:01am UTC](https://discuss.elastic.co/t/use-json-structure-from-input-in-http-request/203044 "2019-10-11T09:01:41Z")
I am trying to use JSON struct from input inside the HTTP filter request. The part in my input looks like this: {"elemId":"1","user":\[{"uid" : 1,"first": "John","last": "Smith"},{"uid" : 2,"first": "Bob","last": "Jone… --- ## [Logstash.licensechecker.licensereader error when enable SSL between logstash and elasticsearch](https://discuss.elastic.co/t/logstash-licensechecker-licensereader-error-when-enable-ssl-between-logstash-and-elasticsearch/202640)
**Author:** [@rambo](https://discuss.elastic.co/u/rambo)\ **Replies:** 1\ **Last updated:** [October 11, 2019, 8:36am UTC](https://discuss.elastic.co/t/logstash-licensechecker-licensereader-error-when-enable-ssl-between-logstash-and-elasticsearch/202640 "2019-10-11T08:36:38Z")
Hi I enabled SSL, but logstash met below error: \[2019-10-08T17:21:54,292\]\[ERROR\]\[logstash.licensechecker.licensereader\] Unable to retrieve license information from license server {:message=\>"No Available connections"} … --- ## [How to create secure connection with elasticsearch in ruby client?](https://discuss.elastic.co/t/how-to-create-secure-connection-with-elasticsearch-in-ruby-client/203031)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 1\ **Last updated:** [October 11, 2019, 8:15am UTC](https://discuss.elastic.co/t/how-to-create-secure-connection-with-elasticsearch-in-ruby-client/203031 "2019-10-11T08:15:02Z")
in using logstash to create client connection with elasticsearch for \_search endpoint. it is running fine with http, but not with https. currently i am using elasticsearch-transport client gem in my code. can someone … --- ## [How to install ssl in logstash server when creating secure (https) connection with elasticsearch?](https://discuss.elastic.co/t/how-to-install-ssl-in-logstash-server-when-creating-secure-https-connection-with-elasticsearch/203167)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 0\ **Last updated:** [October 11, 2019, 7:13am UTC](https://discuss.elastic.co/t/how-to-install-ssl-in-logstash-server-when-creating-secure-https-connection-with-elasticsearch/203167 "2019-10-11T07:13:16Z")
help me ? to resolve this ? --- ## [Posting document to alias instead of index](https://discuss.elastic.co/t/posting-document-to-alias-instead-of-index/203125)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 4\ **Last updated:** [October 11, 2019, 5:00am UTC](https://discuss.elastic.co/t/posting-document-to-alias-instead-of-index/203125 "2019-10-11T05:00:15Z")
I'm not quite sure why this isn't working but I'm trying to post to an alias instead of an index and it's not working. I have indexes called index-september & index-october and they are both under the alias index. When … --- ## [\_grokparsefailure help](https://discuss.elastic.co/t/grokparsefailure-help/203124)
**Author:** [@TechGeekNZ](https://discuss.elastic.co/u/TechGeekNZ)\ **Replies:** 1\ **Last updated:** [October 11, 2019, 2:43am UTC](https://discuss.elastic.co/t/grokparsefailure-help/203124 "2019-10-11T02:43:49Z")
Hi all, Seem to be having trouble getting some of my logs into logstash. Have been able to sucessfully work through most of my logs, but this last one is giving me grief. Example line from logfile; 2019-10-01 22:25:54… --- ## [Elastic filter not executing query? Bool etc. supported?](https://discuss.elastic.co/t/elastic-filter-not-executing-query-bool-etc-supported/202948)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 2\ **Last updated:** [October 11, 2019, 2:17am UTC](https://discuss.elastic.co/t/elastic-filter-not-executing-query-bool-etc-supported/202948 "2019-10-11T02:17:28Z")
Hi, I'm trying to use the elastic filter to get some data from a index. However the elastic filter doesn't appear to be doing anything. When starting logstash I see New ElasticSearch filter client {:hosts=\>\["https://.ur… --- ## [How to have tsvb Gauge visualization showing the last value](https://discuss.elastic.co/t/how-to-have-tsvb-gauge-visualization-showing-the-last-value/203115)
**Author:** [@ferenc0521](https://discuss.elastic.co/u/ferenc0521)\ **Replies:** 0\ **Last updated:** [October 10, 2019, 9:34pm UTC](https://discuss.elastic.co/t/how-to-have-tsvb-gauge-visualization-showing-the-last-value/203115 "2019-10-10T21:34:51Z")
When dashboarding netdata host metrics , using tsvb line graps, I can have the avg by 30s graphed which is more or less the last value I'm interested in. (netdata samples 3000 metrics every sec, logstash http poller sam… --- ## [Grok unable to handle symbol](https://discuss.elastic.co/t/grok-unable-to-handle-symbol/203111)
**Author:** [@Senthil\_ak](https://discuss.elastic.co/u/Senthil_ak)\ **Replies:** 5\ **Last updated:** [October 10, 2019, 10:50pm UTC](https://discuss.elastic.co/t/grok-unable-to-handle-symbol/203111 "2019-10-10T22:50:32Z")
Hi Admin, I have a grok pattern from nginx which is working perfectly when the data is coming. grok { match =\> { "message" =\> '%{DATA:clientip} %{DATA:VirtualHost} %{DATA:User} \\\[%{HTTPDATE:timestamp}\\\] %{DATA:nginx.… --- ## [Cloudwatch vs Cloudwatch Logs input plugins](https://discuss.elastic.co/t/cloudwatch-vs-cloudwatch-logs-input-plugins/203112)
**Author:** [@EZprogramming](https://discuss.elastic.co/u/EZprogramming)\ **Replies:** 0\ **Last updated:** [October 10, 2019, 9:02pm UTC](https://discuss.elastic.co/t/cloudwatch-vs-cloudwatch-logs-input-plugins/203112 "2019-10-10T21:02:54Z")
Hi everyone, what is the difference between cloudwatch Logstash input plugin vs cloudwatch\_logs input plugin written by Lukewaite (Link)? My assumption is that the official cloudwatch input plugin is to scan and pull m… --- ## [How do you tag by multiple source IPs?](https://discuss.elastic.co/t/how-do-you-tag-by-multiple-source-ips/202917)
**Author:** [@meatwad](https://discuss.elastic.co/u/meatwad)\ **Replies:** 7\ **Last updated:** [October 10, 2019, 8:53pm UTC](https://discuss.elastic.co/t/how-do-you-tag-by-multiple-source-ips/202917 "2019-10-10T20:53:26Z")
Hi, I'm transitioning to logstash from an rsyslog config that ingests logs from a list of network devices (routers, switches, firewalls), then does something based on the source IP. Here's an abbreviated example: if … --- ## [Geo\_point and geohash in logstash](https://discuss.elastic.co/t/geo-point-and-geohash-in-logstash/202880)
**Author:** [@rohitg](https://discuss.elastic.co/u/rohitg)\ **Replies:** 13\ **Last updated:** [October 10, 2019, 8:28pm UTC](https://discuss.elastic.co/t/geo-point-and-geohash-in-logstash/202880 "2019-10-10T20:28:07Z")
Hi, I am trying to use coordinate maps in kibana for my visualization. I mutate my longitude and latitude fields with rename and convert to float, but when i tried in Kibana, it showing no compactible field geo\_point. … --- ## [Http-pooler issue . Cannot send API call](https://discuss.elastic.co/t/http-pooler-issue-cannot-send-api-call/203096)
**Author:** [@Xor44](https://discuss.elastic.co/u/Xor44)\ **Replies:** 2\ **Last updated:** [October 10, 2019, 8:26pm UTC](https://discuss.elastic.co/t/http-pooler-issue-cannot-send-api-call/203096 "2019-10-10T20:26:27Z")
Hi Folks , I have some issue with my configuration . I'm not able to get data in my elastic cluster . I'm using the configuration below input { http\_poller { urls =\> { method =\> get headers =\> { X-xxx-APIToken =\> … --- ## [Unable to parse nested JSON using logstash](https://discuss.elastic.co/t/unable-to-parse-nested-json-using-logstash/202906)
**Author:** [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\ **Replies:** 9\ **Last updated:** [October 10, 2019, 8:09pm UTC](https://discuss.elastic.co/t/unable-to-parse-nested-json-using-logstash/202906 "2019-10-10T20:09:00Z")
Nested JSON response - { "application": "FinClient", "occurredOn": "2019-10-07T10:38:39", "host": "U8HPP213", "finEvents": \[ { "@xmlns": "http://jabber.org/protocol/pubsub", "data": { "apiErrors": { "apiError":… --- ## [How to split xml into different events](https://discuss.elastic.co/t/how-to-split-xml-into-different-events/203039)
**Author:** [@griffer98](https://discuss.elastic.co/u/griffer98)\ **Replies:** 2\ **Last updated:** [October 10, 2019, 7:10pm UTC](https://discuss.elastic.co/t/how-to-split-xml-into-different-events/203039 "2019-10-10T19:10:47Z")
I am trying to take certain parts of an xml document and split it into different events that I can look at separately in Kibana. Here is a sample xml document here. \ \ \some company\ **Author:** [@user2416](https://discuss.elastic.co/u/user2416)\ **Replies:** 0\ **Last updated:** [October 10, 2019, 6:57pm UTC](https://discuss.elastic.co/t/listing-all-files-in-s3-everytime-new-file-is-added/203102 "2019-10-10T18:57:58Z") Hi, We are using logstash to read logs from S3, we have millions of files in S3 bucket for Cloudtrail logs. When a new file is added, logstash is listing all files first and then reading the logs from new file, its taki… --- ## [Date filter - how to parse separately grokked DATE and TIME](https://discuss.elastic.co/t/date-filter-how-to-parse-separately-grokked-date-and-time/202712)
**Author:** [@kmiklas](https://discuss.elastic.co/u/kmiklas)\ **Replies:** 2\ **Last updated:** [October 10, 2019, 6:32pm UTC](https://discuss.elastic.co/t/date-filter-how-to-parse-separately-grokked-date-and-time/202712 "2019-10-10T18:32:11Z")
Hello All, I have separately grokked DATE and TIME fields from a comma-separated string. I now wish to filter them from two string into a date/time format with the Logstash date filter plugin. I'm struggling with the … --- ## [How to modify output codec for logstash](https://discuss.elastic.co/t/how-to-modify-output-codec-for-logstash/201296)
**Author:** [@shazbot](https://discuss.elastic.co/u/shazbot)\ **Replies:** 7\ **Last updated:** [October 10, 2019, 5:53pm UTC](https://discuss.elastic.co/t/how-to-modify-output-codec-for-logstash/201296 "2019-10-10T17:53:45Z")
I'm not expecting a step by step on this. But here is the backstory and what I hope to achieve. I have logstash decoding netflow and putting it out using json\_lines This is working well, but it's very verbose and too … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=462) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=464)