# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=625 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 626 --- ## [How to check the tags values in logstash](https://discuss.elastic.co/t/how-to-check-the-tags-values-in-logstash/162474)
**Author:** [@Sripal](https://discuss.elastic.co/u/Sripal)\ **Replies:** 24\ **Last updated:** [January 3, 2019, 6:14am UTC](https://discuss.elastic.co/t/how-to-check-the-tags-values-in-logstash/162474 "2019-01-03T06:14:15Z")
Dear All, I'm trying to read my access log but i'm getting below error. Please help Log lines: \[1047:3a80:421:319:7e91:ac89:eda5:8c9d\], 10.10.10.10:10 - - - \[26/Dec/2018:02:31:14 +0530\] "POST /my/app/path/android/quer… --- ## [Logstash Error - Deadletter queue/plugin - troubleshooting](https://discuss.elastic.co/t/logstash-error-deadletter-queue-plugin-troubleshooting/160250)
**Author:** [@Karthik\_Ramachandran](https://discuss.elastic.co/u/Karthik_Ramachandran)\ **Replies:** 1\ **Last updated:** [January 2, 2019, 9:21pm UTC](https://discuss.elastic.co/t/logstash-error-deadletter-queue-plugin-troubleshooting/160250 "2019-01-02T21:21:45Z")
Dear All, Trying to trouble shoot the logstash issues that I'm facing, and any hints would be helpful Background Note the issue is in both prod and dev, and the numbers below are from dev Log ES Version : 6.2.1 Moni… --- ## [Logstash Error- Failing to send monitoring events](https://discuss.elastic.co/t/logstash-error-failing-to-send-monitoring-events/160252)
**Author:** [@Karthik\_Ramachandran](https://discuss.elastic.co/u/Karthik_Ramachandran)\ **Replies:** 1\ **Last updated:** [January 2, 2019, 9:21pm UTC](https://discuss.elastic.co/t/logstash-error-failing-to-send-monitoring-events/160252 "2019-01-02T21:21:20Z")
Dear All, Trying to trouble shoot the logstash issues that I'm facing, and any hints would be helpful Background Note the issue is in both prod and dev, and the numbers below are from dev Log ES Version : 6.2.1 Moni… --- ## [Logstash cache lookup data](https://discuss.elastic.co/t/logstash-cache-lookup-data/162687)
**Author:** [@roblopes](https://discuss.elastic.co/u/roblopes)\ **Replies:** 3\ **Last updated:** [January 2, 2019, 7:15pm UTC](https://discuss.elastic.co/t/logstash-cache-lookup-data/162687 "2019-01-02T19:15:37Z")
Hi, I am moving a legacy logging system to Elastic stack, and there are couple fields in the existing system that are using foreign keys for a lookup table (around 1000 rows, but it can grow or shrink since users can cr… --- ## [Not getting a listening port for Beats Input with Logstash](https://discuss.elastic.co/t/not-getting-a-listening-port-for-beats-input-with-logstash/161759)
**Author:** [@ciphee](https://discuss.elastic.co/u/ciphee)\ **Replies:** 2\ **Last updated:** [January 2, 2019, 6:19pm UTC](https://discuss.elastic.co/t/not-getting-a-listening-port-for-beats-input-with-logstash/161759 "2019-01-02T18:19:16Z")
Hello, I have logstash installed in my enviornment and it currently only has 1 pipeline setup for syslog messages, this one pipeline is sucessfully opening a listening port for the logs to be received. I setup a second… --- ## [Logstash not writing to ElasticSearch](https://discuss.elastic.co/t/logstash-not-writing-to-elasticsearch/162696)
**Author:** [@mad](https://discuss.elastic.co/u/mad)\ **Replies:** 1\ **Last updated:** [January 2, 2019, 5:55pm UTC](https://discuss.elastic.co/t/logstash-not-writing-to-elasticsearch/162696 "2019-01-02T17:55:16Z")
I am using the following -\> filebeat-\>logstash-\>elasticsearch I am getting 2 types of logs, Apache and Log4J, and trying to create different indexes output { if \[flow\] == "apache" { elasticsearch { hosts =\> \["xx.xx… --- ## [How to monitor logstash remotely?](https://discuss.elastic.co/t/how-to-monitor-logstash-remotely/162682)
**Author:** [@varun1992](https://discuss.elastic.co/u/varun1992)\ **Replies:** 3\ **Last updated:** [January 2, 2019, 5:37pm UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-remotely/162682 "2019-01-02T17:37:51Z")
Is there any logstash monitoring tool like cerebro for elasticsearch ? because currently i have no idea how it is performing i checked visualvm, but it didn't give info other than heap size and cpu usage please help --- ## [Use HDFS as Logstash input](https://discuss.elastic.co/t/use-hdfs-as-logstash-input/161045)
**Author:** [@skluzada](https://discuss.elastic.co/u/skluzada)\ **Replies:** 0\ **Last updated:** [December 16, 2018, 6:08pm UTC](https://discuss.elastic.co/t/use-hdfs-as-logstash-input/161045 "2018-12-16T18:08:45Z")
I have a file saved on HDFS and I would like to use this file as Logstash input, how can I do this? --- ## [Best way to add metadata stored on external server?](https://discuss.elastic.co/t/best-way-to-add-metadata-stored-on-external-server/162679)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 0\ **Last updated:** [January 2, 2019, 3:29pm UTC](https://discuss.elastic.co/t/best-way-to-add-metadata-stored-on-external-server/162679 "2019-01-02T15:29:56Z")
Hi, On our nightly build systems we have currently multiple directories prepared for installation. It looks like this: /nightly/test\_1 /nightly/test\_2 /nightly/test\_3 After the automatic deployment I have additional m… --- ## [How to index a custom field that has been added to logstash.conf](https://discuss.elastic.co/t/how-to-index-a-custom-field-that-has-been-added-to-logstash-conf/162625)
**Author:** [@baidurjya](https://discuss.elastic.co/u/baidurjya)\ **Replies:** 8\ **Last updated:** [January 2, 2019, 12:10pm UTC](https://discuss.elastic.co/t/how-to-index-a-custom-field-that-has-been-added-to-logstash-conf/162625 "2019-01-02T12:10:59Z")
I've added a custom field to the logstash.conf , it is shown in kibana UI Console but the field "actual\_host\_is" is not indexed. how to index a custom field added to logstash.conf mutate{ add\_field =\> { "actual\_host\_i… --- ## [How to load data from large parquet files of 500M records to ES using logstash](https://discuss.elastic.co/t/how-to-load-data-from-large-parquet-files-of-500m-records-to-es-using-logstash/162640)
**Author:** [@Apple23](https://discuss.elastic.co/u/Apple23)\ **Replies:** 3\ **Last updated:** [January 2, 2019, 12:12pm UTC](https://discuss.elastic.co/t/how-to-load-data-from-large-parquet-files-of-500m-records-to-es-using-logstash/162640 "2019-01-02T12:12:08Z")
how to load data from large parquet files of 500M records to ES using logstash --- ## [Logstash metadata](https://discuss.elastic.co/t/logstash-metadata/162632)
**Author:** [@dinesh1](https://discuss.elastic.co/u/dinesh1)\ **Replies:** 0\ **Last updated:** [January 2, 2019, 10:20am UTC](https://discuss.elastic.co/t/logstash-metadata/162632 "2019-01-02T10:20:44Z")
Hi Team Iam using ELK stack 5.6 My logstash configuration is shown below input { file { path =\> \[ "/var/logs/\*" \] } } output { elasticsearch { hosts =\> localhost index =\> "gr-%{+YYYY.MM.dd}" } stdout { code… --- ## [Unable to create nested json output (aggregated) from csv input](https://discuss.elastic.co/t/unable-to-create-nested-json-output-aggregated-from-csv-input/162630)
**Author:** [@Deepak\_Singhal](https://discuss.elastic.co/u/Deepak_Singhal)\ **Replies:** 0\ **Last updated:** [January 2, 2019, 10:05am UTC](https://discuss.elastic.co/t/unable-to-create-nested-json-output-aggregated-from-csv-input/162630 "2019-01-02T10:05:56Z")
Issue I am facing is I need aggregation of Csv inputs on ID, and it contains multiple nesting , I am able to perform single nesting , but on further nesting, I am not able to write correct syntax. PFA input, output I am… --- ## [Please share the logstash filter file for Weblogic/Soa/fussion Middleware](https://discuss.elastic.co/t/please-share-the-logstash-filter-file-for-weblogic-soa-fussion-middleware/162506)
**Author:** [@Sanchit\_Gupta1](https://discuss.elastic.co/u/Sanchit_Gupta1)\ **Replies:** 0\ **Last updated:** [December 31, 2018, 10:54am UTC](https://discuss.elastic.co/t/please-share-the-logstash-filter-file-for-weblogic-soa-fussion-middleware/162506 "2018-12-31T10:54:06Z")
Hi, From the longtime, I am trying to work on Logstash and Kibana but my Logstash filter file not configured properly. Can anyone share his Weblogic logstash filter file. I was using below one but it's not creating th… --- ## [Logstash errors out when running command for config](https://discuss.elastic.co/t/logstash-errors-out-when-running-command-for-config/162541)
**Author:** [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Replies:** 3\ **Last updated:** [January 1, 2019, 3:47pm UTC](https://discuss.elastic.co/t/logstash-errors-out-when-running-command-for-config/162541 "2019-01-01T15:47:54Z")
Hello and happy new year everyone. I am running ELK on Ubuntu 16.04 and I have everything installed correctly it seems except for when I try to execute my commands and then thats when stuff goes down hill real quick. T… --- ## [Issue with salesforce plugin](https://discuss.elastic.co/t/issue-with-salesforce-plugin/162606)
**Author:** [@Surasha](https://discuss.elastic.co/u/Surasha)\ **Replies:** 0\ **Last updated:** [January 2, 2019, 7:51am UTC](https://discuss.elastic.co/t/issue-with-salesforce-plugin/162606 "2019-01-02T07:51:18Z")
I'm getting this error when I run my configuration file: //error=\>"bad URI(is not URI?): /services/data/v26.0/sobjects/Reach Out/describe" I've used salesforce input plugin and file outpt plugin. --- ## [Change Date format from yyyy-MM-dd hh:mm:ss to yyyy-MM-dd](https://discuss.elastic.co/t/change-date-format-from-yyyy-mm-dd-hhss-to-yyyy-mm-dd/162306)
**Author:** [@vsingh](https://discuss.elastic.co/u/vsingh)\ **Replies:** 3\ **Last updated:** [January 2, 2019, 7:28am UTC](https://discuss.elastic.co/t/change-date-format-from-yyyy-mm-dd-hhss-to-yyyy-mm-dd/162306 "2019-01-02T07:28:29Z")
Hello there, I am using logstash logstash-6.4.0. I have mysql req\_date column which format is yyyy-MM-dd hh:mm:ss. i want to change this format into yyyy-MM-dd. Input (mysql req\_date column): 2017-05-31 00:00:00.000 … --- ## [Kv values include spaces](https://discuss.elastic.co/t/kv-values-include-spaces/162568)
**Author:** [@nsheridan](https://discuss.elastic.co/u/nsheridan)\ **Replies:** 1\ **Last updated:** [January 2, 2019, 6:15am UTC](https://discuss.elastic.co/t/kv-values-include-spaces/162568 "2019-01-02T06:15:18Z")
if I have this log message: key1=foo key2=bar key3=foo bar How can I use kv filters to represent the above as: "key1" =\> "foo", "key2" =\> "bar", "key3" =\> "foo bar" As if I change the space delimeter it seems… --- ## [Grok escape tilde and vertical bars in logfile](https://discuss.elastic.co/t/grok-escape-tilde-and-vertical-bars-in-logfile/162574)
**Author:** [@Mahmoud\_Nawito](https://discuss.elastic.co/u/Mahmoud_Nawito)\ **Replies:** 2\ **Last updated:** [January 1, 2019, 11:38pm UTC](https://discuss.elastic.co/t/grok-escape-tilde-and-vertical-bars-in-logfile/162574 "2019-01-01T23:38:24Z")
HI, I'm a elk stack newbie, so please forgive ignorance, I want to extract the data from a logfile which separates most of data using the tilde& vertical bars like below BOO SF55KCK2O12RVDB~|~6635~|~1~|~Music Instrumen… --- ## [Log folder and logstash-plain.log file not showing in Logstash 5.3.2](https://discuss.elastic.co/t/log-folder-and-logstash-plain-log-file-not-showing-in-logstash-5-3-2/162542)
**Author:** [@surajbs](https://discuss.elastic.co/u/surajbs)\ **Replies:** 0\ **Last updated:** [December 31, 2018, 10:33pm UTC](https://discuss.elastic.co/t/log-folder-and-logstash-plain-log-file-not-showing-in-logstash-5-3-2/162542 "2018-12-31T22:33:04Z")
HI, Log folder and logstash-plain.log file not showing in Logstash 5.3.2. When I used logstash one year ago, it used to log all the events/logs like INFO WARN ERROR into logstash-plain-log file, but this file does not … --- ## [How to write the condition for logstash conf with multiple Indice](https://discuss.elastic.co/t/how-to-write-the-condition-for-logstash-conf-with-multiple-indice/162540)
**Author:** [@Thang\_Cong\_Tu](https://discuss.elastic.co/u/Thang_Cong_Tu)\ **Replies:** 0\ **Last updated:** [December 31, 2018, 9:52pm UTC](https://discuss.elastic.co/t/how-to-write-the-condition-for-logstash-conf-with-multiple-indice/162540 "2018-12-31T21:52:25Z")
Hello everyone, I would like creating 2 separate indices to receive logs from 2 linux servers through nput beat port 5044 . My logstash config file is : input { beats{ port =\> 5044 } } output { if \[beat.name\] =… --- ## [Development versions of logstash Docker images](https://discuss.elastic.co/t/development-versions-of-logstash-docker-images/162277)
**Author:** [@skierx](https://discuss.elastic.co/u/skierx)\ **Replies:** 1\ **Last updated:** [December 30, 2018, 10:46pm UTC](https://discuss.elastic.co/t/development-versions-of-logstash-docker-images/162277 "2018-12-30T22:46:29Z")
Are there any development versions of the Logstash Docker images hosted on the Elastic Docker repository? By development versions I mean the resulting image that would be generated if you build from the source Dockerfile… --- ## [Plugin schedules get mixed](https://discuss.elastic.co/t/plugin-schedules-get-mixed/162451)
**Author:** [@nandrik](https://discuss.elastic.co/u/nandrik)\ **Replies:** 1\ **Last updated:** [December 30, 2018, 6:29pm UTC](https://discuss.elastic.co/t/plugin-schedules-get-mixed/162451 "2018-12-30T18:29:20Z")
I have schedule parameters for cron jobs in the input configurations of http\_poller and jdbc plugins: http\_poller runs at the mark of the hour, i.e. schedule =\> { cron =\> "0 \* \* \* \*"} jdbc runs every 2 minutes, i.e. s… --- ## [Import to SQL server](https://discuss.elastic.co/t/import-to-sql-server/162123)
**Author:** [@saeed](https://discuss.elastic.co/u/saeed)\ **Replies:** 5\ **Last updated:** [December 30, 2018, 10:24am UTC](https://discuss.elastic.co/t/import-to-sql-server/162123 "2018-12-30T10:24:46Z")
Is it possible to import data from file into sql server and use logstash? --- ## [No Custom Index in Kibana](https://discuss.elastic.co/t/no-custom-index-in-kibana/162433)
**Author:** [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Replies:** 0\ **Last updated:** [December 30, 2018, 4:24am UTC](https://discuss.elastic.co/t/no-custom-index-in-kibana/162433 "2018-12-30T04:24:53Z")
Question when I run this config file input { file { path =\> "Z:\\output\\not\*.csv" start position =\> "beginning" sincedb\_path =\> "NUL" } } filter { csv { separator =\> "," } } output { elasticsearch { host =\> … --- ## [Logstash Errors out on executing .conf file](https://discuss.elastic.co/t/logstash-errors-out-on-executing-conf-file/162426)
**Author:** [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Replies:** 1\ **Last updated:** [December 30, 2018, 12:36am UTC](https://discuss.elastic.co/t/logstash-errors-out-on-executing-conf-file/162426 "2018-12-30T00:36:03Z")
Hello everyone I have a question I keep getting this error. This is fresh install on a windows machine and all services are running. My config file is correct because you guys helped me build it. What could I be doing w… --- ## [Logstash use https](https://discuss.elastic.co/t/logstash-use-https/162403)
**Author:** [@Yerikshu](https://discuss.elastic.co/u/Yerikshu)\ **Replies:** 1\ **Last updated:** [December 29, 2018, 12:02pm UTC](https://discuss.elastic.co/t/logstash-use-https/162403 "2018-12-29T12:02:25Z")
we need to use logstash in https, and this's our configucation: input { http{ host =\> "0.0.0.0" port =\> 8080 additional\_codecs =\> {"application/json"=\>"json"} codec =\> "plain" threads =\> 4 ssl =\> true ssl\_certi… --- ## [Logstash 6.4.2 stopped processing after 45 days](https://discuss.elastic.co/t/logstash-6-4-2-stopped-processing-after-45-days/161799)
**Author:** [@Santosh\_Kumar\_Arjuna](https://discuss.elastic.co/u/Santosh_Kumar_Arjuna)\ **Replies:** 1\ **Last updated:** [December 29, 2018, 10:51am UTC](https://discuss.elastic.co/t/logstash-6-4-2-stopped-processing-after-45-days/161799 "2018-12-29T10:51:15Z")
Logstash 6.4.2 stopped processing after 45 days.. But the status of logstash shows still active and running. After restarting that, it started receiving the logs from Filebeat 6.4.2 version. We have persistent queues in … --- ## [LogStash Wrong Last Run Data Set](https://discuss.elastic.co/t/logstash-wrong-last-run-data-set/162395)
**Author:** [@Ahmed\_Gamal1](https://discuss.elastic.co/u/Ahmed_Gamal1)\ **Replies:** 0\ **Last updated:** [December 29, 2018, 6:10am UTC](https://discuss.elastic.co/t/logstash-wrong-last-run-data-set/162395 "2018-12-29T06:10:33Z")
Hi Dears, logstash showed me a strange behaviour as it suddenly changed its sql\_last\_value to a previous date, have anyone faced such issue or can give a hint. Below part of logstash log \[2018-12-29T04:48:14,906\]\[INFO … --- ## [Drop message based on lookup table content?](https://discuss.elastic.co/t/drop-message-based-on-lookup-table-content/162352)
**Author:** [@andre22](https://discuss.elastic.co/u/andre22)\ **Replies:** 1\ **Last updated:** [December 28, 2018, 9:49pm UTC](https://discuss.elastic.co/t/drop-message-based-on-lookup-table-content/162352 "2018-12-28T21:49:16Z")
Hi all, I want to drop some messages in Logstash when they contain specific strings, which I would like to maintain in a separate file so I can edit this as needed. An example: the field "syslog\_message" contains (or eq… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=624) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=626)