# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=476 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 477 --- ## [Logstash http input not handling multiple headers with same key](https://discuss.elastic.co/t/logstash-http-input-not-handling-multiple-headers-with-same-key/199828)
**Author:** [@jacowessels](https://discuss.elastic.co/u/jacowessels)\ **Replies:** 0\ **Last updated:** [September 17, 2019, 1:22pm UTC](https://discuss.elastic.co/t/logstash-http-input-not-handling-multiple-headers-with-same-key/199828 "2019-09-17T13:22:18Z")
Good day, I am running into an issue with the HTTP input filter. We would like the full list of X-Forwarder-For IP's from the front-end to our device accessible in the header map. We are receiving HTTP messages from an … --- ## [Logstash: CSV filter pattern-based field name detection from header row](https://discuss.elastic.co/t/logstash-csv-filter-pattern-based-field-name-detection-from-header-row/198266)
**Author:** [@barnetta](https://discuss.elastic.co/u/barnetta)\ **Replies:** 5\ **Last updated:** [September 17, 2019, 1:18pm UTC](https://discuss.elastic.co/t/logstash-csv-filter-pattern-based-field-name-detection-from-header-row/198266 "2019-09-17T13:18:46Z")
Summary: current understanding is that CSV file -\> FileBeat -\> Logstash -\> Logstash CSV filter cannot reliably detect field names from column headings within CSV; workaround or alternative needed. I have read the follow… --- ## [Downcasing and removing special characters via file beats](https://discuss.elastic.co/t/downcasing-and-removing-special-characters-via-file-beats/199823)
**Author:** [@Seetharaman\_K](https://discuss.elastic.co/u/Seetharaman_K)\ **Replies:** 0\ **Last updated:** [September 17, 2019, 1:05pm UTC](https://discuss.elastic.co/t/downcasing-and-removing-special-characters-via-file-beats/199823 "2019-09-17T13:05:52Z")
Hi All , in my previous environment logs generated had a field called "process name" whose value could possibily contain special chars and upper case letters , hence using logstash we downcased it and removed the spec… --- ## [Using conditional to find nil values](https://discuss.elastic.co/t/using-conditional-to-find-nil-values/199660)
**Author:** [@Hyperion0](https://discuss.elastic.co/u/Hyperion0)\ **Replies:** 2\ **Last updated:** [September 17, 2019, 11:56am UTC](https://discuss.elastic.co/t/using-conditional-to-find-nil-values/199660 "2019-09-17T11:56:14Z")
I'm writing a config file for logstash to read in a CSV file. Some of the fields are empty some of the time but when they are not empty they have to have data types assigned (integer, bool, etc...), and when I run log s… --- ## [(LoadError) no such file to load -- maxmind/db](https://discuss.elastic.co/t/loaderror-no-such-file-to-load-maxmind-db/199675)
**Author:** [@muraliv](https://discuss.elastic.co/u/muraliv)\ **Replies:** 1\ **Last updated:** [September 17, 2019, 11:26am UTC](https://discuss.elastic.co/t/loaderror-no-such-file-to-load-maxmind-db/199675 "2019-09-17T11:26:39Z")
I have created a custom maxmind db using perl. I have installed maxminddb module for ruby and was able to run a ruby program outside of logstash. When I am executing the same code from logstash, I get a load error as… --- ## [Ingestion of CEF by syslog](https://discuss.elastic.co/t/ingestion-of-cef-by-syslog/199811)
**Author:** [@franco.federico](https://discuss.elastic.co/u/franco.federico)\ **Replies:** 0\ **Last updated:** [September 17, 2019, 11:13am UTC](https://discuss.elastic.co/t/ingestion-of-cef-by-syslog/199811 "2019-09-17T11:13:07Z")
Hi I'm trying to configure logstash to ingest CEF file that I received by rsyslog and stored in a file in json format. I configure my logstash and the logstash.conf is like this (I try to implement ECS schema and in th… --- ## [No input plugin mongodb](https://discuss.elastic.co/t/no-input-plugin-mongodb/199797)
**Author:** [@Tulkas](https://discuss.elastic.co/u/Tulkas)\ **Replies:** 0\ **Last updated:** [September 17, 2019, 9:57am UTC](https://discuss.elastic.co/t/no-input-plugin-mongodb/199797 "2019-09-17T09:57:23Z")
Hi, New to ELK stack and my organisation is using MongoDB, currently investigating how best to index data from mongo to logstash, I have noticed there is not an official input plugin. Can anyone explain why? --- ## [Does logstash http output plugin has back pressure?](https://discuss.elastic.co/t/does-logstash-http-output-plugin-has-back-pressure/199795)
**Author:** [@Dileep\_Kumar\_Dora](https://discuss.elastic.co/u/Dileep_Kumar_Dora)\ **Replies:** 0\ **Last updated:** [September 17, 2019, 9:51am UTC](https://discuss.elastic.co/t/does-logstash-http-output-plugin-has-back-pressure/199795 "2019-09-17T09:51:07Z")
In the above discussion it's mentioned as Each output has its own retry mechanism - if an event is in an outputs retry queue then the event is still referenced and the JVM does not garbage collect it. Outputs like el… --- ## [How To Separate Time From Logs, Logstash Parsing Logs](https://discuss.elastic.co/t/how-to-separate-time-from-logs-logstash-parsing-logs/199784)
**Author:** [@Fedir\_Yatseta](https://discuss.elastic.co/u/Fedir_Yatseta)\ **Replies:** 0\ **Last updated:** [September 17, 2019, 8:17am UTC](https://discuss.elastic.co/t/how-to-separate-time-from-logs-logstash-parsing-logs/199784 "2019-09-17T08:17:24Z")
Hi everyone, I have logs with a view - "2019-09-17 04:06:01,294 INFO \[org.springframework.extensions.webscripts.AbstractRuntimeContainer\] \[localhost-startStop-1\] Initialised Spring Surf Container Web Script Container (… --- ## [Handling Error in Elasticsearch](https://discuss.elastic.co/t/handling-error-in-elasticsearch/199765)
**Author:** [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Replies:** 0\ **Last updated:** [September 17, 2019, 7:09am UTC](https://discuss.elastic.co/t/handling-error-in-elasticsearch/199765 "2019-09-17T07:09:30Z")
Hello, Can anyone help me in this? I am using jdbc input plugin to fetch data and create indexes in ES. If that fails in between, how we will be identify that? how to handle such scenarios in ES? Thanks and Regards, … --- ## [Logstash Parsing error for time field in correct JSON](https://discuss.elastic.co/t/logstash-parsing-error-for-time-field-in-correct-json/199537)
**Author:** [@aleksei.saiko](https://discuss.elastic.co/u/aleksei.saiko)\ **Replies:** 3\ **Last updated:** [September 17, 2019, 6:16am UTC](https://discuss.elastic.co/t/logstash-parsing-error-for-time-field-in-correct-json/199537 "2019-09-17T06:16:00Z")
Hey guys! I'm receiving Logstash parsing error for this JSON - { "time\_date": "2019-09-11T07:05:32+00:00", "client": "22.222.2.222", "host": "xxxx.com", "scheme": "https", "request\_method": "GET", "request\_uri": … --- ## [BUG: Logstash Elasticsearch output plugin writes to one index](https://discuss.elastic.co/t/bug-logstash-elasticsearch-output-plugin-writes-to-one-index/199466)
**Author:** [@EZprogramming](https://discuss.elastic.co/u/EZprogramming)\ **Replies:** 2\ **Last updated:** [September 16, 2019, 11:16pm UTC](https://discuss.elastic.co/t/bug-logstash-elasticsearch-output-plugin-writes-to-one-index/199466 "2019-09-16T23:16:17Z")
I have different Logstash configurations, for example, one for pulling AWS Cloudwatch logs, one for AWS Athena, etc. Therefore, I have created multiple configuration files, and in my pipeline.yml file, my configuration i… --- ## [Logstash not load conf.d directory](https://discuss.elastic.co/t/logstash-not-load-conf-d-directory/199733)
**Author:** [@mcoa](https://discuss.elastic.co/u/mcoa)\ **Replies:** 0\ **Last updated:** [September 16, 2019, 10:00pm UTC](https://discuss.elastic.co/t/logstash-not-load-conf-d-directory/199733 "2019-09-16T22:00:53Z")
Hello, I'm use Logstash v7.3.2-1 on Centos 7. When started the service from systemctl run fine but not load the /etc/logstash/conf.d/ directory . My filter work fine with the "manual" command: /usr/share/logstash/bin… --- ## [Access dynamic array](https://discuss.elastic.co/t/access-dynamic-array/199728)
**Author:** [@Guy\_E\_Bouchard](https://discuss.elastic.co/u/Guy_E_Bouchard)\ **Replies:** 1\ **Last updated:** [September 16, 2019, 9:57pm UTC](https://discuss.elastic.co/t/access-dynamic-array/199728 "2019-09-16T21:57:12Z")
Hi, I can't figure that out, I'm pretty sure I missed something I want to access the last line of this array. "logs" =\> \[ \[0\] 23:33:39 Sep 15, 2019 - Delivered "19-09-15\_23-33-37.127558/file\_01\_01\_01.xml",… --- ## [Logstash picking up JSON over TCP has non-json 1st line](https://discuss.elastic.co/t/logstash-picking-up-json-over-tcp-has-non-json-1st-line/199338)
**Author:** [@yungnvn](https://discuss.elastic.co/u/yungnvn)\ **Replies:** 2\ **Last updated:** [September 16, 2019, 9:48pm UTC](https://discuss.elastic.co/t/logstash-picking-up-json-over-tcp-has-non-json-1st-line/199338 "2019-09-16T21:48:02Z")
Hi, We are trying to send some metric data over TCP for AIX machines and we are using njmon (a variation of nmon which can send JSON formatted metrics). The way its currently set up, njmon sends the following: B@?=4zG?… --- ## [Split email address from existing field into new field without changing original field](https://discuss.elastic.co/t/split-email-address-from-existing-field-into-new-field-without-changing-original-field/199477)
**Author:** [@hueyg](https://discuss.elastic.co/u/hueyg)\ **Replies:** 4\ **Last updated:** [September 16, 2019, 8:10pm UTC](https://discuss.elastic.co/t/split-email-address-from-existing-field-into-new-field-without-changing-original-field/199477 "2019-09-16T20:10:23Z")
I have a working grok filter for a log file just like I want it. Currently the full email is currently parsed into a single field I call "user". I do not want to split this in the original grok unless I have to. I wou… --- ## [Logstash custom pattern not working](https://discuss.elastic.co/t/logstash-custom-pattern-not-working/199685)
**Author:** [@user3](https://discuss.elastic.co/u/user3)\ **Replies:** 5\ **Last updated:** [September 16, 2019, 8:06pm UTC](https://discuss.elastic.co/t/logstash-custom-pattern-not-working/199685 "2019-09-16T20:06:22Z")
Hello everyone, New to logstash so please bare with me. My logstash config is not working as desired with a pattern file using regex to pull for very specific text from a log message. From the specific log message, he… --- ## [Logstash Filter || Json Message to Split Fields](https://discuss.elastic.co/t/logstash-filter-json-message-to-split-fields/199713)
**Author:** [@deeps](https://discuss.elastic.co/u/deeps)\ **Replies:** 2\ **Last updated:** [September 16, 2019, 7:28pm UTC](https://discuss.elastic.co/t/logstash-filter-json-message-to-split-fields/199713 "2019-09-16T19:28:58Z")
Could someone suggest the best and easy way to split the message into separate fields? I have incoming messages in json format: ex: message {"scenarioName":"x","id":"x","type":"x","source":"x","transaction":"x","appId… --- ## [Newbie in Logstash need suggestion on setting up Logstash on GCP](https://discuss.elastic.co/t/newbie-in-logstash-need-suggestion-on-setting-up-logstash-on-gcp/199710)
**Author:** [@Rahul\_Goyal](https://discuss.elastic.co/u/Rahul_Goyal)\ **Replies:** 0\ **Last updated:** [September 16, 2019, 6:26pm UTC](https://discuss.elastic.co/t/newbie-in-logstash-need-suggestion-on-setting-up-logstash-on-gcp/199710 "2019-09-16T18:26:33Z")
Hey, I have just started using Logstash, and need some guidelines and best practices of running a Logstash pipeline on production. I am familiar with the pipeline.yml and logstash.yml and also have written a basic conf… --- ## [Logstash still running not terminated](https://discuss.elastic.co/t/logstash-still-running-not-terminated/199679)
**Author:** [@elk3](https://discuss.elastic.co/u/elk3)\ **Replies:** 1\ **Last updated:** [September 16, 2019, 6:15pm UTC](https://discuss.elastic.co/t/logstash-still-running-not-terminated/199679 "2019-09-16T18:15:54Z")
while executing logstash its started but still running not terminated ,its doesnot show pipeline terminated message ,but job completed --- ## [Read logs using the ELK stack from the remote server](https://discuss.elastic.co/t/read-logs-using-the-elk-stack-from-the-remote-server/199142)
**Author:** [@Konstantin\_Doncov](https://discuss.elastic.co/u/Konstantin_Doncov)\ **Replies:** 11\ **Last updated:** [September 16, 2019, 5:14pm UTC](https://discuss.elastic.co/t/read-logs-using-the-elk-stack-from-the-remote-server/199142 "2019-09-16T17:14:42Z")
I have successfully installed the Elastic stack(Elasticsearch + Logstash + Kibana) and can use it with Logback logs on the local machine. But now I need to read the logs from the remote server. I think there is no need t… --- ## [How to get the original log from winlogbeat?](https://discuss.elastic.co/t/how-to-get-the-original-log-from-winlogbeat/199697)
**Author:** [@Daniel\_Ben\_Hayoun](https://discuss.elastic.co/u/Daniel_Ben_Hayoun)\ **Replies:** 0\ **Last updated:** [September 16, 2019, 4:01pm UTC](https://discuss.elastic.co/t/how-to-get-the-original-log-from-winlogbeat/199697 "2019-09-16T16:01:25Z")
Hi, I'm using winlogbeat to publish event logs to logstash. I mean that the output from winlogbeat is for another machine that runs logstash. The plugin is : input { # tcp { # port =\> 12345 # } beats { port=\> … --- ## [Logstash pattern does not work after being upgraded to 7.3.0](https://discuss.elastic.co/t/logstash-pattern-does-not-work-after-being-upgraded-to-7-3-0/199681)
**Author:** [@zozo6015](https://discuss.elastic.co/u/zozo6015)\ **Replies:** 0\ **Last updated:** [September 16, 2019, 2:45pm UTC](https://discuss.elastic.co/t/logstash-pattern-does-not-work-after-being-upgraded-to-7-3-0/199681 "2019-09-16T14:45:41Z")
Hello, I am trying to separate java pattern into fields. I have the following pattern set up with custom pattern setup in order to be able to process multiline fields. However since I have upgraded logstash to 7.3.0 th… --- ## [Logstash update my existing Elasticsearch index data](https://discuss.elastic.co/t/logstash-update-my-existing-elasticsearch-index-data/199649)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 0\ **Last updated:** [September 16, 2019, 11:31am UTC](https://discuss.elastic.co/t/logstash-update-my-existing-elasticsearch-index-data/199649 "2019-09-16T11:31:04Z")
I am trying to update an existing data in my Elasticsearch Index. Im using the below Logstash configuration: input { jdbc { jdbc\_connection\_string =\> "jdbc:sqlserver://DB01:1433;databasename=mydb;integratedS… --- ## [Logstash not working if I provide http.host as IP address of my system](https://discuss.elastic.co/t/logstash-not-working-if-i-provide-http-host-as-ip-address-of-my-system/199663)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 0\ **Last updated:** [September 16, 2019, 12:54pm UTC](https://discuss.elastic.co/t/logstash-not-working-if-i-provide-http-host-as-ip-address-of-my-system/199663 "2019-09-16T12:54:20Z")
Hi All, Logstash is not working if I provide http.host as my system IP address in logstash.yml file. Logstash is working well if I provide my system name in logstash.yml file. What changes I need to do to make logstas… --- ## [How to find the filesize of a file via logstash](https://discuss.elastic.co/t/how-to-find-the-filesize-of-a-file-via-logstash/199613)
**Author:** [@basha5](https://discuss.elastic.co/u/basha5)\ **Replies:** 1\ **Last updated:** [September 16, 2019, 9:40am UTC](https://discuss.elastic.co/t/how-to-find-the-filesize-of-a-file-via-logstash/199613 "2019-09-16T09:40:17Z")
Suppose we are loading a file via logstash, how can we display the size of the file using a command in the config file itself? Thanks in advance. --- ## [Error: File paths must be absolute - never happened before! Pls advice](https://discuss.elastic.co/t/error-file-paths-must-be-absolute-never-happened-before-pls-advice/199443)
**Author:** [@Datakids](https://discuss.elastic.co/u/Datakids)\ **Replies:** 2\ **Last updated:** [September 16, 2019, 8:54am UTC](https://discuss.elastic.co/t/error-file-paths-must-be-absolute-never-happened-before-pls-advice/199443 "2019-09-16T08:54:41Z")
Hi, With Logstash 7.3.1 I just tried to add a few csv Demo Samples through logstash but now logstash is rejecting the File Path.. What ? This Never happened before... \[2019-09-13T15:24:59,168\]\[DEBUG\]\[logstash.javapip… --- ## [HIgh CPU usage without processing any data](https://discuss.elastic.co/t/high-cpu-usage-without-processing-any-data/199147)
**Author:** [@stefws](https://discuss.elastic.co/u/stefws)\ **Replies:** 2\ **Last updated:** [September 13, 2019, 8:56pm UTC](https://discuss.elastic.co/t/high-cpu-usage-without-processing-any-data/199147 "2019-09-13T20:56:30Z")
Trying to config a new logstash instance to handle syslog input, so prepared input, output and a dummy working filter as a starting template from another running instance, but not sending any data to input yet. Even so l… --- ## [How to get the AWS EC2/Account details using the logstash AWS plugins](https://discuss.elastic.co/t/how-to-get-the-aws-ec2-account-details-using-the-logstash-aws-plugins/199580)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 0\ **Last updated:** [September 16, 2019, 6:22am UTC](https://discuss.elastic.co/t/how-to-get-the-aws-ec2-account-details-using-the-logstash-aws-plugins/199580 "2019-09-16T06:22:18Z")
How to get the following AWS EC2/Account information using the logstash Cloudwatch plugin or with any other logstash AWS plugin. Account\_name Account\_number resource\_name VM Name Private IP Address Operating System … --- ## [Logstash filter plugin](https://discuss.elastic.co/t/logstash-filter-plugin/199359)
**Author:** [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)\ **Replies:** 3\ **Last updated:** [September 16, 2019, 5:37am UTC](https://discuss.elastic.co/t/logstash-filter-plugin/199359 "2019-09-16T05:37:27Z")
Hi I'm using logstash 7.0 . I'm having multiple events to filter. Instead of having one single config file, I prefer to have multiple filter config files for each event. As an example, I'm having a login event and I sto… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=475) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=477)