# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=143 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 144 --- ## [Why logstash csv filter skip\_header param don't work](https://discuss.elastic.co/t/why-logstash-csv-filter-skip-header-param-dont-work/303894)
**Author:** [@zhanghao116560](https://discuss.elastic.co/u/zhanghao116560)\ **Replies:** 2\ **Last updated:** [May 4, 2022, 6:18am UTC](https://discuss.elastic.co/t/why-logstash-csv-filter-skip-header-param-dont-work/303894 "2022-05-04T06:18:35Z")
Using the Logstash CSV filter to parse my CSV file, I set the skip\_header parameter to true. I wanted the first line not to be printed, but the actual result program didn't skip the first line。 here is my config: input… --- ## [Logstash mutate not working after kv filter applied](https://discuss.elastic.co/t/logstash-mutate-not-working-after-kv-filter-applied/303849)
**Author:** [@splitmessage88](https://discuss.elastic.co/u/splitmessage88)\ **Replies:** 6\ **Last updated:** [May 4, 2022, 5:56am UTC](https://discuss.elastic.co/t/logstash-mutate-not-working-after-kv-filter-applied/303849 "2022-05-04T05:56:19Z")
Hi, I created a filter for cisco syslog, input source cisco FTD. All the fields are parsing correctly but I can't rename/remove fields with mutate after I used the kv {} filter. Is it possible to use mutate after kv … --- ## [Logstash plugin 0.5 fraction on number field](https://discuss.elastic.co/t/logstash-plugin-0-5-fraction-on-number-field/303890)
**Author:** [@Altiano\_Gerung](https://discuss.elastic.co/u/Altiano_Gerung)\ **Replies:** 1\ **Last updated:** [May 4, 2022, 3:27am UTC](https://discuss.elastic.co/t/logstash-plugin-0-5-fraction-on-number-field/303890 "2022-05-04T03:27:46Z")
With logstash google pub/sub output plugin.. How do you set : delay\_threshold\_secs =\> "0.5" ? If I set to 1.x or 2.x, it'll work, but for 0.x it'll never work, so I guess it's just a matter of syntax error.. --- ## [Logstash filter out log lines](https://discuss.elastic.co/t/logstash-filter-out-log-lines/302230)
**Author:** [@shivendra95](https://discuss.elastic.co/u/shivendra95)\ **Replies:** 5\ **Last updated:** [May 4, 2022, 3:11am UTC](https://discuss.elastic.co/t/logstash-filter-out-log-lines/302230 "2022-05-04T03:11:35Z")
Hi, I have json logs that I want to filter based on the message field. {"@timestamp":"2022-04-05T01:20:50.917+00:00","severity":"INFO","service":"service","pid":"18245","thread":"http-nio-8102-exec-1","class":"class","… --- ## [Logstash- /var/log/logstash - has no files](https://discuss.elastic.co/t/logstash-var-log-logstash-has-no-files/303780)
**Author:** [@gurumu](https://discuss.elastic.co/u/gurumu)\ **Replies:** 7\ **Last updated:** [May 3, 2022, 8:46pm UTC](https://discuss.elastic.co/t/logstash-var-log-logstash-has-no-files/303780 "2022-05-03T20:46:35Z")
Hello- I have installed Elasticsearch, Kibana and Logstash version 8.1.0 on my ubuntu VM. I am trying to ingest auth.log into ES by passing it through logstash. attached is the configuration file screenshots. I am not se… --- ## [Logstash Permission denied - NUL error](https://discuss.elastic.co/t/logstash-permission-denied-nul-error/303686)
**Author:** [@Ersan](https://discuss.elastic.co/u/Ersan)\ **Replies:** 3\ **Last updated:** [May 3, 2022, 8:32pm UTC](https://discuss.elastic.co/t/logstash-permission-denied-nul-error/303686 "2022-05-03T20:32:57Z")
Hello I am new to EleasticSearch. I try to run logstash on Windows for the first time with different instrutions/Java versions/Logtash versions but always get the same error. I checked out the forms too but could not f… --- ## [Logstash Conf | Extracting Filename from Path](https://discuss.elastic.co/t/logstash-conf-extracting-filename-from-path/303818)
**Author:** [@srii](https://discuss.elastic.co/u/srii)\ **Replies:** 2\ **Last updated:** [May 3, 2022, 4:36pm UTC](https://discuss.elastic.co/t/logstash-conf-extracting-filename-from-path/303818 "2022-05-03T16:36:14Z")
I am trying to setup Logstash to feed Elasticsearch. In course, I've created the following conf file that seem to work nicely: input { beats { port =\> 5044 } file { path =\> "C:/f1/f2/Logs/f3/LocalHost#b… --- ## [Mount ExistingClaim in Logstash install with Helm](https://discuss.elastic.co/t/mount-existingclaim-in-logstash-install-with-helm/303784)
**Author:** [@Dallas\_Toth](https://discuss.elastic.co/u/Dallas_Toth)\ **Replies:** 1\ **Last updated:** [May 3, 2022, 2:14pm UTC](https://discuss.elastic.co/t/mount-existingclaim-in-logstash-install-with-helm/303784 "2022-05-03T14:14:03Z")
I am using the official Helm chart for Logstash. I want to mount a volume using an existingClaim that has some dictionaries that will be used for translate filters across multiple Logstash instances. How do I bring in … --- ## [Manage "new" index templates with Logstash](https://discuss.elastic.co/t/manage-new-index-templates-with-logstash/303831)
**Author:** [@frank\_esg](https://discuss.elastic.co/u/frank_esg)\ **Replies:** 0\ **Last updated:** [May 3, 2022, 12:41pm UTC](https://discuss.elastic.co/t/manage-new-index-templates-with-logstash/303831 "2022-05-03T12:41:22Z")
Hi, whe´re currently using logstash to process all of our messages and we are using the "manage template" function of the Elasticsearch output. Now i' m working on the migration of our legacy templates to the new (comp… --- ## [Logstash logs want in proper format](https://discuss.elastic.co/t/logstash-logs-want-in-proper-format/303379)
**Author:** [@dharminfadia](https://discuss.elastic.co/u/dharminfadia)\ **Replies:** 4\ **Last updated:** [May 3, 2022, 8:25am UTC](https://discuss.elastic.co/t/logstash-logs-want-in-proper-format/303379 "2022-05-03T08:25:36Z")
Hello Everyone I am recieved message from some alerts and data is look like as belwo I want to set in proper looks can you please help any one ? type=update&date\_time=2022-04-27T04%3A11%3A06-05%3A00&initiated\_from=admi… --- ## [Monitor logstash](https://discuss.elastic.co/t/monitor-logstash/303799)
**Author:** [@bleedgreen](https://discuss.elastic.co/u/bleedgreen)\ **Replies:** 0\ **Last updated:** [May 3, 2022, 4:32am UTC](https://discuss.elastic.co/t/monitor-logstash/303799 "2022-05-03T04:32:35Z")
Hi All, I have a logstash instance with input plugin with different input plugin (kafka,syslog, file) I want to monitor and detect when and event cannot be published to a target . Say for example a connection with sysl… --- ## [Http input does not result in the expected number of documents](https://discuss.elastic.co/t/http-input-does-not-result-in-the-expected-number-of-documents/303797)
**Author:** [@hughjarse](https://discuss.elastic.co/u/hughjarse)\ **Replies:** 0\ **Last updated:** [May 3, 2022, 3:32am UTC](https://discuss.elastic.co/t/http-input-does-not-result-in-the-expected-number-of-documents/303797 "2022-05-03T03:32:49Z")
Hi. I would like ideas about what Logstash functionality I can use to troubleshoot a problem I'm experiencing with the http input plugin. Problem My http input does not seem to receive?/process? the expected number of e… --- ## [GeoIP data not creating "location" field and duplicating data](https://discuss.elastic.co/t/geoip-data-not-creating-location-field-and-duplicating-data/303612)
**Author:** [@joshn](https://discuss.elastic.co/u/joshn)\ **Replies:** 7\ **Last updated:** [May 2, 2022, 8:29pm UTC](https://discuss.elastic.co/t/geoip-data-not-creating-location-field-and-duplicating-data/303612 "2022-05-02T20:29:39Z")
I'm ingesting logs from my firewall, and as part of that I thought it would be nice to look at geoip data. geoip { add\_tag =\> \[ "GeoIP" \] source =\> "src\_ip" } geoip { … --- ## [Logstash fine tuning for ingesting more events (s3 input)](https://discuss.elastic.co/t/logstash-fine-tuning-for-ingesting-more-events-s3-input/303681)
**Author:** [@antonisnyc94](https://discuss.elastic.co/u/antonisnyc94)\ **Replies:** 3\ **Last updated:** [May 2, 2022, 7:48pm UTC](https://discuss.elastic.co/t/logstash-fine-tuning-for-ingesting-more-events-s3-input/303681 "2022-05-02T19:48:46Z")
Hello, We sending events for vpc flowlogs from multiple AWS accounts into a central s3 bucket and due to the large number of events we are always 5-6 days behind in Elasticsearch. I already set the batch.size to 6000 a… --- ## [Clean method for adding field when the values may be missing?](https://discuss.elastic.co/t/clean-method-for-adding-field-when-the-values-may-be-missing/303769)
**Author:** [@jbrowe](https://discuss.elastic.co/u/jbrowe)\ **Replies:** 0\ **Last updated:** [May 2, 2022, 7:42pm UTC](https://discuss.elastic.co/t/clean-method-for-adding-field-when-the-values-may-be-missing/303769 "2022-05-02T19:42:28Z")
I have logs with variable field names. For example: {"field\_one":"first", "field\_two":"second"} ("field\_two":"second", "field\_three": "third"} To get the correct index mapping, I currently I use: mutate { add\_fi… --- ## [Testing Logs in Logstash |Automation with Ansible](https://discuss.elastic.co/t/testing-logs-in-logstash-automation-with-ansible/303768)
**Author:** [@Amanda\_Ruzza](https://discuss.elastic.co/u/Amanda_Ruzza)\ **Replies:** 0\ **Last updated:** [May 2, 2022, 7:24pm UTC](https://discuss.elastic.co/t/testing-logs-in-logstash-automation-with-ansible/303768 "2022-05-02T19:24:32Z")
Dear all, My team is currently working on deploying an ELK stack on AWS using Ansible and GitHub Actions. My question is: Is there a way to test that the the Log files sent from MetricBeats are being properly collecte… --- ## [Logstash best practices for multiple sources](https://discuss.elastic.co/t/logstash-best-practices-for-multiple-sources/303722)
**Author:** [@fropa](https://discuss.elastic.co/u/fropa)\ **Replies:** 1\ **Last updated:** [May 2, 2022, 4:27pm UTC](https://discuss.elastic.co/t/logstash-best-practices-for-multiple-sources/303722 "2022-05-02T16:27:21Z")
Hi folks, I'm starting to use ELK, I've multiple source servers, some of them have also multiple logs to send. Now I'm trying to set up logstash sample configuration to just receive and send logs to the elasticsearch I… --- ## [Logstash not receiving messages when started via docker-compose (Docker is ok)](https://discuss.elastic.co/t/logstash-not-receiving-messages-when-started-via-docker-compose-docker-is-ok/303645)
**Author:** [@alex\_london](https://discuss.elastic.co/u/alex_london)\ **Replies:** 2\ **Last updated:** [May 2, 2022, 1:41pm UTC](https://discuss.elastic.co/t/logstash-not-receiving-messages-when-started-via-docker-compose-docker-is-ok/303645 "2022-05-02T13:41:30Z")
I've encountered strange behaviour when running Logstash via docker-compose which I have been unable to figure out. Logstash starts (seemingly) fine, but it does not receive any events from Beats or other sources. Events… --- ## [Don't know how to handle \`Java::JavaLang::IllegalStateException\` for \`PipelineAction](https://discuss.elastic.co/t/dont-know-how-to-handle-java-illegalstateexception-for-pipelineaction/303735)
**Author:** [@rinki\_kumari](https://discuss.elastic.co/u/rinki_kumari)\ **Replies:** 1\ **Last updated:** [May 2, 2022, 10:08am UTC](https://discuss.elastic.co/t/dont-know-how-to-handle-java-illegalstateexception-for-pipelineaction/303735 "2022-05-02T10:08:47Z")
input { http\_poller { urls =\> { test =\> { method =\> get url =\> "http://172.24.105.27:8092/" headers =\> { Accept =\> "application/json" Authorization =\> "Basic QXBp" } } } } } o… --- ## [Need help in file.conf for Logstash - Salesforce pipeline](https://discuss.elastic.co/t/need-help-in-file-conf-for-logstash-salesforce-pipeline/303720)
**Author:** [@Simone1](https://discuss.elastic.co/u/Simone1)\ **Replies:** 0\ **Last updated:** [May 2, 2022, 8:21am UTC](https://discuss.elastic.co/t/need-help-in-file-conf-for-logstash-salesforce-pipeline/303720 "2022-05-02T08:21:57Z")
I'm trying to configure a pipeline to send Salesforce's data to Elasticsearch using Logstash. I created the a Connected App and flagged the OAuth 2.0 and i've put the callback url. Now, i have the client secret and the… --- ## [How to get IP address of collectd client](https://discuss.elastic.co/t/how-to-get-ip-address-of-collectd-client/303674)
**Author:** [@Rostam](https://discuss.elastic.co/u/Rostam)\ **Replies:** 4\ **Last updated:** [May 1, 2022, 5:25pm UTC](https://discuss.elastic.co/t/how-to-get-ip-address-of-collectd-client/303674 "2022-05-01T17:25:36Z")
Hello I have configured collectd on all my devices (running Linux). The configuration is: /etc/collectd.conf LoadPlugin cpu LoadPlugin interface LoadPlugin df \ server "10.2.9.35" "5555" /Plugin My l… --- ## [Grokparsefailure ... but it's all working and logs are being parsed...?](https://discuss.elastic.co/t/grokparsefailure-but-its-all-working-and-logs-are-being-parsed/303614)
**Author:** [@joshn](https://discuss.elastic.co/u/joshn)\ **Replies:** 7\ **Last updated:** [May 1, 2022, 4:54pm UTC](https://discuss.elastic.co/t/grokparsefailure-but-its-all-working-and-logs-are-being-parsed/303614 "2022-05-01T16:54:29Z")
I'm seeing pretty much all of my logs in Elastic/Kibana tagged with the following tags: \_grokparsefailure\_sysloginput, \_grokparsefailure, \_geoip\_lookup\_failure Which is odd, as they are being processed just fine. Exam… --- ## [Logstash to elastic search TCP connection error](https://discuss.elastic.co/t/logstash-to-elastic-search-tcp-connection-error/303624)
**Author:** [@newelastic](https://discuss.elastic.co/u/newelastic)\ **Replies:** 5\ **Last updated:** [April 29, 2022, 7:33pm UTC](https://discuss.elastic.co/t/logstash-to-elastic-search-tcp-connection-error/303624 "2022-04-29T19:33:52Z")
Hi, I'm trying to connect to Elasticsearch using logstash and running into an TCP connection error. Below is my input plugin for logstash input { elasticsearch { hosts =\> \["https://esipAddress:9200"\] index =\> "tes… --- ## [Logstash-filter-http Post Body error](https://discuss.elastic.co/t/logstash-filter-http-post-body-error/303532)
**Author:** [@zurb3](https://discuss.elastic.co/u/zurb3)\ **Replies:** 9\ **Last updated:** [April 29, 2022, 3:24pm UTC](https://discuss.elastic.co/t/logstash-filter-http-post-body-error/303532 "2022-04-29T15:24:49Z")
I'm trying to get some kind of data enrichment using an API call to Abuse.ch as documents come into the pipeline. The logstash-filter-http plugin seems like the right answer, however it isn't working for what I need it t… --- ## [Grok Filter..so close... yet so far](https://discuss.elastic.co/t/grok-filter-so-close-yet-so-far/303592)
**Author:** [@joshn](https://discuss.elastic.co/u/joshn)\ **Replies:** 2\ **Last updated:** [April 29, 2022, 2:13pm UTC](https://discuss.elastic.co/t/grok-filter-so-close-yet-so-far/303592 "2022-04-29T14:13:16Z")
I'm using this website to debug my Grok code: https://grokdebug.herokuapp.com/ Two different logs I'm trying to ingest: 1: \<134\>1 1651225979.448642514 EMEA\_ISP ip\_flow\_end src=192.168.15.6 dst=8.8.4.4 protocol=udp spo… --- ## [I want Message values as feild using logstash](https://discuss.elastic.co/t/i-want-message-values-as-feild-using-logstash/303519)
**Author:** [@dharminfadia](https://discuss.elastic.co/u/dharminfadia)\ **Replies:** 9\ **Last updated:** [April 29, 2022, 11:59am UTC](https://discuss.elastic.co/t/i-want-message-values-as-feild-using-logstash/303519 "2022-04-29T11:59:32Z")
Hello Every One Following is my message I want to split message value and add as field for filter and visulize how can I achive that please help any one message url=&type=click&date\_time=2013-01-01+12:00:00&initiated… --- ## [Confused on LS outputs and data not going to right index](https://discuss.elastic.co/t/confused-on-ls-outputs-and-data-not-going-to-right-index/303498)
**Author:** [@joshn](https://discuss.elastic.co/u/joshn)\ **Replies:** 4\ **Last updated:** [April 29, 2022, 10:26am UTC](https://discuss.elastic.co/t/confused-on-ls-outputs-and-data-not-going-to-right-index/303498 "2022-04-29T10:26:51Z")
Hey there, I'm not understanding the "output" section of my config for Logstash \> Elasticsearch. I've been following this guide: How To Install Elasticsearch, Logstash, and Kibana (Elastic Stack) on Ubuntu 22.04 | Digi… --- ## [Splitting json file](https://discuss.elastic.co/t/splitting-json-file/303488)
**Author:** [@javidr](https://discuss.elastic.co/u/javidr)\ **Replies:** 10\ **Last updated:** [April 29, 2022, 8:35am UTC](https://discuss.elastic.co/t/splitting-json-file/303488 "2022-04-29T08:35:43Z")
Hi I am totally new in logstash, and i am having some issues when splitting a json file. My file is super easy { "data": \[ { "name": "name1" }, { "name": "name2" … --- ## [Logstash filter only particular fields and index those fields](https://discuss.elastic.co/t/logstash-filter-only-particular-fields-and-index-those-fields/303470)
**Author:** [@venkatesh\_prasanth](https://discuss.elastic.co/u/venkatesh_prasanth)\ **Replies:** 7\ **Last updated:** [April 29, 2022, 5:08am UTC](https://discuss.elastic.co/t/logstash-filter-only-particular-fields-and-index-those-fields/303470 "2022-04-29T05:08:15Z")
Hi, I have heartbeat which pushed the data to 5044 through which logstash consumes. I want to index only if monitor.status is down. I am using tag heartbeat for differentiating the various beat input. My data: { "\_i… --- ## [After update to 8.1.3 Logstash in cluster no longer functioning](https://discuss.elastic.co/t/after-update-to-8-1-3-logstash-in-cluster-no-longer-functioning/303439)
**Author:** [@hueyg](https://discuss.elastic.co/u/hueyg)\ **Replies:** 2\ **Last updated:** [April 28, 2022, 7:35pm UTC](https://discuss.elastic.co/t/after-update-to-8-1-3-logstash-in-cluster-no-longer-functioning/303439 "2022-04-28T19:35:48Z")
Running an 8.1.3 cluster with all components updated. Filebeat agent is also latest. This particular host has multiple log files that I was shipping, then creating a different index on each logfile based on the "type" … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=142) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=144)