# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=375 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 376 --- ## [Dynamically injecting custom fields to logstash from filebeat](https://discuss.elastic.co/t/dynamically-injecting-custom-fields-to-logstash-from-filebeat/172104)
**Author:** [@sachhiiiinn](https://discuss.elastic.co/u/sachhiiiinn)\ **Replies:** 1\ **Last updated:** [March 13, 2019, 2:32pm UTC](https://discuss.elastic.co/t/dynamically-injecting-custom-fields-to-logstash-from-filebeat/172104 "2019-03-13T14:32:20Z")
I have set up filebeat to send logs from one of our servers to logstash. It's working fine but the message view in Kibana is pretty ugly and hard to debug as the message is pretty big(it's web service message,xml or json… --- ## [Filebeat prerequisite](https://discuss.elastic.co/t/filebeat-prerequisite/172054)
**Author:** [@gvsr1991](https://discuss.elastic.co/u/gvsr1991)\ **Replies:** 1\ **Last updated:** [March 13, 2019, 2:21pm UTC](https://discuss.elastic.co/t/filebeat-prerequisite/172054 "2019-03-13T14:21:33Z")
Hi Team, I am using Filebeat + ElaseticSearch+ Kibana to monitor the WSO2 EI logs. FIlebeat is installed in my WSO2 server and elasticsearch and kibana are in other server. Once I have started the filebeat, the CPU ut… --- ## [Metricbeat fails to index docker metadata](https://discuss.elastic.co/t/metricbeat-fails-to-index-docker-metadata/172155)
**Author:** [@Ivan.fernandez](https://discuss.elastic.co/u/Ivan.fernandez)\ **Replies:** 0\ **Last updated:** [March 13, 2019, 1:15pm UTC](https://discuss.elastic.co/t/metricbeat-fails-to-index-docker-metadata/172155 "2019-03-13T13:15:45Z")
Hi, I have deployed Metricbeat and Auditbeat (7.1.0-SNAPSHOT) as a daemonSet on a k8s cluster, the container Metricbeat module is reporting the info of an event related to Auditbeat daemonSet and it fails to store the d… --- ## [Creating indices](https://discuss.elastic.co/t/creating-indices/172060)
**Author:** [@Dv\_Thiyanesh](https://discuss.elastic.co/u/Dv_Thiyanesh)\ **Replies:** 0\ **Last updated:** [March 13, 2019, 5:33am UTC](https://discuss.elastic.co/t/creating-indices/172060 "2019-03-13T05:33:35Z")
Hii all, I have one server with Many clients My question is instead of having winlogbeat create daily indices? I can create separate index for different client(logs) (I need different index for different client) And a… --- ## [Filebeat incorrect metrics](https://discuss.elastic.co/t/filebeat-incorrect-metrics/170433)
**Author:** [@bbking](https://discuss.elastic.co/u/bbking)\ **Replies:** 2\ **Last updated:** [March 12, 2019, 7:09pm UTC](https://discuss.elastic.co/t/filebeat-incorrect-metrics/170433 "2019-03-12T19:09:05Z")
Hello! I'm testing Filebeat when Logstash is not available, but the filebeat metrics I got from HTTP endpoint don't match what I configured. There is no documentations to describe fields in the metrics, so I'm not sure i… --- ## [Beats: Pure go packet sniffer - Final Year Project](https://discuss.elastic.co/t/beats-pure-go-packet-sniffer-final-year-project/171632)
**Author:** [@eloyekunle](https://discuss.elastic.co/u/eloyekunle)\ **Replies:** 2\ **Last updated:** [March 12, 2019, 6:03pm UTC](https://discuss.elastic.co/t/beats-pure-go-packet-sniffer-final-year-project/171632 "2019-03-12T18:03:35Z")
@steffens @Nicolas\_Ruflin Hi everyone, I'm a Golang programmer, and a final year Computer Science student at the Federal University of Technology, Akure, Nigeria. I would like to work on Beats for my Final Year projec… --- ## [Journalbeat on Kubernetes](https://discuss.elastic.co/t/journalbeat-on-kubernetes/160603)
**Author:** [@przemek\_danysz](https://discuss.elastic.co/u/przemek_danysz)\ **Replies:** 2\ **Last updated:** [March 12, 2019, 4:21pm UTC](https://discuss.elastic.co/t/journalbeat-on-kubernetes/160603 "2019-03-12T16:21:31Z")
Dears, I'm trying to run Journalbeat on Kubernetes cluster and almost all works fine :slight\_smile: I run pod with Privileges and access to /var/log/journal directory, so Journalbeat is reading it correctly, but.... B… --- ## [Why not integrate beats](https://discuss.elastic.co/t/why-not-integrate-beats/171897)
**Author:** [@jack\_liang](https://discuss.elastic.co/u/jack_liang)\ **Replies:** 1\ **Last updated:** [March 12, 2019, 12:25pm UTC](https://discuss.elastic.co/t/why-not-integrate-beats/171897 "2019-03-12T12:25:16Z")
if i must use filebeat、metricbeat、packetbeat、winlogbeat. i must install filebeat、metricbeat、packetbeat、winlogbeat on the server. why not integrate all beats to one beat, then i only start the beat can do all. --- ## [Filebeat send json format log, kibana can't recognize filed](https://discuss.elastic.co/t/filebeat-send-json-format-log-kibana-cant-recognize-filed/171658)
**Author:** [@lijin\_liu](https://discuss.elastic.co/u/lijin_liu)\ **Replies:** 2\ **Last updated:** [March 12, 2019, 11:21am UTC](https://discuss.elastic.co/t/filebeat-send-json-format-log-kibana-cant-recognize-filed/171658 "2019-03-12T11:21:27Z")
Hello, I'm new to filebeat/elk. My problem is the Kibana can't recognize the field in my log. The log content looks(via less log.log): ... {"@timestamp":"2019-03-11T03:01:53.205+00:00","@version":"1","message":"spread… --- ## [Beats vs Kinesis Firehose for ELK Stack](https://discuss.elastic.co/t/beats-vs-kinesis-firehose-for-elk-stack/171800)
**Author:** [@lar](https://discuss.elastic.co/u/lar)\ **Replies:** 1\ **Last updated:** [March 12, 2019, 10:11am UTC](https://discuss.elastic.co/t/beats-vs-kinesis-firehose-for-elk-stack/171800 "2019-03-12T10:11:40Z")
I am planning to use Kinesis Firehose for both Windows and Linux instances to send logs to Logstash. Has anyone used this approach ? How will the data be collected from the Instances and sent over to Logstash ? What is t… --- ## [Filebeat 6.6.1(arm build) Harvester generate different inode for the same file](https://discuss.elastic.co/t/filebeat-6-6-1-arm-build-harvester-generate-different-inode-for-the-same-file/171882)
**Author:** [@kk1983](https://discuss.elastic.co/u/kk1983)\ **Replies:** 1\ **Last updated:** [March 12, 2019, 8:46am UTC](https://discuss.elastic.co/t/filebeat-6-6-1-arm-build-harvester-generate-different-inode-for-the-same-file/171882 "2019-03-12T08:46:19Z")
Hello, I build the filebeat for ARM according to the wiki https://discuss.elastic.co/t/how-to-install-filebeat-on-a-arm-based-sbc-eg-raspberry-pi-3/103670/3 Then i replace the filebeat in filebeat-6.1.1-linux-x86\_64 w… --- ## [Doubts regardind the field system.diskio.iostat.await](https://discuss.elastic.co/t/doubts-regardind-the-field-system-diskio-iostat-await/171744)
**Author:** [@killdaemon](https://discuss.elastic.co/u/killdaemon)\ **Replies:** 1\ **Last updated:** [March 12, 2019, 8:45am UTC](https://discuss.elastic.co/t/doubts-regardind-the-field-system-diskio-iostat-await/171744 "2019-03-12T08:45:44Z")
Hi, I'm trying to define a threshold for the alerting using the field system.diskio.iostat.await from a metricbeat running on some VMs. The documentation indicates that this field is a float with the average time spent f… --- ## [Cannot index event publisher.Event](https://discuss.elastic.co/t/cannot-index-event-publisher-event/167898)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 6\ **Last updated:** [March 11, 2019, 3:37pm UTC](https://discuss.elastic.co/t/cannot-index-event-publisher-event/167898 "2019-03-11T15:37:24Z")
Hello World! I'm using Elastic stack 6.6.1/6.7.1 and while following Enroll Beats in central management | Filebeat Reference \[6.6\] | Elastic, I've noticed few of beats have following WARN in /var/logfilebeat/filebeat: 2… --- ## [Filebeat input logs from file](https://discuss.elastic.co/t/filebeat-input-logs-from-file/171825)
**Author:** [@kharvey](https://discuss.elastic.co/u/kharvey)\ **Replies:** 0\ **Last updated:** [March 11, 2019, 6:37pm UTC](https://discuss.elastic.co/t/filebeat-input-logs-from-file/171825 "2019-03-11T18:37:49Z")
Filebeat version 6.6.1 Is it possible to use a file as my input for filebeat? I have an application that generates thousands of log files a day. This application generates an index log file that references all of the l… --- ## [Add \*built-in\* rate limiting/throttling](https://discuss.elastic.co/t/add-built-in-rate-limiting-throttling/171176)
**Author:** [@amomchilov](https://discuss.elastic.co/u/amomchilov)\ **Replies:** 6\ **Last updated:** [March 11, 2019, 6:12pm UTC](https://discuss.elastic.co/t/add-built-in-rate-limiting-throttling/171176 "2019-03-11T18:12:23Z")
There are multiple threads on this, but they're locked now. I think the existing solutions are all inadequate for a server environment with shared hardware/network infrastructures, where "playing nice" is important. Des… --- ## [Unexpected behavior for autodiscover appenders with processors](https://discuss.elastic.co/t/unexpected-behavior-for-autodiscover-appenders-with-processors/171415)
**Author:** [@walfie](https://discuss.elastic.co/u/walfie)\ **Replies:** 4\ **Last updated:** [March 11, 2019, 3:25pm UTC](https://discuss.elastic.co/t/unexpected-behavior-for-autodiscover-appenders-with-processors/171415 "2019-03-11T15:25:53Z")
I'm trying to use autodiscover, where I have some processors defined in the templates config, as well as some processors defined in the appenders section under certain conditions, like so: filebeat.autodiscover: provi… --- ## [Throttling log output from Filebeat directly?](https://discuss.elastic.co/t/throttling-log-output-from-filebeat-directly/168862)
**Author:** [@georgejdli](https://discuss.elastic.co/u/georgejdli)\ **Replies:** 7\ **Last updated:** [March 11, 2019, 3:03pm UTC](https://discuss.elastic.co/t/throttling-log-output-from-filebeat-directly/168862 "2019-03-11T15:03:08Z")
I'm working with a multi-tenant Kubernetes cluster and we're seeing issues where certain apps running as Docker containers are writing to log files at a really fast rate, such as spikes in app activity or some apps loggi… --- ## [How to backfil a saved evtx file into elasticsearch using winlogbeat](https://discuss.elastic.co/t/how-to-backfil-a-saved-evtx-file-into-elasticsearch-using-winlogbeat/171597)
**Author:** [@niki](https://discuss.elastic.co/u/niki)\ **Replies:** 1\ **Last updated:** [March 11, 2019, 2:51pm UTC](https://discuss.elastic.co/t/how-to-backfil-a-saved-evtx-file-into-elasticsearch-using-winlogbeat/171597 "2019-03-11T14:51:37Z")
can we backfil saved evtx files from different windows machines into elasticsearch and use them for correlation and forensics stuff? I used to do something like this with filebeat and webservers logs. zcat logname.gz |… --- ## [Akismet hiding my post](https://discuss.elastic.co/t/akismet-hiding-my-post/171788)
**Author:** [@eloyekunle](https://discuss.elastic.co/u/eloyekunle)\ **Replies:** 0\ **Last updated:** [March 11, 2019, 2:38pm UTC](https://discuss.elastic.co/t/akismet-hiding-my-post/171788 "2019-03-11T14:38:39Z")
Hello, I made a post here, and got informed that Akismet hid my post. Please can it be reviewed soon? Thanks. --- ## [Adding custom field for Haproxy filebeat module?](https://discuss.elastic.co/t/adding-custom-field-for-haproxy-filebeat-module/171770)
**Author:** [@10acc](https://discuss.elastic.co/u/10acc)\ **Replies:** 2\ **Last updated:** [March 11, 2019, 2:37pm UTC](https://discuss.elastic.co/t/adding-custom-field-for-haproxy-filebeat-module/171770 "2019-03-11T14:37:37Z")
Hello, I am using filebeat to ship my haproxy logs to the elastic cloud. But the thing I am missing in exported fields is "frontend\_port" , referred to as %fp in haproxy custom log format. Is there a relatively easy wa… --- ## [A Little guidance please](https://discuss.elastic.co/t/a-little-guidance-please/171598)
**Author:** [@10acc](https://discuss.elastic.co/u/10acc)\ **Replies:** 7\ **Last updated:** [March 11, 2019, 2:34pm UTC](https://discuss.elastic.co/t/a-little-guidance-please/171598 "2019-03-11T14:34:52Z")
Hello, and first of all I would like to say that your product is looking awesome , and pretty noob friendly ( having in mind that I managed to sort of set it up in one day). I am mainly looking to filtering Haproxy logs… --- ## [Filebeat-Logstash back pressure without persistent queues](https://discuss.elastic.co/t/filebeat-logstash-back-pressure-without-persistent-queues/171570)
**Author:** [@cciaccio](https://discuss.elastic.co/u/cciaccio)\ **Replies:** 3\ **Last updated:** [March 11, 2019, 2:10pm UTC](https://discuss.elastic.co/t/filebeat-logstash-back-pressure-without-persistent-queues/171570 "2019-03-11T14:10:53Z")
If Logstash persistent queues are disabled, is there back pressure between Logstash and Filebeat? --- ## [Field Extraction/Parsing](https://discuss.elastic.co/t/field-extraction-parsing/171352)
**Author:** [@elborni96](https://discuss.elastic.co/u/elborni96)\ **Replies:** 4\ **Last updated:** [March 11, 2019, 2:03pm UTC](https://discuss.elastic.co/t/field-extraction-parsing/171352 "2019-03-11T14:03:06Z")
Hi everyone, I'm a new user of the ELK stack. I'm monitoring a file and I would like to extract fields (hereinafter a small extraction): \ \ Vulnerability in Cisco Products (March 6, 2019) \ \ h… --- ## [Journalbeat input filtering (journalbeat processing its own logs and then logging that it processed them)?](https://discuss.elastic.co/t/journalbeat-input-filtering-journalbeat-processing-its-own-logs-and-then-logging-that-it-processed-them/171513)
**Author:** [@matp](https://discuss.elastic.co/u/matp)\ **Replies:** 3\ **Last updated:** [March 11, 2019, 1:58pm UTC](https://discuss.elastic.co/t/journalbeat-input-filtering-journalbeat-processing-its-own-logs-and-then-logging-that-it-processed-them/171513 "2019-03-11T13:58:44Z")
Howdy. This is in reference to journalbeat 6.6.1 Currently there appears to be 2 ways to filter input. I can filter by explicitly listing specific journal files and/or directories in paths I can use include\_matches … --- ## [Parse csv log file using filebeat](https://discuss.elastic.co/t/parse-csv-log-file-using-filebeat/171422)
**Author:** [@Tayyab](https://discuss.elastic.co/u/Tayyab)\ **Replies:** 4\ **Last updated:** [March 11, 2019, 1:36pm UTC](https://discuss.elastic.co/t/parse-csv-log-file-using-filebeat/171422 "2019-03-11T13:36:57Z")
I have a log file which is in csv format and I need it to parse to elastic search using filebeat with the fields like IP, Client.OS, url, datafield etc in that line of csv file. I have read that I have to use logstash t… --- ## [Journalbeats stops immediately after starting (Openshift)](https://discuss.elastic.co/t/journalbeats-stops-immediately-after-starting-openshift/171475)
**Author:** [@llech](https://discuss.elastic.co/u/llech)\ **Replies:** 2\ **Last updated:** [March 11, 2019, 12:58pm UTC](https://discuss.elastic.co/t/journalbeats-stops-immediately-after-starting-openshift/171475 "2019-03-11T12:58:15Z")
Hello, When I start Journalbeats 6.6.1 on Openshift 3.9, it stops immediately after starting with following log lines: 2019-03-08T10:55:02.215Z WARN \[cfgwarn\] beater/journalbeat.go:49 EXPERIMENTAL: Journalbeat is exper… --- ## [Filebeat is collecting Logs but not showing in kibana](https://discuss.elastic.co/t/filebeat-is-collecting-logs-but-not-showing-in-kibana/171291)
**Author:** [@Suresh\_Pal](https://discuss.elastic.co/u/Suresh_Pal)\ **Replies:** 4\ **Last updated:** [March 11, 2019, 6:16am UTC](https://discuss.elastic.co/t/filebeat-is-collecting-logs-but-not-showing-in-kibana/171291 "2019-03-11T06:16:17Z")
Hi Team, i'm using filebeat 6.4 It is collecting and parsing the logs but not showing at kibana. Note: Filebeat is running on my ELK system and I'm trying to parse some files from same sytem. --- ## [Filebeat multiline is ignoring my settings](https://discuss.elastic.co/t/filebeat-multiline-is-ignoring-my-settings/171194)
**Author:** [@eran.yo](https://discuss.elastic.co/u/eran.yo)\ **Replies:** 2\ **Last updated:** [March 10, 2019, 11:24am UTC](https://discuss.elastic.co/t/filebeat-multiline-is-ignoring-my-settings/171194 "2019-03-10T11:24:29Z")
Hi All, I'm trying to use the filebeat multiline option to marge logs data that have header and footer. My system logs have fixed header (#SQ#)and footer (#EOM#). for example : #SQ#|1551902277335100043|2019-03-06 21:… --- ## [How To Setup filebeat / elasticsearch in Docker-Swarm?](https://discuss.elastic.co/t/how-to-setup-filebeat-elasticsearch-in-docker-swarm/171620)
**Author:** [@rsoika](https://discuss.elastic.co/u/rsoika)\ **Replies:** 0\ **Last updated:** [March 10, 2019, 9:39am UTC](https://discuss.elastic.co/t/how-to-setup-filebeat-elasticsearch-in-docker-swarm/171620 "2019-03-10T09:39:22Z")
I am working on a Github project providing a lightweight docker-swarm environment. One feature I would like to add is a centralized logging stack based on Filebeat, Elasticsearch and Kibana. And my goal is to startup th… --- ## [Multiline Pattern not breaking logs into multiple events\[Closed\]](https://discuss.elastic.co/t/multiline-pattern-not-breaking-logs-into-multiple-events-closed/171610)
**Author:** [@ankur\_singla](https://discuss.elastic.co/u/ankur_singla)\ **Replies:** 1\ **Last updated:** [March 10, 2019, 8:44am UTC](https://discuss.elastic.co/t/multiline-pattern-not-breaking-logs-into-multiple-events-closed/171610 "2019-03-10T08:44:51Z")
Hi Guys, I am facing problem with multiline pattern in filebeat. My usecase for multiline for log data is very simple if log is not starting with date format(yyyy-mm-dd) then it should be considered as single event else… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=374) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=376)