# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=442 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 443 --- ## [Persist Filebeat registry on Kubernetes host](https://discuss.elastic.co/t/persist-filebeat-registry-on-kubernetes-host/137285)
**Author:** [@mariomechoulam](https://discuss.elastic.co/u/mariomechoulam)\ **Replies:** 2\ **Last updated:** [June 26, 2018, 1:54pm UTC](https://discuss.elastic.co/t/persist-filebeat-registry-on-kubernetes-host/137285 "2018-06-26T13:54:27Z")
Hello everyone! We have Filebeat running as a DaemonSet in our Kubernetes cluster, using, for the most part, the yaml provided in the Filebeat documentation. I am trying to decide whether we should be using the tail\_fi… --- ## [Docker autodiscover feature: losing tail of logs when container gets stopped](https://discuss.elastic.co/t/docker-autodiscover-feature-losing-tail-of-logs-when-container-gets-stopped/137060)
**Author:** [@smamontov](https://discuss.elastic.co/u/smamontov)\ **Replies:** 2\ **Last updated:** [June 26, 2018, 12:58pm UTC](https://discuss.elastic.co/t/docker-autodiscover-feature-losing-tail-of-logs-when-container-gets-stopped/137060 "2018-06-26T12:58:43Z")
I'm running docker container without --rm option to make sure this is not because log file is removed by docker. The problem is that not all events are getting sent to elasticsearch: messages are ignored since the point … --- ## [Index name based on tags](https://discuss.elastic.co/t/index-name-based-on-tags/137065)
**Author:** [@hobapolis](https://discuss.elastic.co/u/hobapolis)\ **Replies:** 2\ **Last updated:** [June 26, 2018, 12:51pm UTC](https://discuss.elastic.co/t/index-name-based-on-tags/137065 "2018-06-26T12:51:19Z")
trying to accomplish: index: "filebeat-%{\[beat.version\]}-%{\[tags\]}-%{+yyyy.MM.dd}" my prospector looks similar to this: - type: log enabled: true paths: - /var/log/auth.log tags: auth exclude\_files: \['\\.g… --- ## [K8s filebeat](https://discuss.elastic.co/t/k8s-filebeat/137389)
**Author:** [@Saravana\_Kumar](https://discuss.elastic.co/u/Saravana_Kumar)\ **Replies:** 1\ **Last updated:** [June 26, 2018, 8:47am UTC](https://discuss.elastic.co/t/k8s-filebeat/137389 "2018-06-26T08:47:41Z")
Have deployed filebeat on kubernetes as per below link : https://www.elastic.co/guide/en/beats/filebeat/current/running-on-kubernetes.html How to add grok patterns on the filebeat, we are using an external ELK.. since … --- ## [Filebeat Load balance logstash with kubernetes service](https://discuss.elastic.co/t/filebeat-load-balance-logstash-with-kubernetes-service/137373)
**Author:** [@sivakr](https://discuss.elastic.co/u/sivakr)\ **Replies:** 2\ **Last updated:** [June 26, 2018, 8:04am UTC](https://discuss.elastic.co/t/filebeat-load-balance-logstash-with-kubernetes-service/137373 "2018-06-26T08:04:23Z")
In my kubernetes cluster I have Filebeat running on a node and I have 6 instances of logstash for load balancing. This is the configuration in Filebeat output.logstash: hosts: \[logstash.logstash:5044\] bulk\_max\_size… --- ## [MetricBeat shipping incomplete data from raspberry](https://discuss.elastic.co/t/metricbeat-shipping-incomplete-data-from-raspberry/134033)
**Author:** [@amitphulera](https://discuss.elastic.co/u/amitphulera)\ **Replies:** 4\ **Last updated:** [June 26, 2018, 5:21am UTC](https://discuss.elastic.co/t/metricbeat-shipping-incomplete-data-from-raspberry/134033 "2018-06-26T05:21:14Z")
Hi All, I have recently started exploring the elastic products and was able to run ElasticSearch, Kibana, and MetricBeat on my workstation and I was able to view system metrics on Kibana dashboards perfectly. I wanted … --- ## [My metricbeats stopped working after the latest ugprade! and I have errors on mapping 6.3.0](https://discuss.elastic.co/t/my-metricbeats-stopped-working-after-the-latest-ugprade-and-i-have-errors-on-mapping-6-3-0/137158)
**Author:** [@Billias](https://discuss.elastic.co/u/Billias)\ **Replies:** 1\ **Last updated:** [June 25, 2018, 4:54pm UTC](https://discuss.elastic.co/t/my-metricbeats-stopped-working-after-the-latest-ugprade-and-i-have-errors-on-mapping-6-3-0/137158 "2018-06-25T16:54:04Z")
My metricbeats stopped working after the latest ugprade! and I have errors on mapping Dear all, my metricbeat became problematic after upgrading to 6.3.0. My whole cluster running on 6.3.0 my beats are sending data to … --- ## [Filebeat not publishing multi-line event after timeout](https://discuss.elastic.co/t/filebeat-not-publishing-multi-line-event-after-timeout/137235)
**Author:** [@dev.net](https://discuss.elastic.co/u/dev.net)\ **Replies:** 2\ **Last updated:** [June 25, 2018, 3:18pm UTC](https://discuss.elastic.co/t/filebeat-not-publishing-multi-line-event-after-timeout/137235 "2018-06-25T15:18:02Z")
Hi, I have configured filebeat with a multi line pattern to watch log files that will be written to once and contain a single root XML element with no XML declaration. Sometimes the root XML element can span multiple li… --- ## [Filebeat startup failed](https://discuss.elastic.co/t/filebeat-startup-failed/137173)
**Author:** [@weilinqwe](https://discuss.elastic.co/u/weilinqwe)\ **Replies:** 2\ **Last updated:** [June 25, 2018, 10:48am UTC](https://discuss.elastic.co/t/filebeat-startup-failed/137173 "2018-06-25T10:48:34Z")
I use filebeat to collect file log. the filebeat version is 6.2.2, the os version is Redhat Enterprise 5.1. when start filebeat, it failed. the log showed: Exiting: open /proc/sys/kernel/hostname: errno 530. I guess it … --- ## [Filebeat can't keep up with single log file](https://discuss.elastic.co/t/filebeat-cant-keep-up-with-single-log-file/136979)
**Author:** [@JensVanDeynse](https://discuss.elastic.co/u/JensVanDeynse)\ **Replies:** 9\ **Last updated:** [June 25, 2018, 6:14am UTC](https://discuss.elastic.co/t/filebeat-cant-keep-up-with-single-log-file/136979 "2018-06-25T06:14:57Z")
Hello Logstash version: 6.2.4 Filebeat version: 6.2.4 We've setup an ELK stack in our company but recently we've been experiencing some issues during the morning ramp-up on our filebeats. The problem is as follows: F… --- ## [Parsing multiple lines CSV with filebeat & Logstash](https://discuss.elastic.co/t/parsing-multiple-lines-csv-with-filebeat-logstash/137070)
**Author:** [@TestCandidate](https://discuss.elastic.co/u/TestCandidate)\ **Replies:** 1\ **Last updated:** [June 24, 2018, 8:37am UTC](https://discuss.elastic.co/t/parsing-multiple-lines-csv-with-filebeat-logstash/137070 "2018-06-24T08:37:08Z")
Hey Guys, I have a CSV file that looks like the attached image. \[Capture\] I want to send this csv file to logstash. The configuration is working well except that the whole file is being sent as a single message. file… --- ## [Filebeat decoding Docker logs error](https://discuss.elastic.co/t/filebeat-decoding-docker-logs-error/136961)
**Author:** [@d54c364cb1c1253c751d](https://discuss.elastic.co/u/d54c364cb1c1253c751d)\ **Replies:** 4\ **Last updated:** [June 23, 2018, 4:29pm UTC](https://discuss.elastic.co/t/filebeat-decoding-docker-logs-error/136961 "2018-06-23T16:29:10Z")
Filebeat Version:6.2.4 Docker version: 1.12.5, build 7392c3b get decoding error when try to get docker logs ERROR log/harvester.go:243 Read line error: decoding docker JSON: EOF; File: %! logs are like this {"log":… --- ## [\[FILEBEAT\] How to read, send to ElasticSearch and close each files](https://discuss.elastic.co/t/filebeat-how-to-read-send-to-elasticsearch-and-close-each-files/137029)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 2\ **Last updated:** [June 22, 2018, 6:10pm UTC](https://discuss.elastic.co/t/filebeat-how-to-read-send-to-elasticsearch-and-close-each-files/137029 "2018-06-22T18:10:01Z")
Hello everybody. I would like to know if I had possibility for each files that Filebeat listen to do : Open the file. Reading the file. Do a logstash treatment. Send the file to logstash output. Close this file. I fo… --- ## [How to Ship/Parse JSON logs (from Kibana) to Elasticsearch?](https://discuss.elastic.co/t/how-to-ship-parse-json-logs-from-kibana-to-elasticsearch/136345)
**Author:** [@brolo](https://discuss.elastic.co/u/brolo)\ **Replies:** 6\ **Last updated:** [June 22, 2018, 3:51pm UTC](https://discuss.elastic.co/t/how-to-ship-parse-json-logs-from-kibana-to-elasticsearch/136345 "2018-06-22T15:51:54Z")
I am trying to ship logs from a Kibana instance running in a docker container. I noticed that there is a module for Kibana in development (posted Github link below) and I pulled the pipeline code from Github and attempte… --- ## [Data.host is empty when running on ecs](https://discuss.elastic.co/t/data-host-is-empty-when-running-on-ecs/136931)
**Author:** [@anhlon](https://discuss.elastic.co/u/anhlon)\ **Replies:** 2\ **Last updated:** [June 22, 2018, 4:10pm UTC](https://discuss.elastic.co/t/data-host-is-empty-when-running-on-ecs/136931 "2018-06-22T16:10:36Z")
Hi All, I have setup up metricbeat with docker provider and autodiscover. It works really well when running it locally via docker-compose or swarm for local testing. I proceeded to deploy metricbeat to ecs and have it… --- ## [Winlogbeat.event\_logs not working properly](https://discuss.elastic.co/t/winlogbeat-event-logs-not-working-properly/136871)
**Author:** [@Astarandel](https://discuss.elastic.co/u/Astarandel)\ **Replies:** 5\ **Last updated:** [June 22, 2018, 12:02pm UTC](https://discuss.elastic.co/t/winlogbeat-event-logs-not-working-properly/136871 "2018-06-22T12:02:15Z")
Hello guys, I'm testing a setup atm where Winlogbeat sends Windows events logs to Graylog server. I wanted to limit the number of events Winlogbeat is collecting, so I tried to use winlogbeat.event\_logs. Here is my wi… --- ## [Cannot validate certificate for X.X.X.X because it doesn’t contain any IP SANs](https://discuss.elastic.co/t/cannot-validate-certificate-for-x-x-x-x-because-it-doesn-t-contain-any-ip-sans/136997)
**Author:** [@Ihjaz](https://discuss.elastic.co/u/Ihjaz)\ **Replies:** 1\ **Last updated:** [June 22, 2018, 11:54am UTC](https://discuss.elastic.co/t/cannot-validate-certificate-for-x-x-x-x-because-it-doesn-t-contain-any-ip-sans/136997 "2018-06-22T11:54:44Z")
Hi, I have seen the Note below from https://www.elastic.co/guide/en/beats/filebeat/current/securing-communication-elasticsearch.html For any given connection, the SSL/TLS certificates must have a subject that matches … --- ## [Monitoring remote Elasticsearch performance with metricbeats and Kibana "Monitoring"](https://discuss.elastic.co/t/monitoring-remote-elasticsearch-performance-with-metricbeats-and-kibana-monitoring/136867)
**Author:** [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\ **Replies:** 4\ **Last updated:** [June 22, 2018, 10:56am UTC](https://discuss.elastic.co/t/monitoring-remote-elasticsearch-performance-with-metricbeats-and-kibana-monitoring/136867 "2018-06-22T10:56:56Z")
Running elasticsearch/kibana (central) and metricbeats version 6.3 I've just enabled metricbeats to monitor 6 edge servers that are running local elasticsearch-based application instances. metricbeats is shipping the m… --- ## [How to get 10 before and after lines if pattern is matched in multiline in Filebeat](https://discuss.elastic.co/t/how-to-get-10-before-and-after-lines-if-pattern-is-matched-in-multiline-in-filebeat/136689)
**Author:** [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\ **Replies:** 2\ **Last updated:** [June 22, 2018, 5:16am UTC](https://discuss.elastic.co/t/how-to-get-10-before-and-after-lines-if-pattern-is-matched-in-multiline-in-filebeat/136689 "2018-06-22T05:16:48Z")
here is my input / # \[ 349.472726\] audit: rate limit exceeded \[ 359.080131\] android\_reboot: Start sync before filesystem remount as read-only ... \[ 359.177696\] android\_reboot: sync done \[ 359.182012\] SysRq : Emerg… --- ## [No xpack monitoring for metricbeat](https://discuss.elastic.co/t/no-xpack-monitoring-for-metricbeat/136573)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 3\ **Last updated:** [June 21, 2018, 9:53pm UTC](https://discuss.elastic.co/t/no-xpack-monitoring-for-metricbeat/136573 "2018-06-21T21:53:49Z")
I have enabled xpack monitoring in 6.3.0 in filebeat, auditbeat, and metricbeat. Security is disabled. filebeat and auditbeat show up just fine, metricbeat does not. metricbeat logs an error every 10 seconds 2018-06-19T… --- ## [Filebeat - Decoding Json logs with Linebreaks](https://discuss.elastic.co/t/filebeat-decoding-json-logs-with-linebreaks/136394)
**Author:** [@motamedi791](https://discuss.elastic.co/u/motamedi791)\ **Replies:** 5\ **Last updated:** [June 21, 2018, 9:50pm UTC](https://discuss.elastic.co/t/filebeat-decoding-json-logs-with-linebreaks/136394 "2018-06-21T21:50:04Z")
I was able to make FileBeat work with json log files. However, when a log message contains line breaks, the json parser can not decode the json. For example, the following log line can not be decoded correctly: {"times… --- ## [Winlogbeat doesnot have system logged time](https://discuss.elastic.co/t/winlogbeat-doesnot-have-system-logged-time/136830)
**Author:** [@mariyana](https://discuss.elastic.co/u/mariyana)\ **Replies:** 1\ **Last updated:** [June 21, 2018, 9:42pm UTC](https://discuss.elastic.co/t/winlogbeat-doesnot-have-system-logged-time/136830 "2018-06-21T21:42:09Z")
hi it has the @timestamp fileld. But this timestamp filed has the time when the event has been parsed by winlogbeat rather than the time when it has been logged(created) as windows event. Please help me with this issu… --- ## [Geolocation not showing up](https://discuss.elastic.co/t/geolocation-not-showing-up/136675)
**Author:** [@nairihar](https://discuss.elastic.co/u/nairihar)\ **Replies:** 3\ **Last updated:** [June 21, 2018, 9:03pm UTC](https://discuss.elastic.co/t/geolocation-not-showing-up/136675 "2018-06-21T21:03:14Z")
Hi, I enabled Nginx logs in Kibana using Beats, also installed two elasticsearch plugins which written in Kibana Getting Started for DEB. But I not see any map data. Nginx request graphs shows all, but geo data not sh… --- ## [Embedding libbeat in an application](https://discuss.elastic.co/t/embedding-libbeat-in-an-application/132707)
**Author:** [@dustindecker](https://discuss.elastic.co/u/dustindecker)\ **Replies:** 3\ **Last updated:** [June 21, 2018, 4:42pm UTC](https://discuss.elastic.co/t/embedding-libbeat-in-an-application/132707 "2018-06-21T16:42:00Z")
Previously discussed by others here: Embedding a custom beat in my application So far libbeat has been intended to be used to produce standalone data shippers by using the beats framework to generate the project layout.… --- ## [Importing indicies for filebeat](https://discuss.elastic.co/t/importing-indicies-for-filebeat/136756)
**Author:** [@alexserd](https://discuss.elastic.co/u/alexserd)\ **Replies:** 3\ **Last updated:** [June 21, 2018, 4:04pm UTC](https://discuss.elastic.co/t/importing-indicies-for-filebeat/136756 "2018-06-21T16:04:50Z")
I've deleted all data from elasticsearch which included indicies invoke-webrequest -method delete http://localhost:9200/\* After the deletion the server was rebooted I'm able to see filebeat-6.2.4 under http://localhos… --- ## [Filebeat to parse modsecurity json logs](https://discuss.elastic.co/t/filebeat-to-parse-modsecurity-json-logs/134662)
**Author:** [@angeL](https://discuss.elastic.co/u/angeL)\ **Replies:** 8\ **Last updated:** [June 21, 2018, 3:24pm UTC](https://discuss.elastic.co/t/filebeat-to-parse-modsecurity-json-logs/134662 "2018-06-21T15:24:34Z")
Hi team, I'm using filebeat 6.2.4 to parse ModSecurity logs in Json format, so far so good using the following config: filebeat.prospectors: - type: log paths: - /var/log/httpd/modsec\_audit.log … --- ## [Auditbeat with multiple outputs](https://discuss.elastic.co/t/auditbeat-with-multiple-outputs/136747)
**Author:** [@data\_smith](https://discuss.elastic.co/u/data_smith)\ **Replies:** 2\ **Last updated:** [June 21, 2018, 1:02pm UTC](https://discuss.elastic.co/t/auditbeat-with-multiple-outputs/136747 "2018-06-21T13:02:51Z")
Can users configure Auditbeat to have multiple outputs? For example one output Elasticsearch and one output the file system. It appears that's not possible. --- ## [Filebeat module - auditd](https://discuss.elastic.co/t/filebeat-module-auditd/136681)
**Author:** [@charan.gandra](https://discuss.elastic.co/u/charan.gandra)\ **Replies:** 2\ **Last updated:** [June 21, 2018, 12:37pm UTC](https://discuss.elastic.co/t/filebeat-module-auditd/136681 "2018-06-21T12:37:35Z")
Hi, I am using auditd module in Filebeat to send the audit logs to Logstash. Do I still need to parse the audit logs in Logstash? When I sent the output to elasticsearch everything was working fine but whenswitched to … --- ## [Delete files using the "registry" file](https://discuss.elastic.co/t/delete-files-using-the-registry-file/136788)
**Author:** [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Replies:** 2\ **Last updated:** [June 21, 2018, 10:55am UTC](https://discuss.elastic.co/t/delete-files-using-the-registry-file/136788 "2018-06-21T10:55:03Z")
Hi, I have a use case in which, multiple large files are pushed on a server. These are read by Filebeat and sent to Logstash. I have written a script to read the registry file and get the filenames that have been ackno… --- ## [Monitoring Beats: Instance list too short (old)](https://discuss.elastic.co/t/monitoring-beats-instance-list-too-short-old/136365)
**Author:** [@kaem2111](https://discuss.elastic.co/u/kaem2111)\ **Replies:** 1\ **Last updated:** [June 21, 2018, 8:21am UTC](https://discuss.elastic.co/t/monitoring-beats-instance-list-too-short-old/136365 "2018-06-21T08:21:16Z")
As you see here: \[grafik\] Total beats are 64 (instead of metric 64 + file 63 + winlog 1 = 128), so only the newest half of all beats are shown in the list (see 1-20 of 64, I would expect 1-20 of 128). Moreover the lis… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=441) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=443)