# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=427 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 428 --- ## [Kafka metricbeat config](https://discuss.elastic.co/t/kafka-metricbeat-config/144679)
**Author:** [@InspurKelly](https://discuss.elastic.co/u/InspurKelly)\ **Replies:** 2\ **Last updated:** [August 17, 2018, 12:51am UTC](https://discuss.elastic.co/t/kafka-metricbeat-config/144679 "2018-08-17T00:51:31Z")
Hi, I use metricbeat collecting kafka metrics, i want to filter data when kafka.partition.partition.leader not equals kafka.partition.partition.replica, metricbeat.modules: module: kafka name: "kafka" metricsets: \["… --- ## [Server Spec for Filebeat](https://discuss.elastic.co/t/server-spec-for-filebeat/144683)
**Author:** [@iwai1493](https://discuss.elastic.co/u/iwai1493)\ **Replies:** 2\ **Last updated:** [August 17, 2018, 12:48am UTC](https://discuss.elastic.co/t/server-spec-for-filebeat/144683 "2018-08-17T00:48:50Z")
We are trying to use Filebeat in order to send the log data from the log collection server to the SIEM. The log data would amount to 100GB per day, the average throughput is about 1.2MB/s (9.5Mbps). Q1. Would you te… --- ## [Filebeat configuration](https://discuss.elastic.co/t/filebeat-configuration/144780)
**Author:** [@TwoDragon](https://discuss.elastic.co/u/TwoDragon)\ **Replies:** 2\ **Last updated:** [August 16, 2018, 9:59pm UTC](https://discuss.elastic.co/t/filebeat-configuration/144780 "2018-08-16T21:59:46Z")
Hi, I am new to Filebeat, would someone show me where in the filebeat.yml to set the option for close\_removed close\_inactive Thanks --- ## [Auditbeat running with Auditd](https://discuss.elastic.co/t/auditbeat-running-with-auditd/144782)
**Author:** [@ryannelle](https://discuss.elastic.co/u/ryannelle)\ **Replies:** 1\ **Last updated:** [August 16, 2018, 9:27pm UTC](https://discuss.elastic.co/t/auditbeat-running-with-auditd/144782 "2018-08-16T21:27:53Z")
Looking for some feedback on running auditbeat with auditd. Seems these two data points contradict each other. Blog says they run side by side 6.3 Docs Say turn it off Can someone explain what will happen or why it's… --- ## [Winlogbeat events not showing in AWS Elasticsearch](https://discuss.elastic.co/t/winlogbeat-events-not-showing-in-aws-elasticsearch/144572)
**Author:** [@fionamacd](https://discuss.elastic.co/u/fionamacd)\ **Replies:** 6\ **Last updated:** [August 16, 2018, 8:09pm UTC](https://discuss.elastic.co/t/winlogbeat-events-not-showing-in-aws-elasticsearch/144572 "2018-08-16T20:09:49Z")
Hi, I am just starting out with ELK and setting it up on AWS and using AWS ElasticSearch Service. I have configured Filebeat to send files from my AWS Windows instance to an AWS Ubuntu instance running Logstash on it. … --- ## [Dynamically update filebeat filebeat.yml file](https://discuss.elastic.co/t/dynamically-update-filebeat-filebeat-yml-file/144467)
**Author:** [@debayo](https://discuss.elastic.co/u/debayo)\ **Replies:** 3\ **Last updated:** [August 16, 2018, 7:34pm UTC](https://discuss.elastic.co/t/dynamically-update-filebeat-filebeat-yml-file/144467 "2018-08-16T19:34:55Z")
Hello guys kindly take a look at this question on https://stackoverflow.com/questions/51854887/dynamically-update-filebeat-config-yml-file --- ## [Filebeat syscall/dll\_windows.go Error](https://discuss.elastic.co/t/filebeat-syscall-dll-windows-go-error/144525)
**Author:** [@perryparktung](https://discuss.elastic.co/u/perryparktung)\ **Replies:** 4\ **Last updated:** [August 16, 2018, 5:33pm UTC](https://discuss.elastic.co/t/filebeat-syscall-dll-windows-go-error/144525 "2018-08-16T17:33:43Z")
Hi, I got following error when booting filebeat as a service in Windows Server 2003. Does anyone know what's going on? It said "recovering", but I still want to know what happened... 2018-08-15T09:19:51.147-0400 ERROR … --- ## [Cpu.total.value field](https://discuss.elastic.co/t/cpu-total-value-field/144621)
**Author:** [@Sundar\_V](https://discuss.elastic.co/u/Sundar_V)\ **Replies:** 1\ **Last updated:** [August 16, 2018, 3:21pm UTC](https://discuss.elastic.co/t/cpu-total-value-field/144621 "2018-08-16T15:21:57Z")
In the list of exported fields listed here, system.cpu.total.value is listed system.cpu.total.value type: long The value of CPU usage since starting the process. However, from the CPU output and from source file he… --- ## [How to drop field in message](https://discuss.elastic.co/t/how-to-drop-field-in-message/144689)
**Author:** [@nxdu93](https://discuss.elastic.co/u/nxdu93)\ **Replies:** 1\ **Last updated:** [August 16, 2018, 1:42pm UTC](https://discuss.elastic.co/t/how-to-drop-field-in-message/144689 "2018-08-16T13:42:04Z")
Hi all ! processors: drop\_fields: fields: \["\_source.message.index", "\_source.message.layers", "\_source.message.frame", "\_source.message.eth", "fields", "tags"\] I want to delete the fields not required in message as:… --- ## [Apache2 SSL log Parsing Issue](https://discuss.elastic.co/t/apache2-ssl-log-parsing-issue/144588)
**Author:** [@AlexB](https://discuss.elastic.co/u/AlexB)\ **Replies:** 2\ **Last updated:** [August 16, 2018, 11:55am UTC](https://discuss.elastic.co/t/apache2-ssl-log-parsing-issue/144588 "2018-08-16T11:55:31Z")
I'm using Filebeat to push Apache2 logs directly to Elasticsearch but I'm getting an error "Provided Grok expressions do not match field value:" when I try to add ssl\_request\_log file with two newly defined fields. I've… --- ## [\[ANNOUNCEMENT\] Spring Boot 2 for metricmeter.io Beat released](https://discuss.elastic.co/t/announcement-spring-boot-2-for-metricmeter-io-beat-released/144641)
**Author:** [@philkra](https://discuss.elastic.co/u/philkra)\ **Replies:** 4\ **Last updated:** [August 16, 2018, 11:00am UTC](https://discuss.elastic.co/t/announcement-spring-boot-2-for-metricmeter-io-beat-released/144641 "2018-08-16T11:00:02Z")
Hi Beats community, I'm please to announce that, I added a Metricbeat for Spring Boot 2 leveraging the micrometer.io metrics facade: https://github.com/philkra/springboot2beat The Metricbeat pulls all available metrics… --- ## [Filebeat Copy Field Processor](https://discuss.elastic.co/t/filebeat-copy-field-processor/143098)
**Author:** [@prometheus1](https://discuss.elastic.co/u/prometheus1)\ **Replies:** 2\ **Last updated:** [August 16, 2018, 10:58am UTC](https://discuss.elastic.co/t/filebeat-copy-field-processor/143098 "2018-08-16T10:58:01Z")
I found the rename processor for filebeat, couldn't find anything related to a copy field processor. My usecase is I want to copy some fields from the kubernetes processor to a root field with the original fields remaini… --- ## [MetricBeat NFS](https://discuss.elastic.co/t/metricbeat-nfs/144622)
**Author:** [@LeeKyungMin](https://discuss.elastic.co/u/LeeKyungMin)\ **Replies:** 1\ **Last updated:** [August 16, 2018, 7:46am UTC](https://discuss.elastic.co/t/metricbeat-nfs/144622 "2018-08-16T07:46:47Z")
I want to check information about mounted file system on my server. I’ve got a MetricBeat installed host server accessing NAS directory through NFS protocol. How can I view informations about this mounted directory (di… --- ## [Multiline matching not working as aspected](https://discuss.elastic.co/t/multiline-matching-not-working-as-aspected/143989)
**Author:** [@shani\_angarkadu](https://discuss.elastic.co/u/shani_angarkadu)\ **Replies:** 4\ **Last updated:** [August 16, 2018, 4:47am UTC](https://discuss.elastic.co/t/multiline-matching-not-working-as-aspected/143989 "2018-08-16T04:47:24Z")
I have tried all the combinations of multiline. multiline.pattern: '^(\[0-9\]{4})-(\[0-1\]\[0-9\])-(\[0-3\]\[0-9\])\\s(\[0-1\]\[0-9\]|\[2\]\[0-3\]):(\[0-5\]\[0-9\]):(\[0-5\]\[0-9\])$' //timestamp pattern multiline.negate: true multiline.match: … --- ## [How the server generation log is passed to logstash after startup after filebeat unexpectedly stops?](https://discuss.elastic.co/t/how-the-server-generation-log-is-passed-to-logstash-after-startup-after-filebeat-unexpectedly-stops/144459)
**Author:** [@ss.z](https://discuss.elastic.co/u/ss.z)\ **Replies:** 2\ **Last updated:** [August 16, 2018, 2:58am UTC](https://discuss.elastic.co/t/how-the-server-generation-log-is-passed-to-logstash-after-startup-after-filebeat-unexpectedly-stops/144459 "2018-08-16T02:58:07Z")
Hi I'm using ELK-6.3.1 now I have a problem: after the filebeat server unexpectedly stopped, filebeat did not start after restarting the server, and after restarting filebeat, I have accumulated a lot of logs. How do I … --- ## [Filebeat log rotation](https://discuss.elastic.co/t/filebeat-log-rotation/144573)
**Author:** [@chandra0651](https://discuss.elastic.co/u/chandra0651)\ **Replies:** 1\ **Last updated:** [August 16, 2018, 1:54am UTC](https://discuss.elastic.co/t/filebeat-log-rotation/144573 "2018-08-16T01:54:18Z")
I know there is a way in file beat where we can mention the size at which the log can be rotated and number of files it should keep as retention , but is there a way where we can rotate the logs based on each day ? --- ## [Metricbeat RabbitMQ module not sending all metric sets](https://discuss.elastic.co/t/metricbeat-rabbitmq-module-not-sending-all-metric-sets/144264)
**Author:** [@joshsmoore](https://discuss.elastic.co/u/joshsmoore)\ **Replies:** 5\ **Last updated:** [August 16, 2018, 12:50am UTC](https://discuss.elastic.co/t/metricbeat-rabbitmq-module-not-sending-all-metric-sets/144264 "2018-08-16T00:50:17Z")
I am using metric beat 6.3.0 to collect metrics from rabbitMQ. I was able to get it working for the node metricset. However, the connection and queue metric sets do not report any data. I am not seeing any errors in m… --- ## ["Failed to connect" error](https://discuss.elastic.co/t/failed-to-connect-error/144180)
**Author:** [@g.myznikov.tinkoff](https://discuss.elastic.co/u/g.myznikov.tinkoff)\ **Replies:** 10\ **Last updated:** [August 15, 2018, 3:40pm UTC](https://discuss.elastic.co/t/failed-to-connect-error/144180 "2018-08-15T15:40:11Z")
Hello! I've used ELK stack with filebeat on docker. There are my config file. elasticsearch.yml: path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch network.host: 0.0.0.0 kibana.yml: server.port: 5601… --- ## [Change Grok in Filebeat (Log Input)](https://discuss.elastic.co/t/change-grok-in-filebeat-log-input/144358)
**Author:** [@lumi](https://discuss.elastic.co/u/lumi)\ **Replies:** 4\ **Last updated:** [August 15, 2018, 2:44pm UTC](https://discuss.elastic.co/t/change-grok-in-filebeat-log-input/144358 "2018-08-15T14:44:53Z")
Hello I've found the file /usr/share/filebeat/module/system/syslog/ingest/pipeline.json which parses syslog messages. but it parses just the format of Redhat Ubuntu and so on. Format of RHEL: Aug 14 14:20:48 server1 … --- ## [Filtering processors do not work for syslog](https://discuss.elastic.co/t/filtering-processors-do-not-work-for-syslog/144301)
**Author:** [@Fpernet](https://discuss.elastic.co/u/Fpernet)\ **Replies:** 8\ **Last updated:** [August 15, 2018, 1:00pm UTC](https://discuss.elastic.co/t/filtering-processors-do-not-work-for-syslog/144301 "2018-08-15T13:00:25Z")
Hi. Using ES 6.3.0 and Filebeat 6.3.2. Try to configure filtering to exclude some lines from the syslog file. I tried processors and/or "exclude\_lines" but none of them are working. Since i can see in debug level some l… --- ## [Problem getting autodiscover docker to work with filebeat](https://discuss.elastic.co/t/problem-getting-autodiscover-docker-to-work-with-filebeat/144349)
**Author:** [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\ **Replies:** 10\ **Last updated:** [August 15, 2018, 12:31pm UTC](https://discuss.elastic.co/t/problem-getting-autodiscover-docker-to-work-with-filebeat/144349 "2018-08-15T12:31:16Z")
I'm trying to get the filebeat.autodiscover feature working with type:docker. If I put in this default configuration, I don't see anything coming into Elastic/Kibana (although I am getting the system, audit, and other l… --- ## [Migration from Logstash 2.x to Filebeat, Logstash and ElasticSearch 6.x](https://discuss.elastic.co/t/migration-from-logstash-2-x-to-filebeat-logstash-and-elasticsearch-6-x/144488)
**Author:** [@brjisc](https://discuss.elastic.co/u/brjisc)\ **Replies:** 1\ **Last updated:** [August 15, 2018, 12:24pm UTC](https://discuss.elastic.co/t/migration-from-logstash-2-x-to-filebeat-logstash-and-elasticsearch-6-x/144488 "2018-08-15T12:24:09Z")
Overview of current system Logstash 2.x reading from Apache log files. These log files come from a large number fo web services that we run. The intention of this process to record the various requests made to service… --- ## [Kubernetes data collection error with metricbeat!](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511)
**Author:** [@b22565875b81d856c5b0](https://discuss.elastic.co/u/b22565875b81d856c5b0)\ **Replies:** 21\ **Last updated:** [August 15, 2018, 10:45am UTC](https://discuss.elastic.co/t/kubernetes-data-collection-error-with-metricbeat/143511 "2018-08-15T10:45:33Z")
Hello all! I am trying to get kubernetes describing data (kubernetes.) via metricbeat. For installation i used metricbeat-kubernetes.yaml template. I have installed locally single-node kubernetes via docker, and many-no… --- ## [File beat keeps crashing](https://discuss.elastic.co/t/file-beat-keeps-crashing/141234)
**Author:** [@mugil1988](https://discuss.elastic.co/u/mugil1988)\ **Replies:** 17\ **Last updated:** [August 15, 2018, 6:48am UTC](https://discuss.elastic.co/t/file-beat-keeps-crashing/141234 "2018-08-15T06:48:58Z")
Hi Team, We have installed filebeats to ship logs to logstash..we haven't faced any issues for first 3 months.. From last week, file beat is keep crashing for every 5 minutes. There is no error in file beat logs and l… --- ## [Filebeat is sending the empty value to logstash for the paramters configured in filebeat](https://discuss.elastic.co/t/filebeat-is-sending-the-empty-value-to-logstash-for-the-paramters-configured-in-filebeat/143632)
**Author:** [@swethamahesh](https://discuss.elastic.co/u/swethamahesh)\ **Replies:** 5\ **Last updated:** [August 14, 2018, 5:24pm UTC](https://discuss.elastic.co/t/filebeat-is-sending-the-empty-value-to-logstash-for-the-paramters-configured-in-filebeat/143632 "2018-08-14T17:24:14Z")
Hi, Further to my discussion in the logstash forum regarding the junk directory getting created by logstash. It is conveyed that filebeat is sending the data empty, Hence logstash is writing the directory with junk nam… --- ## [Multi-cluster Kubernetes monitoring - best practices? Cluster name field?](https://discuss.elastic.co/t/multi-cluster-kubernetes-monitoring-best-practices-cluster-name-field/143990)
**Author:** [@Yoav\_Zuri](https://discuss.elastic.co/u/Yoav_Zuri)\ **Replies:** 2\ **Last updated:** [August 14, 2018, 4:37pm UTC](https://discuss.elastic.co/t/multi-cluster-kubernetes-monitoring-best-practices-cluster-name-field/143990 "2018-08-14T16:37:31Z")
Hi, I want to monitor multiple Kubernetes Clusters to one ES cluster using Filebeats. Are there any best practices on how to separate the monitored clusters? Can I add custom field to Beats per cluster? So it'll send … --- ## [Packetbeat does not show data when Absolute timestamp is entered?](https://discuss.elastic.co/t/packetbeat-does-not-show-data-when-absolute-timestamp-is-entered/143112)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 1\ **Last updated:** [August 14, 2018, 3:33pm UTC](https://discuss.elastic.co/t/packetbeat-does-not-show-data-when-absolute-timestamp-is-entered/143112 "2018-08-14T15:33:32Z")
Hi Team, I am unable to query the packetbeat when Absolute timestamp is entered. My search returns nothing while it perfectly works with Relative or Quick timestamp. What could be the issue? --- ## [Auditd vs Beats performance](https://discuss.elastic.co/t/auditd-vs-beats-performance/142993)
**Author:** [@nwed](https://discuss.elastic.co/u/nwed)\ **Replies:** 1\ **Last updated:** [August 14, 2018, 3:21pm UTC](https://discuss.elastic.co/t/auditd-vs-beats-performance/142993 "2018-08-14T15:21:19Z")
Hello, I recently setup auditd rules within auditbeat. Awesome solution that will really help us make better sense of audit logs. Obviously a big issue with auditd is performance. We are working on generating metric arou… --- ## [Metricset 'audit/kernel' is not registered, module not found](https://discuss.elastic.co/t/metricset-audit-kernel-is-not-registered-module-not-found/119353)
**Author:** [@rdesanno](https://discuss.elastic.co/u/rdesanno)\ **Replies:** 5\ **Last updated:** [August 14, 2018, 2:58pm UTC](https://discuss.elastic.co/t/metricset-audit-kernel-is-not-registered-module-not-found/119353 "2018-08-14T14:58:05Z")
I'm stumped here. I'm using salt to deploy auditbeat across multiple CentOS7 hosts but found at least one server where auditbeat will not start and giving me the following error: 2018-02-10T19:30:18.733-0500 ERROR insta… --- ## [Kubernetes autodiscover with hints, default/fallback newline from annotations](https://discuss.elastic.co/t/kubernetes-autodiscover-with-hints-default-fallback-newline-from-annotations/143241)
**Author:** [@havlan](https://discuss.elastic.co/u/havlan)\ **Replies:** 6\ **Last updated:** [August 14, 2018, 12:22pm UTC](https://discuss.elastic.co/t/kubernetes-autodiscover-with-hints-default-fallback-newline-from-annotations/143241 "2018-08-14T12:22:51Z")
I was wondering if setting annotations with a newline and other options will override default config for that specific event. If a pod does not have those annotations, will it get default options from filebeat config fil… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=426) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=428)