# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=38 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 39 --- ## [Sending output to different indices depending on conditions](https://discuss.elastic.co/t/sending-output-to-different-indices-depending-on-conditions/343423)
**Author:** [@DetlefG](https://discuss.elastic.co/u/DetlefG)\ **Replies:** 0\ **Last updated:** [September 20, 2023, 7:31am UTC](https://discuss.elastic.co/t/sending-output-to-different-indices-depending-on-conditions/343423 "2023-09-20T07:31:17Z")
Hi, I try to write logs via filebeat to different indices depending on a field in the logs. But I'm not sure, how the rule setting when is working. Is it correct, that if the first when condition is fullfilled the seco… --- ## [Fleet-server and elastic-agent metricbeat x509 unknown CA on kubernetes](https://discuss.elastic.co/t/fleet-server-and-elastic-agent-metricbeat-x509-unknown-ca-on-kubernetes/343279)
**Author:** [@Eric-Domeier](https://discuss.elastic.co/u/Eric-Domeier)\ **Replies:** 1\ **Last updated:** [September 19, 2023, 9:30pm UTC](https://discuss.elastic.co/t/fleet-server-and-elastic-agent-metricbeat-x509-unknown-ca-on-kubernetes/343279 "2023-09-19T21:30:04Z")
Environment details Kubernetes cluster RKE2 v1.27.3 with DISA STIG's ECK Operator: 2.9.0 (Ironbank image) Elastic Agent Image: 8.9.0 (Ironbank image) Issue: After getting the pod(s) fleet server and agents to a runn… --- ## [Unable to authenticate user \[elastic\] for REST request \[/\]](https://discuss.elastic.co/t/unable-to-authenticate-user-elastic-for-rest-request/343393)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 1\ **Last updated:** [September 19, 2023, 8:14pm UTC](https://discuss.elastic.co/t/unable-to-authenticate-user-elastic-for-rest-request/343393 "2023-09-19T20:14:00Z")
Hello, I'm trying to configure logs for my Elasticsearch cluster, by following this: and even though i set verification\_mode to none, i still getting 401 {"log.level":"error","@timestamp":"2023-09-19T19:13:02.623Z… --- ## [Pipeline date\_time parser failure beats me (sorry for the pun :)](https://discuss.elastic.co/t/pipeline-date-time-parser-failure-beats-me-sorry-for-the-pun/343380)
**Author:** [@stefws](https://discuss.elastic.co/u/stefws)\ **Replies:** 4\ **Last updated:** [September 19, 2023, 5:30pm UTC](https://discuss.elastic.co/t/pipeline-date-time-parser-failure-beats-me-sorry-for-the-pun/343380 "2023-09-19T17:30:51Z")
I'm ingesting Redhat AMQ log with filebeat, only filebeat claims the ingest pipeline fails to parse date time of every event. But testing a sample event/document from fiebeat log, pipeline works fine, that beats me. Hint… --- ## [FileBeat filestream ndjson breaking array with nested objects](https://discuss.elastic.co/t/filebeat-filestream-ndjson-breaking-array-with-nested-objects/343383)
**Author:** [@dusatvoj](https://discuss.elastic.co/u/dusatvoj)\ **Replies:** 0\ **Last updated:** [September 19, 2023, 3:07pm UTC](https://discuss.elastic.co/t/filebeat-filestream-ndjson-breaking-array-with-nested-objects/343383 "2023-09-19T15:07:30Z")
Hello, I have ndjson which is scraped by filebeat, transferred via redis and logstash (which has no filter rule, except date) into elasticsearch. The ndjson structure is smth like: { "array\_of\_objects": \[ { "a… --- ## [Add\_field processor on empty env provider fields stop ingest](https://discuss.elastic.co/t/add-field-processor-on-empty-env-provider-fields-stop-ingest/343371)
**Author:** [@sholzhauer](https://discuss.elastic.co/u/sholzhauer)\ **Replies:** 0\ **Last updated:** [September 19, 2023, 1:36pm UTC](https://discuss.elastic.co/t/add-field-processor-on-empty-env-provider-fields-stop-ingest/343371 "2023-09-19T13:36:06Z")
Hi, Our nodes have some attributes to define what asset they belong to (environment, application, component). With migrating to agent these fields got lost and we have utilized the environment provider and the add\_field… --- ## [Clarification regarding filebeat and metricbeat support policy](https://discuss.elastic.co/t/clarification-regarding-filebeat-and-metricbeat-support-policy/342940)
**Author:** [@ishaq](https://discuss.elastic.co/u/ishaq)\ **Replies:** 3\ **Last updated:** [September 19, 2023, 1:04pm UTC](https://discuss.elastic.co/t/clarification-regarding-filebeat-and-metricbeat-support-policy/342940 "2023-09-19T13:04:53Z")
Hey :wave: I've been going over the docs and this forum for an official version support policy for metricbeat and filebeat but I have not had any luck yet. I'd be grateful if someone could link me to it, if it exists. I… --- ## [Filestream take\_over mode seems to be ignored](https://discuss.elastic.co/t/filestream-take-over-mode-seems-to-be-ignored/343220)
**Author:** [@gparks](https://discuss.elastic.co/u/gparks)\ **Replies:** 1\ **Last updated:** [September 19, 2023, 1:09am UTC](https://discuss.elastic.co/t/filestream-take-over-mode-seems-to-be-ignored/343220 "2023-09-19T01:09:06Z")
I'm running filebeat 8.8.2 on centos 7 I'm in the process of switching from log inputs to filestream inputs on pre-existing servers so I was trying to use the take\_over mode in order to not re-process the logs. I'm not… --- ## [Ironbank Elastic Agent 8.9.0 Issues - tinit, group writeable components](https://discuss.elastic.co/t/ironbank-elastic-agent-8-9-0-issues-tinit-group-writeable-components/343274)
**Author:** [@Eric-Domeier](https://discuss.elastic.co/u/Eric-Domeier)\ **Replies:** 0\ **Last updated:** [September 18, 2023, 2:28pm UTC](https://discuss.elastic.co/t/ironbank-elastic-agent-8-9-0-issues-tinit-group-writeable-components/343274 "2023-09-18T14:28:02Z")
Hello, Environment information Kubernetes RKE2 Cluster v1.27.3 (DISA STIG Hardened) Ironbank ECK-operator image 2.9.0 I managed to get the agents running and report a "Healthy" status however wanted to post here to m… --- ## [Why my filebeat settings can only read /var/log/messages. I need to read all files in /var/log and my customized log folder: /suselv/log](https://discuss.elastic.co/t/why-my-filebeat-settings-can-only-read-var-log-messages-i-need-to-read-all-files-in-var-log-and-my-customized-log-folder-suselv-log/343232)
**Author:** [@huanghaiqing1](https://discuss.elastic.co/u/huanghaiqing1)\ **Replies:** 0\ **Last updated:** [September 18, 2023, 8:20am UTC](https://discuss.elastic.co/t/why-my-filebeat-settings-can-only-read-var-log-messages-i-need-to-read-all-files-in-var-log-and-my-customized-log-folder-suselv-log/343232 "2023-09-18T08:20:27Z")
filebeat.yml part about included logs --\> type: filestream Unique ID among all inputs, an ID is required. id: autoyast1-filestream Change to true to enable this input configuration. enabled: true Paths that should … --- ## [Filebeat queue.disk keeps piling up even when Logstash persisted queue remains relatively empty](https://discuss.elastic.co/t/filebeat-queue-disk-keeps-piling-up-even-when-logstash-persisted-queue-remains-relatively-empty/343221)
**Author:** [@sergeyarl](https://discuss.elastic.co/u/sergeyarl)\ **Replies:** 0\ **Last updated:** [September 18, 2023, 6:40am UTC](https://discuss.elastic.co/t/filebeat-queue-disk-keeps-piling-up-even-when-logstash-persisted-queue-remains-relatively-empty/343221 "2023-09-18T06:40:03Z")
Hi! So we are using the following chain: Filebeats, that run in a K8s cluster (1 Filebeat instance on each k8s worker node) -\> 2 Logstash nodes behind AWS ALB -\> Elastic search cluster Everything works pretty well. … --- ## [Winlogbeat unable to start due to error](https://discuss.elastic.co/t/winlogbeat-unable-to-start-due-to-error/343190)
**Author:** [@risshukla](https://discuss.elastic.co/u/risshukla)\ **Replies:** 0\ **Last updated:** [September 17, 2023, 9:23am UTC](https://discuss.elastic.co/t/winlogbeat-unable-to-start-due-to-error/343190 "2023-09-17T09:23:28Z")
We've been using Winlogbeat to forward Workstation logs to Logstash. However, we've encountered an issue after installing Winlogbeat (versions 8.9.2 and 8.10.0) on our Windows Server 2022. The issue is as follows: Exce… --- ## [Filebeat query EKS worker node /var/log](https://discuss.elastic.co/t/filebeat-query-eks-worker-node-var-log/342745)
**Author:** [@xUmaRix](https://discuss.elastic.co/u/xUmaRix)\ **Replies:** 2\ **Last updated:** [September 17, 2023, 3:38am UTC](https://discuss.elastic.co/t/filebeat-query-eks-worker-node-var-log/342745 "2023-09-17T03:38:12Z")
Hi, I'm trying to ship EKS worker node auth.log, syslog and audit.log files which located under /var/log. I've deploy filebeat and logstash in EKS cluster however I saw under filebeat pods there's a lot of error log s… --- ## [\[Netflow\] Issues with Module](https://discuss.elastic.co/t/netflow-issues-with-module/343158)
**Author:** [@Rossana](https://discuss.elastic.co/u/Rossana)\ **Replies:** 0\ **Last updated:** [September 15, 2023, 7:16pm UTC](https://discuss.elastic.co/t/netflow-issues-with-module/343158 "2023-09-15T19:16:02Z")
Hi! I try to avoid the use of netflow codec of logstash, cause i understand that is deprecated. I configure Netflow Module on filbeat. But does not work. I also configure as an input but still does not work. Netflow … --- ## [Metricbeat module Apache error - error fetching data: HTTP error 404 in : 404 Not Found](https://discuss.elastic.co/t/metricbeat-module-apache-error-error-fetching-data-http-error-404-in-404-not-found/343078)
**Author:** [@Carlos\_T](https://discuss.elastic.co/u/Carlos_T)\ **Replies:** 2\ **Last updated:** [September 15, 2023, 6:24pm UTC](https://discuss.elastic.co/t/metricbeat-module-apache-error-error-fetching-data-http-error-404-in-404-not-found/343078 "2023-09-15T18:24:41Z")
Hi all I having problems when trying to get metrics from my Apache installation running on a Centos 7 VM Env: ECK 2.6.1 1 ES Master Node 8.6.2 running on a single node K3S Kubernetes Cluster installed on Centos 7 Ser… --- ## [Custom filebeat docker image error](https://discuss.elastic.co/t/custom-filebeat-docker-image-error/343140)
**Author:** [@matheuscirillo](https://discuss.elastic.co/u/matheuscirillo)\ **Replies:** 1\ **Last updated:** [September 15, 2023, 1:05pm UTC](https://discuss.elastic.co/t/custom-filebeat-docker-image-error/343140 "2023-09-15T13:05:45Z")
The custom image configuration section on the docs says that we can do the following to create a customized image: FROM docker.elastic.co/beats/filebeat:8.10.0 COPY --chown=root:filebeat filebeat.yml /usr/share/filebeat… --- ## [Filebeat not sending docker logs to logstash after enabling x-pack security features](https://discuss.elastic.co/t/filebeat-not-sending-docker-logs-to-logstash-after-enabling-x-pack-security-features/343100)
**Author:** [@Ajai\_Raj](https://discuss.elastic.co/u/Ajai_Raj)\ **Replies:** 0\ **Last updated:** [September 15, 2023, 5:02am UTC](https://discuss.elastic.co/t/filebeat-not-sending-docker-logs-to-logstash-after-enabling-x-pack-security-features/343100 "2023-09-15T05:02:01Z")
Please help me in this i've been trying to create a user in ELK after enabling the x-pack security feature in my elasticsearch.yml file. The docker container logs are not syncing in kibana after i enable the security fea… --- ## [How should I configure HAProxy logging to make it work with Filebeat?](https://discuss.elastic.co/t/how-should-i-configure-haproxy-logging-to-make-it-work-with-filebeat/342547)
**Author:** [@jerrac](https://discuss.elastic.co/u/jerrac)\ **Replies:** 1\ **Last updated:** [September 14, 2023, 4:21pm UTC](https://discuss.elastic.co/t/how-should-i-configure-haproxy-logging-to-make-it-work-with-filebeat/342547 "2023-09-14T16:21:46Z")
The Filebeat docs do not give any instructions on what format to use. The Elastic Agent Integration docs mention support for the default, tcplog, httplog, httpslog, and errorlog formats. So far though, I only get "Provi… --- ## [I want to use Beats to generate files with event-type data to later analyze on my own](https://discuss.elastic.co/t/i-want-to-use-beats-to-generate-files-with-event-type-data-to-later-analyze-on-my-own/343062)
**Author:** [@Oscar\_Llerena](https://discuss.elastic.co/u/Oscar_Llerena)\ **Replies:** 0\ **Last updated:** [September 14, 2023, 3:04pm UTC](https://discuss.elastic.co/t/i-want-to-use-beats-to-generate-files-with-event-type-data-to-later-analyze-on-my-own/343062 "2023-09-14T15:04:42Z")
Hello everyone, I am new to Elastic Search and I've been referred to this solution to solve the following problem. However, given the tons of documentation, I barely know where to start. Here is my problem: I want to … --- ## [I want only get Disk Usage of multiple path with metricbeat](https://discuss.elastic.co/t/i-want-only-get-disk-usage-of-multiple-path-with-metricbeat/343055)
**Author:** [@martel](https://discuss.elastic.co/u/martel)\ **Replies:** 0\ **Last updated:** [September 14, 2023, 1:41pm UTC](https://discuss.elastic.co/t/i-want-only-get-disk-usage-of-multiple-path-with-metricbeat/343055 "2023-09-14T13:41:39Z")
Hey, I want only disk usage for some path. example : du -sh /home/applicationA/data du -sh /home/applicationB/data 9.1G /home/applicationA/data 2.5G /home/applicationB/data how do that ? i dont want /home t… --- ## [Filebeat: send data from a dynamic log + static file version.txt in one event](https://discuss.elastic.co/t/filebeat-send-data-from-a-dynamic-log-static-file-version-txt-in-one-event/343033)
**Author:** [@john123](https://discuss.elastic.co/u/john123)\ **Replies:** 0\ **Last updated:** [September 14, 2023, 9:27am UTC](https://discuss.elastic.co/t/filebeat-send-data-from-a-dynamic-log-static-file-version-txt-in-one-event/343033 "2023-09-14T09:27:00Z")
Hi, we have a dynamic log 'app.log' and a static file version.txt with th version of the app. We have to send both as a single event with the purpose to have a trace of the errors in function of changing version of the … --- ## [Elastic Logging Plugin SSL certificates issue](https://discuss.elastic.co/t/elastic-logging-plugin-ssl-certificates-issue/343021)
**Author:** [@Vladimir7172](https://discuss.elastic.co/u/Vladimir7172)\ **Replies:** 0\ **Last updated:** [September 14, 2023, 7:23am UTC](https://discuss.elastic.co/t/elastic-logging-plugin-ssl-certificates-issue/343021 "2023-09-14T07:23:48Z")
Hello! I'm trying to transfer logs from a docker container to ELK using elastic-logging-plugin:8.9.2 On the Elastic side I see this type of error: "error.message":"javax.net.ssl.SSLHandshakeException: Received fatal a… --- ## [Help me - The speed of filebeat collection cannot keep up with the speed of file writing](https://discuss.elastic.co/t/help-me-the-speed-of-filebeat-collection-cannot-keep-up-with-the-speed-of-file-writing/343017)
**Author:** [@evanzhang87](https://discuss.elastic.co/u/evanzhang87)\ **Replies:** 0\ **Last updated:** [September 14, 2023, 6:35am UTC](https://discuss.elastic.co/t/help-me-the-speed-of-filebeat-collection-cannot-keep-up-with-the-speed-of-file-writing/343017 "2023-09-14T06:35:50Z")
My log writing rate is about 3M/s, single log input, my output is kafka, I checked the metrics, pipeline.events.active keeps 4118, {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":1024565234}},"mem… --- ## [Filebeat unable to collect logs from 'nodeSelector' deployment](https://discuss.elastic.co/t/filebeat-unable-to-collect-logs-from-nodeselector-deployment/343011)
**Author:** [@Achu](https://discuss.elastic.co/u/Achu)\ **Replies:** 0\ **Last updated:** [September 14, 2023, 5:58am UTC](https://discuss.elastic.co/t/filebeat-unable-to-collect-logs-from-nodeselector-deployment/343011 "2023-09-14T05:58:25Z")
I’m experiencing a peculiar issue with Filebeat. I can collect logs from all deployments except a couple of deployments that have a node selection. Filebeat Daemonset is running on this node, and I don’t see any errors f… --- ## [Read the current date file in filebeat](https://discuss.elastic.co/t/read-the-current-date-file-in-filebeat/342726)
**Author:** [@Golam\_Rabbi](https://discuss.elastic.co/u/Golam_Rabbi)\ **Replies:** 4\ **Last updated:** [September 12, 2023, 7:08pm UTC](https://discuss.elastic.co/t/read-the-current-date-file-in-filebeat/342726 "2023-09-12T19:08:19Z")
I have some custom log files for my company. The log file naming pattern is like this "api\_datalogger\_11-09-23". Here 11-09-23 means that the log file of September 11, 2023. Now I want to set my filebeat inputs to read t… --- ## [Anyone have an easy way to make Metricbeat use Time Series Data Streams (TSDS)?](https://discuss.elastic.co/t/anyone-have-an-easy-way-to-make-metricbeat-use-time-series-data-streams-tsds/342375)
**Author:** [@jerrac](https://discuss.elastic.co/u/jerrac)\ **Replies:** 4\ **Last updated:** [September 12, 2023, 3:17pm UTC](https://discuss.elastic.co/t/anyone-have-an-easy-way-to-make-metricbeat-use-time-series-data-streams-tsds/342375 "2023-09-12T15:17:44Z")
Long story short, we need to shrink our resource usage with our Elastic Stack. Part of my attempts at that has been implementing down sampling. But, after finally reading the docs carefully enough, I found out that we ne… --- ## [Filebeat registry/log.json size keeps increasing though there are no new log entries in my application log](https://discuss.elastic.co/t/filebeat-registry-log-json-size-keeps-increasing-though-there-are-no-new-log-entries-in-my-application-log/342757)
**Author:** [@palansk](https://discuss.elastic.co/u/palansk)\ **Replies:** 1\ **Last updated:** [September 12, 2023, 3:14pm UTC](https://discuss.elastic.co/t/filebeat-registry-log-json-size-keeps-increasing-though-there-are-no-new-log-entries-in-my-application-log/342757 "2023-09-12T15:14:15Z")
Filebeat is making entry to log.json file even though are no new logs being added to my application log file. Below is the excerpt of the log.json file {"k":"filebeat::logs::native::670096-64768","v":{"ttl":-1,"FileSta… --- ## [The metric beat index size is incrasing rapidly.can we remove some of the fields present in the index](https://discuss.elastic.co/t/the-metric-beat-index-size-is-incrasing-rapidly-can-we-remove-some-of-the-fields-present-in-the-index/342804)
**Author:** [@Ambikaguntu](https://discuss.elastic.co/u/Ambikaguntu)\ **Replies:** 1\ **Last updated:** [September 12, 2023, 12:10pm UTC](https://discuss.elastic.co/t/the-metric-beat-index-size-is-incrasing-rapidly-can-we-remove-some-of-the-fields-present-in-the-index/342804 "2023-09-12T12:10:42Z")
My metricbeat default field brings in too much unnecessary data. Can i remove the fields in the index what I need. I have removed the Metricbeat processor to drop fields in the metricbeat configuaration .Still there are… --- ## [Is it possible to get only the types of fields I want from metricbeat?](https://discuss.elastic.co/t/is-it-possible-to-get-only-the-types-of-fields-i-want-from-metricbeat/342770)
**Author:** [@20wjsdudtj](https://discuss.elastic.co/u/20wjsdudtj)\ **Replies:** 2\ **Last updated:** [September 12, 2023, 9:12am UTC](https://discuss.elastic.co/t/is-it-possible-to-get-only-the-types-of-fields-i-want-from-metricbeat/342770 "2023-09-12T09:12:43Z")
My metricbeat default field brings in too much unnecessary data. (For example, kubernetes. Kubernetes-related data such as pod.name, uid, namespace.., etc. There are only a few data I need. Can I specify and import this?… --- ## [Error pipeline/output.go:180 failed to publish events: temporary bulk send failure](https://discuss.elastic.co/t/error-pipeline-output-go-180-failed-to-publish-events-temporary-bulk-send-failure/342788)
**Author:** [@Sevde\_Nur\_Canli](https://discuss.elastic.co/u/Sevde_Nur_Canli)\ **Replies:** 0\ **Last updated:** [September 12, 2023, 8:09am UTC](https://discuss.elastic.co/t/error-pipeline-output-go-180-failed-to-publish-events-temporary-bulk-send-failure/342788 "2023-09-12T08:09:56Z")
Hello I recently got the following error, pipeline/output.go:180 failed to publish events: temporary bulk send failure I tried everything to solve this problem and look every info in the internet but no solution still. … [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=37) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=39)