# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=206 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 207 --- ## [Filebeat/elasticsearch create daily index](https://discuss.elastic.co/t/filebeat-elasticsearch-create-daily-index/251532)
**Author:** [@Jack\_Daniels](https://discuss.elastic.co/u/Jack_Daniels)\ **Replies:** 2\ **Last updated:** [October 13, 2020, 3:54am UTC](https://discuss.elastic.co/t/filebeat-elasticsearch-create-daily-index/251532 "2020-10-13T03:54:41Z")
Hi, im trying create per day index with filebeat. Can you help me? Here is part of filebeat.yml file output.elasticsearch: # Array of hosts to connect to. hosts: \["test2019:9200"\] pipeline: "logpipe… --- ## [S3 Filebeat Input error](https://discuss.elastic.co/t/s3-filebeat-input-error/246654)
**Author:** [@haseev](https://discuss.elastic.co/u/haseev)\ **Replies:** 4\ **Last updated:** [October 12, 2020, 9:09pm UTC](https://discuss.elastic.co/t/s3-filebeat-input-error/246654 "2020-10-12T21:09:10Z")
There is an error within filebeat plugin after integrating aws sqs with filebeat. The error occured after receivng this message: handleSQSMessage failed: json unmarshal sqs message body failed: invalid character 'e' in … --- ## [Filebeat breaks with error data path already locked by another beat](https://discuss.elastic.co/t/filebeat-breaks-with-error-data-path-already-locked-by-another-beat/251807)
**Author:** [@Joe\_Cool2016](https://discuss.elastic.co/u/Joe_Cool2016)\ **Replies:** 0\ **Last updated:** [October 12, 2020, 7:50pm UTC](https://discuss.elastic.co/t/filebeat-breaks-with-error-data-path-already-locked-by-another-beat/251807 "2020-10-12T19:50:36Z")
How do I get around this error "data path already locked by another beat"? Can you give a yml file example for windows? --- ## [7.9.2 FileBeat CloudTrail AWS Module Changes Break Log Collection](https://discuss.elastic.co/t/7-9-2-filebeat-cloudtrail-aws-module-changes-break-log-collection/251661)
**Author:** [@djcullen](https://discuss.elastic.co/u/djcullen)\ **Replies:** 2\ **Last updated:** [October 12, 2020, 6:34pm UTC](https://discuss.elastic.co/t/7-9-2-filebeat-cloudtrail-aws-module-changes-break-log-collection/251661 "2020-10-12T18:34:26Z")
It appears that the changes made to the FileBeat AWS module in 7.9.2 (specifically 21086) break the module for anyone whose CloudTrail S3 object key does not conform to the following regex: ^AWSLogs/\\d+/CloudTrail/ My … --- ## [Suricata-IDS and ELK](https://discuss.elastic.co/t/suricata-ids-and-elk/251725)
**Author:** [@hack3rcon](https://discuss.elastic.co/u/hack3rcon)\ **Replies:** 3\ **Last updated:** [October 12, 2020, 5:15pm UTC](https://discuss.elastic.co/t/suricata-ids-and-elk/251725 "2020-10-12T17:15:42Z")
Hello, I have a Suricata-IDS server and I want to see the logs by ELK. I don't like to install the ELK package on my Suricata-IDS server. I want to know is "Beats" enough for my Suricata-IDS server? How can I harden "Be… --- ## [Filebeat - Dissect Message String](https://discuss.elastic.co/t/filebeat-dissect-message-string/251435)
**Author:** [@iccMe](https://discuss.elastic.co/u/iccMe)\ **Replies:** 3\ **Last updated:** [October 12, 2020, 11:15am UTC](https://discuss.elastic.co/t/filebeat-dissect-message-string/251435 "2020-10-12T11:15:50Z")
Hi, I am looking for advise on how to use the processor-\> dissect within Filebeat for a log file. Below is an example of the log file date: \[08/10/2020 09:31:57\] servername - Processor Queue Ok 3 WMI (localhost:Pro… --- ## [Unable to get Filebeat setup correctly](https://discuss.elastic.co/t/unable-to-get-filebeat-setup-correctly/251651)
**Author:** [@jonckvanderkogel](https://discuss.elastic.co/u/jonckvanderkogel)\ **Replies:** 7\ **Last updated:** [October 12, 2020, 9:09am UTC](https://discuss.elastic.co/t/unable-to-get-filebeat-setup-correctly/251651 "2020-10-12T09:09:15Z")
Hi everyone, I'm trying to get Filebeat set up on a local Mac Mini so it will ingest log files of a process to Elasticsearch. The log files contain a custom JSON format. I got it sort of working but in Elasticsearch the… --- ## [Auto-upgrade Elastic Agent to bugfix releases](https://discuss.elastic.co/t/auto-upgrade-elastic-agent-to-bugfix-releases/250845)
**Author:** [@nemhods](https://discuss.elastic.co/u/nemhods)\ **Replies:** 3\ **Last updated:** [October 12, 2020, 7:38am UTC](https://discuss.elastic.co/t/auto-upgrade-elastic-agent-to-bugfix-releases/250845 "2020-10-12T07:38:27Z")
Hi there, I've set the agent upgrading setting in the Ingest Manager to Automatically update agent binaries to use the latest minor version.. My stack is on 7.9.2, and I have agents on 7.9.0 and 7.9.1. Those probably w… --- ## [Filebeat is unable to send logs into kafka](https://discuss.elastic.co/t/filebeat-is-unable-to-send-logs-into-kafka/251674)
**Author:** [@Ayush\_Agrahari](https://discuss.elastic.co/u/Ayush_Agrahari)\ **Replies:** 4\ **Last updated:** [October 12, 2020, 5:36am UTC](https://discuss.elastic.co/t/filebeat-is-unable-to-send-logs-into-kafka/251674 "2020-10-12T05:36:56Z")
Earlier we were using Filebeat to logstash setup for logs but to make sure no log is lost during down time so we thought to use kafka in between of logstash and filebeat so that kafka would save logs incase of down time. … --- ## [Filebeat multiline message display](https://discuss.elastic.co/t/filebeat-multiline-message-display/251386)
**Author:** [@vadrix](https://discuss.elastic.co/u/vadrix)\ **Replies:** 2\ **Last updated:** [October 12, 2020, 2:56am UTC](https://discuss.elastic.co/t/filebeat-multiline-message-display/251386 "2020-10-12T02:56:41Z")
Hi.. i am trying to create multiline for "t message" on elasticsearch but it's always showing sing line for each log on elasticsearch this from elasticsearch and this from logs file on filebeat.inputs i am us… --- ## [If enable ILM in config file, then cannot set custom elastic output index](https://discuss.elastic.co/t/if-enable-ilm-in-config-file-then-cannot-set-custom-elastic-output-index/251307)
**Author:** [@tonysue](https://discuss.elastic.co/u/tonysue)\ **Replies:** 7\ **Last updated:** [October 10, 2020, 5:45pm UTC](https://discuss.elastic.co/t/if-enable-ilm-in-config-file-then-cannot-set-custom-elastic-output-index/251307 "2020-10-10T17:45:24Z")
setup.template.overwrite: true output.elasticsearch.index: "winlogbeat=%{\[agent.version\]}"-city-firm-hostname-%{+yyyy,MM,dd}-000001" setup.template.name: "winlogbeat" setup.template.pattern: "winlogbeat-\*" setup.ilm.… --- ## [Logging to BOTH cloud and onpremise logstash/elasticsearch?](https://discuss.elastic.co/t/logging-to-both-cloud-and-onpremise-logstash-elasticsearch/251653)
**Author:** [@Porton](https://discuss.elastic.co/u/Porton)\ **Replies:** 2\ **Last updated:** [October 11, 2020, 6:09am UTC](https://discuss.elastic.co/t/logging-to-both-cloud-and-onpremise-logstash-elasticsearch/251653 "2020-10-11T06:09:25Z")
Hi is it possible to send hearbeats to BOTH cloud and on-premise logstash/elasticsearch from one hearbeat application ? Currently running 2 hearbeat applications, one is sending heartbeat to cloud and another is sending… --- ## [FIlebeat terminating || service/service.go:56 Received sighup, stopping](https://discuss.elastic.co/t/filebeat-terminating-service-service-go-56-received-sighup-stopping/250595)
**Author:** [@Boris\_Joseph](https://discuss.elastic.co/u/Boris_Joseph)\ **Replies:** 3\ **Last updated:** [October 10, 2020, 11:22am UTC](https://discuss.elastic.co/t/filebeat-terminating-service-service-go-56-received-sighup-stopping/250595 "2020-10-10T11:22:05Z")
Hi, I am using filebeats 7.9.2 to send log data + internal collection metrics from our prod environment servers(Linux box) to ELK Stack (7.9.2) via 3 node Kafka Cluster(2.6.0). But every time the filebeat stops/termina… --- ## [Problem with decode\_json\_fields](https://discuss.elastic.co/t/problem-with-decode-json-fields/251399)
**Author:** [@Kay\_Khan](https://discuss.elastic.co/u/Kay_Khan)\ **Replies:** 4\ **Last updated:** [October 9, 2020, 12:42pm UTC](https://discuss.elastic.co/t/problem-with-decode-json-fields/251399 "2020-10-09T12:42:42Z")
I have ECK setup and im using filebeat to ship logs from Kubernetes to elasticsearch. Ive recently added decode\_json\_fields processor to my configuration, so that im able decode the json that is usually in the message … --- ## [Why cant i get k8s event?](https://discuss.elastic.co/t/why-cant-i-get-k8s-event/250320)
**Author:** [@HarveyYang](https://discuss.elastic.co/u/HarveyYang)\ **Replies:** 7\ **Last updated:** [October 9, 2020, 7:04am UTC](https://discuss.elastic.co/t/why-cant-i-get-k8s-event/250320 "2020-10-09T07:04:17Z")
here is my metricbeat yaml: --- apiVersion: v1 kind: ConfigMap metadata: name: metricbeat-daemonset-config namespace: kube-system labels: k8s-app: metricbeat data: metricbeat.yml: |- metricbeat.config.mo… --- ## [How to exclude other namespaces?](https://discuss.elastic.co/t/how-to-exclude-other-namespaces/246544)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 23\ **Last updated:** [October 9, 2020, 2:39am UTC](https://discuss.elastic.co/t/how-to-exclude-other-namespaces/246544 "2020-10-09T02:39:05Z")
filebeat.autodiscover: providers: - type: kubernetes node: ${NODE\_NAME} hints.enabled: false hints.default\_config: type: container finished: true paths: - "/var/… --- ## [Metricbeat + AWS Module: runtime error: index out of range \[0\] with length 0](https://discuss.elastic.co/t/metricbeat-aws-module-runtime-error-index-out-of-range-0-with-length-0/248592)
**Author:** [@dudumiquim](https://discuss.elastic.co/u/dudumiquim)\ **Replies:** 4\ **Last updated:** [October 8, 2020, 11:49pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-runtime-error-index-out-of-range-0-with-length-0/248592 "2020-10-08T23:49:16Z")
I'm trying to configure the AWS module but when I try to start the metricbeat, I receive this error: FATAL \[metricbeat\] instance/beat.go:164 Failed due to panic This error are not "too clear" and I started my metricb… --- ## [Import Audibeat dashboard into Kibana](https://discuss.elastic.co/t/import-audibeat-dashboard-into-kibana/251453)
**Author:** [@Thomas74](https://discuss.elastic.co/u/Thomas74)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 2:00pm UTC](https://discuss.elastic.co/t/import-audibeat-dashboard-into-kibana/251453 "2020-10-08T14:00:28Z")
Hi, I configured my cluster to integrate Auditbeat logs. How I can import, in a specific space, auditbeat dashboards into Kibana, without to set configuration into auditbeat.yml ? Can I find dashboards somewhere and im… --- ## [Winlogbeat Sysmon Registry events missing event.category](https://discuss.elastic.co/t/winlogbeat-sysmon-registry-events-missing-event-category/251282)
**Author:** [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 1:14pm UTC](https://discuss.elastic.co/t/winlogbeat-sysmon-registry-events-missing-event-category/251282 "2020-10-08T13:14:35Z")
Hello, My Sysmon registry events (Registry object added or deleted (rule: RegistryEvent)) seem to be missing an event.category field. (7.8) What event.category should be given to these Sysmon events? Grtz Willem --- ## [Filebeats to elasticsearch ingress documentation](https://discuss.elastic.co/t/filebeats-to-elasticsearch-ingress-documentation/251384)
**Author:** [@sillencem](https://discuss.elastic.co/u/sillencem)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 9:54am UTC](https://discuss.elastic.co/t/filebeats-to-elasticsearch-ingress-documentation/251384 "2020-10-08T09:54:54Z")
When using the config from here: https://www.elastic.co/guide/en/beats/filebeat/current/configuring-ingest-node.html output.elasticsearch: hosts: \["localhost:9200"\] pipeline: "test-pipeline" the ingress wasn't used… --- ## [K8s metricbeat not consistently recognizing metric endpoints](https://discuss.elastic.co/t/k8s-metricbeat-not-consistently-recognizing-metric-endpoints/246574)
**Author:** [@jonas27](https://discuss.elastic.co/u/jonas27)\ **Replies:** 15\ **Last updated:** [October 8, 2020, 6:55am UTC](https://discuss.elastic.co/t/k8s-metricbeat-not-consistently-recognizing-metric-endpoints/246574 "2020-10-08T06:55:28Z")
Hello, I experienced that after restarting metricbeat in kubernetes it recognizes different pods to scan Recognizes fine after restart: Only recognizing before: Only recognizing after: I have checked that all … --- ## [Metricbeat writes data randomly, instead every 10 sec](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865)
**Author:** [@Razby](https://discuss.elastic.co/u/Razby)\ **Replies:** 12\ **Last updated:** [October 8, 2020, 6:51am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865 "2020-10-08T06:51:04Z")
Hi, We got a problem. Metricbeat writes data randomly even though the period=10s in case the query instance is set to all "\*". The data are recorded randomly for specific instance, but not every 10 seconds. If I set t… --- ## [Metricbeat have default parameter](https://discuss.elastic.co/t/metricbeat-have-default-parameter/250633)
**Author:** [@jonas27](https://discuss.elastic.co/u/jonas27)\ **Replies:** 1\ **Last updated:** [October 8, 2020, 6:51am UTC](https://discuss.elastic.co/t/metricbeat-have-default-parameter/250633 "2020-10-08T06:51:03Z")
Hi, Is there a way to set a config params as optional. I want to give our freedom to set the period (scrape interval) optionally. What I tried so far was this templates: - condition.and: - equ… --- ## [Filebeat multiline pattern](https://discuss.elastic.co/t/filebeat-multiline-pattern/251274)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 2\ **Last updated:** [October 8, 2020, 5:22am UTC](https://discuss.elastic.co/t/filebeat-multiline-pattern/251274 "2020-10-08T05:22:44Z")
In the picture stacktrace of log message taking the new log line I want them in one log. How to do that ? Here is my log example : 2020-10-01 16:39:18.4120 ERROR LeadsController bhavin http://bhavin/Leads/New POS… --- ## [Auditbeat, Filebeat error: system/socket dataset setup failed: unable to monitor probe p:inet6\_create inet6\_create proto={{.P3}}:s32: perf\_event\_open: too many open files](https://discuss.elastic.co/t/auditbeat-filebeat-error-system-socket-dataset-setup-failed-unable-to-monitor-probe-p-inet6-create-inet6-create-proto-p3-perf-event-open-too-many-open-files/251210)
**Author:** [@bilal\_abudan](https://discuss.elastic.co/u/bilal_abudan)\ **Replies:** 3\ **Last updated:** [October 8, 2020, 4:44am UTC](https://discuss.elastic.co/t/auditbeat-filebeat-error-system-socket-dataset-setup-failed-unable-to-monitor-probe-p-inet6-create-inet6-create-proto-p3-perf-event-open-too-many-open-files/251210 "2020-10-08T04:44:13Z")
Can someone help me please ... ??? i am using Filebeat and Auditbeat, Lets have Auditbeat one, This issue when i started the auditbeat service and theres error below : Exiting: 1 error: system/socket dataset setup fa… --- ## [Packetbeat not resolving and connecting to kibana in Daemonset](https://discuss.elastic.co/t/packetbeat-not-resolving-and-connecting-to-kibana-in-daemonset/251359)
**Author:** [@MANI\_M](https://discuss.elastic.co/u/MANI_M)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 1:12am UTC](https://discuss.elastic.co/t/packetbeat-not-resolving-and-connecting-to-kibana-in-daemonset/251359 "2020-10-08T01:12:32Z")
With this as reference I created a daemonset for packetbeat. It seems kibana is not getting resolved properly due to which packet beat keeps failing. Here is the packetbeat.yaml file for reference: --- apiVersion: v1 k… --- ## [Filebeat installation question?](https://discuss.elastic.co/t/filebeat-installation-question/251140)
**Author:** [@pacy1](https://discuss.elastic.co/u/pacy1)\ **Replies:** 5\ **Last updated:** [October 7, 2020, 10:15pm UTC](https://discuss.elastic.co/t/filebeat-installation-question/251140 "2020-10-07T22:15:40Z")
where to install filebeat, after i finish to installing elsticsearch ,logstash and kibana. how to configure logstash to use log file downloaded. Kindly assist me on this, Thank you in advance --- ## [Send event logs to pipeline based on event\_id](https://discuss.elastic.co/t/send-event-logs-to-pipeline-based-on-event-id/251150)
**Author:** [@tedfs](https://discuss.elastic.co/u/tedfs)\ **Replies:** 5\ **Last updated:** [October 7, 2020, 2:12pm UTC](https://discuss.elastic.co/t/send-event-logs-to-pipeline-based-on-event-id/251150 "2020-10-07T14:12:21Z")
Hi there. I am trying to send specific windows event logs to a pipeline based on event\_id in my winlogbeat.yml config file. However, when the conditional is added to the config file, it appears to bypass the pipeline com… --- ## [Filebeat: input: awscloudwatch (expression pattern for log\_group)](https://discuss.elastic.co/t/filebeat-input-awscloudwatch-expression-pattern-for-log-group/250781)
**Author:** [@srolskyi](https://discuss.elastic.co/u/srolskyi)\ **Replies:** 2\ **Last updated:** [October 7, 2020, 2:09pm UTC](https://discuss.elastic.co/t/filebeat-input-awscloudwatch-expression-pattern-for-log-group/250781 "2020-10-07T14:09:07Z")
I have many log\_groups in clodwatch ~ 400. But I need parse only 77 from this long list. Can I grab events from log group use some pattern? For example: log\_group\_arn: arn:aws:logs:us-east-1:0000000000:log-group:/dev… --- ## [Filebeat service failing to start](https://discuss.elastic.co/t/filebeat-service-failing-to-start/251199)
**Author:** [@aviationfan](https://discuss.elastic.co/u/aviationfan)\ **Replies:** 8\ **Last updated:** [October 7, 2020, 2:03pm UTC](https://discuss.elastic.co/t/filebeat-service-failing-to-start/251199 "2020-10-07T14:03:27Z")
Ever since updating to 7.9.2, filebeat has failed to start as a service. Ubuntu 20.04.1 LTS (Focal Fossa) systemctl status filebeat ● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=205) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=207)