I'm running both Filebeat 8.3.3 and Metricbeat 8.3.3 on my RHEL 7.9 server, and sending the logs to another server which is hosting Elasticsearch and Kibana.
My filebeat is sending syslog to the ES (I'm simply using this as a connectivity test, I'll be sending Netflow to ES later), and Metricbeat is sending the server's system stats to ES. I want to also send Filebeat's logs to ES using Metricbeat, i.e. the logs written to /var/log/filebeat.
I have configured beat-xpack.yml in Metricbeat, and everything seems to be fine. My question is how do I check if the filebeat logs are in the ES? The default Kibana dashboards that come with Metricbeat seem to only show the server system stats.
I don't see any index related to the filebeat logs under the Discover app. In fact, I don't think any indices are created other than the reserved indices for monitoring and the filebeat-*, metricbeat-* data views.
Could you run GET _cat/indices from dev tools and share the results? Also would be good to see the configuration as text of filebeat and metricbeat. Please make sure to remove any confidential or sensitive data in what you post (passwords, etc).
Why do you want to use Metricbeat to send Filebeat Logs? That is not a normal pattern? What are you trying to accomplish? Metricbeat is not purpose-built to send logs, Filebeat is.
You can use Filebeat to send its own logs.
In the filebeat.yml just setup up
# filestream is an input for collecting log messages from files.
- type: filestream
id: my-filebeat-logs-stream-id
enabled: true
# Path to filebeat logs
paths:
- /var/log/filebeat/*.log
You can also just configure filebeat to write it's logs to syslog then your system logs will pick them up. See here
logging.to_syslog
When true, writes all logging output to the syslog.
I guess in my head, I was thinking filebeat was used to ingest external logs into ES, while metricbeat was used for internal logs. Thanks for clearing that up for me!
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.