Can't find Metricbeat log file

[cylon86]

Hello,

I'm trying to setup metricbeat to monitor my cluster but I have some trouble. The first one is that I can't find the metricbeat execution logs to debug the issue...

When I run systemctl status metricbeat.service I can see that the service is running but there are some errors:

metricbeat.service - Metricbeat is a lightweight shipper for metrics.
   Loaded: loaded (/lib/systemd/system/metricbeat.service; disabled; vendor preset: enabled)
   Active: active (running) since Sat 2020-04-04 16:12:30 UTC; 2s ago
     Docs: https://www.elastic.co/products/beats/metricbeat
 Main PID: 17509 (metricbeat)
    Tasks: 16 (limit: 2361)
   CGroup: /system.slice/metricbeat.service
           └─17509 /usr/share/metricbeat/bin/metricbeat -e -c /etc/metricbeat/metricbeat.yml -path.home /usr/share/metricbeat -path.config /etc/metricbeat -path.data /var/lib/metricbeat -path.logs /var/log/metricbeat

Apr 04 16:12:30 ip-10-0-0-22 metricbeat[17509]: 2020-04-04T16:12:30.158Z        INFO        add_cloud_metadata/add_cloud_metadata.go:93        add_cloud_metadata: hosting provider type detected as aws, metadata={"account":{"id":"032854191254"},"availability_zone":"eu-west
Apr 04 16:12:32 ip-10-0-0-22 metricbeat[17509]: 2020-04-04T16:12:32.166Z        INFO        module/wrapper.go:252        Error fetching data for metricset elasticsearch.ccr: error determining if connected Elasticsearch node is master: error making http request: Get https:
Apr 04 16:12:32 ip-10-0-0-22 metricbeat[17509]: 2020-04-04T16:12:32.166Z        INFO        module/wrapper.go:252        Error fetching data for metricset elasticsearch.index_summary: error determining if connected Elasticsearch node is master: error making http request:
Apr 04 16:12:32 ip-10-0-0-22 metricbeat[17509]: 2020-04-04T16:12:32.166Z        INFO        module/wrapper.go:252        Error fetching data for metricset elasticsearch.index: error determining if connected Elasticsearch node is master: error making http request: Get http
Apr 04 16:12:32 ip-10-0-0-22 metricbeat[17509]: 2020-04-04T16:12:32.166Z        INFO        module/wrapper.go:252        Error fetching data for metricset elasticsearch.shard: error determining if connected Elasticsearch node is master: error making http request: Get http
Apr 04 16:12:32 ip-10-0-0-22 metricbeat[17509]: 2020-04-04T16:12:32.167Z        INFO        module/wrapper.go:252        Error fetching data for metricset elasticsearch.node_stats: error making http request: Get https://elasticsearch.node1.com:9200/_nodes/_local/stats: di
Apr 04 16:12:32 ip-10-0-0-22 metricbeat[17509]: 2020-04-04T16:12:32.167Z        INFO        module/wrapper.go:252        Error fetching data for metricset elasticsearch.ml_job: error determining if connected Elasticsearch node is master: error making http request: Get htt
Apr 04 16:12:32 ip-10-0-0-22 metricbeat[17509]: 2020-04-04T16:12:32.167Z        INFO        module/wrapper.go:252        Error fetching data for metricset elasticsearch.cluster_stats: error determining if connected Elasticsearch node is master: error making http request:
Apr 04 16:12:32 ip-10-0-0-22 metricbeat[17509]: 2020-04-04T16:12:32.167Z        INFO        module/wrapper.go:252        Error fetching data for metricset elasticsearch.enrich: error determining if connected Elasticsearch node is master: error making http request: Get htt
Apr 04 16:12:32 ip-10-0-0-22 metricbeat[17509]: 2020-04-04T16:12:32.167Z        INFO        module/wrapper.go:252        Error fetching data for metricset elasticsearch.index_recovery: error determining if connected Elasticsearch node is master: error making http request:

I would like to check the log file to have the full error message but in the /var/log/metricbeat I only have this:

2020-04-04T15:39:55.106Z	INFO	instance/beat.go:622	Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat]. 
2020-04-04T15:39:55.107Z	INFO	instance/beat.go:630	Beat ID: 6f497682-9759-4632-bfb4-ec4aea766a80

Also in the systemctl status metricbeat.service message I can see that the log file is indeed suppose to be in /var/log/metricbeat:

17509 /usr/share/metricbeat/bin/metricbeat -e -c /etc/metricbeat/metricbeat.yml -path.home /usr/share/metricbeat -path.config /etc/metricbeat -path.data /var/lib/metricbeat -path.logs /var/log/metricbeat

So where are my logs ?

Thanks for helping

PS: I'm on ubuntu and installed metricbeat using .deb package

[weltenwort (Felix Stürmer)]

Hi @cylon86,

the -e in the service command tells metricbeat to log to stderr (equivalent to setting logging.to_stderr: true). That means you'll find the output in your system journal, which you see section of below your systemctl status metricbeat call. To see the full stdout and stderr of the service you can query the journal using journalctl -u metricbeat.service.

Alternatively you should be able to configure metricbeat to also log everything to the file by setting logging.to_files: true.