Metricbeats reported version mismatch

Elastic Stack Beats
1

Hello!

I'm installing beats v. 8.5.1 on my machines. When I query installed version with /var/lib/metricbeat/metricbeat version, it reports itself as 8.5.1. However in Humio where I ship the metrics, Agent version is popping up as 8.4.3 – a version we never installed anywhere in our environment.

"agent":
{
"name":"some-host",
"id":"some_id",
"ephemeral_id":"another_id",
"type":"metricbeat",
"version":"8.4.3"

Any ideas why? Is this expected?

2

Hi @serge1,

Is there any change you're running two instances of Metricbeat?

Some things you can try:

  1. run ps aux | grep metricbeat to make sure there is only one instance running.
  2. Look the IDs in the logs and compare with the ones you see in the event. When Metricbeat starts it logs its IDs (I suppressed a few lines on this example):
{"log.level":"info","@timestamp":"2022-12-13T09:57:25.876+0100","log.origin":{"file.name":"instance/beat.go","file.line":716},"message":"Beat ID: f466ed8f-15ea-4948-ae6c-0521c3bf95f9","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-12-13T09:57:25.894+0100","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1082},"message":"Beat info","service.name":"metricbeat","system_info":{"beat":{"path":{"config":"/home/tiago/devel/beats/metricbeat","data":"/home/tiago/devel/beats/metricbeat/data","home":"/home/tiago/devel/beats/metricbeat","logs":"/home/tiago/devel/beats/metricbeat/logs"},"type":"metricbeat","uuid":"f466ed8f-15ea-4948-ae6c-0521c3bf95f9"},"ecs.version":"1.6.0"}}
  1. Stop all Metricbeat process and start them again. If for some reason you did an upgrade and the old process is still running, that should solve it.
3

Thanks for your reply!

There's just one instance running. I did get around this by setting allow_older_versions: true and setup.ilm.enabled: false, but have no metrics flowing to elastic. Any ideas at all? Filebeats 8.5.1 seem to be working fine -- I can see logs being shipped. Just metricbeats stopped.

4

some additional data -- locally I can see metrics work with metricbeat -e, no real errors, and all server settings are configured the same between metricbeats and filebeats, and filebeats work OK, so probably not something like token / URL / etc

5

Try running Metricbeat with -e -v -d "*", like: ./metricbeat -e -v -d "*" (if you're running it maually), this will log all events sent to the output, compare the agent.* data there with Metricbeat's version and the data you're seeing at the final destination.

6

That will enable debug logging, it's quite verbose, specially if you have a high throughput.

7

thanks for suggestions. I ended up giving up and going back to 7.x which works fine. I'll do some testing on a single machine with verbose output and see if it gives me anything useful.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.