What’s the means about this params in the metricbeat log file?

Elastic Stack Beats
1

Hi,

We run some metricbeat in our machines. And collect the run log every 10 mins.
After check the log at /etc/log/metricbeat, we feel puzzled. And we don't find any document about it.

One log record like followed:
fetches.system-core.events=40 fetches.system-core.success=10 fetches.system-cpu.events=10 fetches.system-cpu.success=10 fetches.system-diskio.events=12 fetches.system-diskio.success=2 fetches.system-load.events=10 fetches.system-load.success=10 fetches.system-memory.events=10 fetches.system-memory.success=10 fetches.system-network.events=710 fetches.system-network.success=10 fetches.system-process.events=58 fetches.system-process.success=2 libbeat.es.call_count.PublishEvents=22 libbeat.es.publish.read_bytes=14517 libbeat.es.publish.write_bytes=570384 libbeat.es.published_and_acked_events=850 libbeat.publisher.messages_in_worker_queues=850 libbeat.publisher.published_events=850
I guess the "fetches" means the collector, and the libbeat is the transfer.

"fetches.system-core.events=40" & "fetches.system-core.success=10"
The above means there are 30 event transfer failed to the Elasticsearch?

If it's failed, where can I find the data, or it be dropped?

But the "libbeat.publisher.published_events=850" means all of the event is transfer successful?

Thank you

2

fetches.system-core.events indicates the number of fetches from the system. The number of fetches which is succesful is equals to the value of fetches.system-core.success. The value offetches.system-core.error show how many fetches failed. These events are not forwarded to ES, because there is nothing to be forwarded due to errors. You cannot find that data, because MB hes failed to fetch it from the system.

libbeat.publisher.published_events=850 means that 850 events were published. But it is not yet acknowledged by the output, so it is unknown if the events had made it succesfully to the output. libbeat.es.published_and_acked_events=850 shows that all 850 events were acknowledged, meaning that those made it to ES successfully.

1 Like
3

@kvch
thanks so much,I have learned the meaning of each parameter，but why so many events are failed to fetch ,and how to find out why， could you give me some suggestions?

4

Which Metricbeat version are you using? Do you have log lines which contain e.g fetches.system-core.errors counter?

5

@kvch
My metricbeat version is 5.6.2, and here's a passage of my metricbeat log:


2017-11-24T19:38:46+08:00 INFO Non-zero metrics in the last 5m0s:
 fetches.redis-info.events=5 
fetches.redis-info.success=5 
fetches.system-core.events=40 
fetches.system-core.success=5 
fetches.system-cpu.events=5 
fetches.system-cpu.success=5 
fetches.system-diskio.events=4 
fetches.system-diskio.success=1 
fetches.system-load.events=5 
fetches.system-load.success=5 
fetches.system-memory.events=5 
fetches.system-memory.success=5 
fetches.system-network.events=610 
fetches.system-network.success=5 
fetches.system-process.events=53 
fetches.system-process.success=1 
libbeat.es.call_count.PublishEvents=5 
libbeat.es.publish.read_bytes=5071 
libbeat.es.publish.read_errors=5 
libbeat.es.publish.write_bytes=55089 
libbeat.es.published_and_acked_events=727 
libbeat.publisher.messages_in_worker_queues=727 
libbeat.publisher.published_events=727

thanks.
6

I don't think that there is an error in your metirbeat or host. The sum of fetches.*.events equals the sum of libbeat.es.published_and_acked_events. If you don't have a log line which includes fetches.system-core.errors, it means that it is 0, so there were not any errors.
If you don't see error field in your metricbeat events, it works correctly.

7

@kvch
I see, thank you so much.:grinning:

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.