How to read CPU usage, RAM usage and Disk usage using filebeat?

[Vinit_Kumar (Vinit Jordan)]

I am using filebeat, ELK stack. I want to get CPU, RAM, and Disk usage information using filebeat and send it to logstash to elasticsearch to kibana.

Version of ELK stack is:-

filebeat 6.5.4
ELK 5.6.4

Any body have any idea about the solution? Thanks in advance.

[Christian_Dahlqvist]

This type of information is gathered by Metricbeat, not Filebeat.

[shrikantgulia (Shrikant)]

Hello @Vinit_Kumar,

you can use metricbeat and in documentation you can check

https://www.elastic.co/guide/en/beats/metricbeat/6.5/index.html

Best Regards
Shrikant

[Vinit_Kumar (Vinit Jordan)]

Hi @Christian_Dahlqvist and @shrikantgulia,

Thanks for your quick response, I have some questions since long back. I'll heartily appreciate your response on my below question.

1. Is Merticbeat capable to read total RAM size, disk size used and free size and used or free CPU?
2. If I change the filebeat to metricbeat then Should I need to change anything in logstash or elasticsearch?
3. Can we do the same work in merticbeat whatever filebeat is doing?
4. Can we exclude or include the lines in merticbeat?
5. Can we read the same file again in merticbeat? Because I was not able to read same file using filebeat.

Thanks,
Vinit Kumar

[shrikantgulia (Shrikant)]

Hello @Vinit_Kumar,

Yes you fully need to change the metricbeat config
this is the sample metricbeat config file for your use case
go through the documents .

metricbeat.modules:

• module: system
  metricsets:
  • cpu
  • load
  • core
  • fsstat
  • diskio
  • filesystem
  • memory
  • network
  • process
    enabled: true
    period: 50s
    processes: ['.*']
    output.logstash:
    hosts: ['localhost:5044']

[Vinit_Kumar (Vinit Jordan)]

@shrikantgulia Thank you so much again.

Actully I'm new to metricbeat so I'm not able to find where to give input to my multiple custom logs in metricbeat.

[Christian_Dahlqvist]

Custom logs are read by Filebeat, metrics are collected by Metricbeat. You will need to use both and configure them separately.

[Vinit_Kumar (Vinit Jordan)]

Yeah it's working but not according to my acceptations.

My requirement is to see the actual statistics of CPU, RAM and Disk.

[Christian_Dahlqvist]

What statistics are you expecting to see that you do not? Can you explain in a bit more detail exactly what you are after?

[Vinit_Kumar (Vinit Jordan)]

Actually, my system memory is getting used more than 80% and CPU as well while running multiple servers so I am trying to configure these things using ELK stack and metricbeat as you suggest.

But the query which I'm running on kibana Timelion is not showing me anything.

.es(index=apache-info-log, timefield='@timestamp', metric='max:system.memory.actual.used.pct'), .es(index=apache-info-log, timefield='@timestamp', metric='max:system.memory.actual.used.pct').if(gt,0.8,.es(index=apache-info-log, timefield='@timestamp', metric='max:system.memory.actual.used.pct'),null).label('warning').color('#FFCC11'), .es(index=apache-info-log, timefield='@timestamp', metric='max:system.memory.actual.used.pct').if(gt,0.9,.es(index=apache-info-log, timefield='@timestamp', metric='max:system.memory.actual.used.pct'),null).label('severe').color('red')

The response I'm getting is below-

In my Kibana Discover Metricbeat fields coming as question mark (?). Below is screen shot

@Christian_Dahlqvist, I don't know why this not giving any data or error.

[Vinit_Kumar (Vinit Jordan)]

I got the solution for this on another web page regarding question mark "?" on Kibana Discover Metricbeat fields.

But I have another question, should I write the different grok for metricbeat because I have two grok for Apache logs and Jboss logs both are under different if conditions.