Issue with getting information on process

[Faten (Hi)]

Hi,
We are collecting data every 5 sec from the system and Postgres modules that include processes, every 1 min we will get an empty row of the process as the following:
'''
:"2020-02-19T17: 28: 29.721Z","@metadata":{"beat":"metricbeat","type":"_doc","version":"7.5.2"},"user":{"name":"postgres"},"process":{"name":"postgres","pid":2980,"ppid":2762,"pgid":2980,"working_directory":"/var/lib/postgresql/10/main","executable":"/usr/lib/postgresql/10/bin/postgres","args":["postgres: 10/main: writerprocess","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""]}
'''
Is there any configuration needed for us to stop getting empty record?
What could be the cause of this issue? and what should be done in order to stop this behavior?

Best Regards,
Faten

[Mario_Castro]

Hi @Faten

Can you provide a way to reproduce this issue outside of your environment?

[Faten (Hi)]

Hi Mario ,
First of all, Thank you for helping me out!
This issue happens when Metricbeat is connected to Kafka as a data buffer, although While investigation, I change the flow and connected Metricbeat directly to Elasticsearch no change occurred.
The only noted configuration we have in Metricbeat is the following:

1. system.yml:-
   module: system
   period: 5s
   metricsets:
   • cpu
   • core
   • load
   • memory
   • network
   • diskio
   • filesystem
   • fsstat
   • process
   • socket
   • process_summary
   • socket_summary

cpu.metrics: [percentages, normalized_percentages, ticks]

if there is anything else that you need me to provide please let me know.

Thank you again,
Faten

[Mario_Castro]

Thank you for the info. Please, try to format it next time with triple tick at least, this is markdown and it's easier to read.

I don't know if you can do a quick test around this but can you launch it only with the metricset that returns this "weird" data? I guess it's activating only proccess metricset. In order to investigate, we need a narrower scope to look at.

Also, can you tell us the version of Postgres, Metricbeat and Elasticsearch? It will be really nice if you can reproduce this issue in a clean environment (say Docker, for example) because I don't remember to see anything like this before and it might be something unrelated to metricbeat. I have no clue what it might be now, that's why I'm asking you to try to narrow a bit the scope

[Faten (Hi)]

Hi Mario,
again thank you for your input and help!
regarding your question:

1. The environment is a Ubuntu 16.04.4
2. Postgres is 9.5
3. Metricbeat 6.7
4. ElasticSearch 7.3
   As for your suggestion, I will try to produce in a docker environment, and I will update you about it later.
   Just for crusty what is the best rate recommended for sample the process?
   could the rate of sampling cause this issue should it be higher/lower?

Thank you again for putting up with my questions and for your assist

Best Regards,
Faten