Hello Elastic team,
I really need your help on this. I am trying to setup the internal Queue, so when my system is offline for some time and after connecting it back to the network, i could still receive a metricbeat data in Kibana, during the system was offline. I configured everything correct, here is the conf
flush.timeout: 5s queue.spool:
but it can only provide data for only 30 minutes. For example if my system disconnected at 10 am, and i connect it back to the network at 12 pm, i can only only see the data up untill 10:30 am.
I ve also tried configure logstash, but samething there as well. Do you have any idea on how we can fix it ?
Your config snippet is a tad difficult to read, but I would make sure have
queue.spool configured properly and try and set a fairly high file size, which it looks like you have. https://www.elastic.co/guide/en/beats/metricbeat/current/configuring-internal-queue.html#configuration-internal-queue-spool
You might also want to make sure you've set
max_retries properly so you're not dropping events: https://www.elastic.co/guide/en/beats/metricbeat/7.3/elasticsearch-output.html#_literal_max_retries_literal
However, considering your circumstances and the fact that you're disconnected for a number of hours, I would recommend trying to have something between metricbeat and ES that's available for longer periods of time.
Thank you very much for replying me.
I have increased the size file to: 4096MiB and also set max_retries to 500, backoff.max and backoff.init to 120s, but looks like still same output.
What would you recommend to have between metricbeat and ES to help me increase time? I aslo checked the file path where the data while the system is off should go, but coundt find /spool.dat file. I checked the /var/lib/metricbeat and its empty
Anything that can remain up while ES is down that you can use as a "buffer" for extended periods of time.
Are you sure file spooling is enabled? If you check your metricbeat log, it should print a message warning you that it's on. Also, try commenting out all of the
queue.mem config and just leaving
didn't see any messages that it's on. Do i have to do anything to enable spooling ? i configured it and restarted metricbeat service, should't it be enabled after that?
Just commented queue.mem, will try to disconnect again
still same 25 min. i disconnected it at 12pm and connected back at 1 pm, and it only restores data up until 12:25.
I noticed that even i i will comment all spooling configuration, it still remember those 25 minutes. I think queue.spool configuration not working at all.
Can you post your entire metricbeat config using the code formatting with backtics?
Y mean metricbeat.yml file?
Thank you, Alex I figured what was the problem. Apparently it was something with permissions.
The only question i have now is, the size of my /spool.dat file increased by 300MB and after connecting it back to the network, it didn't empty /spool.dat file. How can we set events to immediately remove from the queue?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.