I am new in ELK. I have installed filebeat on machines on AWS to collect logs and send them to local ELK server.
I find out the log on the server are being collected in certain pattern like this
From my understanding the is the bar graph in the tab of discover in kibana showing how many files have been collected in a certain period of time and i have changed the @timestamp to equal to the timestamp of the log so this graph is showing if the log generated at certain time have been collected or not collected.
As you can see, many logs generated at the beginning of each hour have been collected, on the hand, most log generated at the end of each hour have not been collected.
Our log is stored on a new log file each hour.
Is it normal? It seems to me that filebeat collect log from multiple log files in the same folder at the same time. Is it possible to configure it to collect log from single file only at the same time in order to collect faster?