Configuration of Filebeat pushing logs


(Atul Patel) #1

We are using ELK stack with Filebeat pushing the logs to the Centeralized elastic search.

Is anyone suggest how to inform filebeat to push the logs file based on the timestamp .How should i configure in the yml file so that filebeat will push the logs based on the timestamp


(Magnus B├Ąck) #2

I don't understand what "push the log based on the timestamp" means. Do you want to push the logs at specific point in time rather than push them continuously as new log messages arrive?


(Atul Patel) #3

HI
Pls find the real problem
We have 2
sets of logs /folderA/app.log and /folderB/app1.log.filebeat is pushing
the logs from these folders. we want to push them
continuously as new
log messages arrive but in sequential order which will be based on
timestamp of logs getting updated in different files.

Let say we have 2 files
and both are getting within fraction of millisecond difference , So while
receiving log line at logstash we want to maintain the logs to be received in same sequence.
Which is not happening currently in filebeat.


(Steffen Siering) #4

filebeat does not support parsing logs + does not implement sorting/correlation features. This is fully out of scope of filebeat. Not sure if this can be done with logstash.


(system) #5