I am trying to design the centralized logging structure for our org.
I was looking for a following pipeline :
RemoteServers(FileBeat) => Central Server (LogStash) => ElasticSearch
I want FileBeat to copy files from client servers to a central repository (specific directory) and then Logstash to read from copied files instead of directly pushing data to logstash from FileBeat
I am trying to do this because I want all my log files in a central location and only filtered content will go to elastic search. There may be situations when I will use "grep" command in files on central server (where all log files are present)
So can this role of "scp" be configured by fileBeat ?