I would like to ingest and correlate messages from a number of log files that I have on disk. I have installed Elasticsearch, Kibana and Filebeat. I have configured Filebeat to read the log files from a directory, and I can see some indications of that in the Kibana Discover page.
So, I have signs of life.
There are some basic points that I need guidance on please. I am hoping that there is a suitable tutorial for my use case, i.e. how to ingest and analyse arbitrary log files, that I could be referred to. So, the points below could be answered directly here (thank you), or via a pointer to a tutorial (thanks even more). I know there are lots of tutorials, but none that quite seem to fit my needs.
How do I remove the messages that are showing as a result of previous iterations of changing Filebeat config so that I only see the latest messages. As matters stand now, for example, I am seeing messages that appear to represent a directory listing, which is probably an artefact of an earlier configuration attempt.
How can I see what kind of data is being read from a given log file so that I can see whether Filebeat is making sense of the log file format?
Since Filebeat will likely get confused by some of these log files, what do I about that? For example, should I add a specific content format filter, and, if so, how?