I am using filebeat to ship logs from a centralized log server. In the prospectors section, I defined multiple files, each with its own type, for example:
-/tmp/manager.log input_type:log document_type: manager.log -/tmp/worker.log input_type:log document_type: worker.log
in the output part, I defined a redis server. I leave the default index "filebeat". This can be configurable but my issue is it's global for all types, so I end up having redis with a single key storing all documents. This is an issue if I want to replay just a single type of log into the system while it's working (hot) because I don't have a proper way of removing only the keys for that log type.
Best case scenario - can I define dynamically a key by document type?
A good solution as well - can I define explicitly a key per type? or path? or something else?