I am looking for best practices for following use case:
I have a lot of clients (linux vms) which going to ship logs (and maybe metrics) over logstash to elasticseach cluster. My idea is to create one index per client. In logstash I have now possibility to create one pipeline per each client, that'd mean all clients send to dedicated port and on logstash side I just forward the logs to elasticsearch and define the respective index name. Easy. But at the end I will end up having insane amount of config files and open ports, so I will need somehow to keep the overview and maybe I will run into problems when I am using loadbalancer infront of logstash.
I setup one single pipeline for all vms. All vms send logs to same port and in the pipeline I will somehow seperate the incoming loglines to the respective elasticsearch index, which means it comsumes compute power, makes ingesting slower and I end up having large config file with ifs and elses etc.
I give up the idea of seperation the clients to dedicated indeces and I write all vm logs to one single index, which mean I have no opportunity to grant permissions based on the vm anymore since everything is in one index and when I want to delete logs from a specific machine I would somehow need to delete documents from one big index, which is expensive
Does anyone of you have some advice how this can be solved?
I can cre