Hello everyone,
I've been using ES Stack for quite a while but I'm still struggling to figure out what's the best architecture to ingest logs for a given use case.
I've developed an application customers have to install in their local computers (Windows and Android) to get access to my service, I'm still wondering what're the best practices to gather the logs this app produces locally in their devices.
Is there any service inside the ES ecosystem that exposes an endpoint I could use in this apps to ship the logs into? Or do I have to wrap an endpoint in my Backend API to act as a middleware to ship the logs into ES?
My necessities are:
- Be able to protect this endpoint or at least the way these logs are ingested to avoid DDoS attacks and dirty logs in case anyone starts to attack this endpoint.
- Restrict what this endpoint can do in terms of operations and targeted indexes.
- Be able to blacklist IPs in case they are not doing a well use of the endpoint.
- If the application adoption starts to grow to tens of thousands of clients this architecture would have to support the load.
I know the use of Filebeat and different agents but considering the computers this app will run are not mine I don't see reasonable to inject these agents into the installation package of my application.
I hope it's clear enough and thanks in advance everyone
