Hi there,
following scenery:
We have a log producer running on a machine which which has internet access.
We have a logstash instance which is connected to elasticsearch in a different which has currently no internet access.
Logfile will be quite small, I assume about 20-50 MB per day.
ELK and logfile producer are in different networks with no access between them.
Since the log producer may write sensitive data inside the logs, I need to set up a secure way to shipper the logs.
We have following ideas and we ask for best practice:
- create a new logstash instance on public webserver which is able to forward to elasticsearch.
pro:
- server running log producer can directly shipper the the logs to logstash
con: - logstash needs to be hardened. Is it possible to secure it by user and password? TLS is not enough, because anyone else may send data to it and may flood / expose logstash
- create secure ftp server with tls and user password. create task on producer side which pushes the logfile to webserver. On Webserver a filebeat / logstash forwarder will run to move the logs to internal logstash / elk.
- pro: seems to be secure
- con: is it possible with ftp to append the file, that logstash / filebeat is not detecting a truncated logfile each time it is updated?
-
VPN between producer and webserver, which runs logstash instance
pro: secure if logstash is only accessible via vpn
con: more complex architecture because of need to setup a vpn -
Any other Idea we don't have in mind yet.
What is the best practice?
Thanks, Andreas