Best secure way of getting remote logs?

Hi there,

following scenery:

We have a log producer running on a machine which which has internet access.
We have a logstash instance which is connected to elasticsearch in a different which has currently no internet access.
Logfile will be quite small, I assume about 20-50 MB per day.
ELK and logfile producer are in different networks with no access between them.

Since the log producer may write sensitive data inside the logs, I need to set up a secure way to shipper the logs.

We have following ideas and we ask for best practice:

  1. create a new logstash instance on public webserver which is able to forward to elasticsearch.
    pro:
  • server running log producer can directly shipper the the logs to logstash
    con:
  • logstash needs to be hardened. Is it possible to secure it by user and password? TLS is not enough, because anyone else may send data to it and may flood / expose logstash
  1. create secure ftp server with tls and user password. create task on producer side which pushes the logfile to webserver. On Webserver a filebeat / logstash forwarder will run to move the logs to internal logstash / elk.
  • pro: seems to be secure
  • con: is it possible with ftp to append the file, that logstash / filebeat is not detecting a truncated logfile each time it is updated?
  1. VPN between producer and webserver, which runs logstash instance
    pro: secure if logstash is only accessible via vpn
    con: more complex architecture because of need to setup a vpn

  2. Any other Idea we don't have in mind yet.

What is the best practice?

Thanks, Andreas

TLS is not enough, because anyone else may send data to it and may flood / expose logstash

This isn't true if you're able to use client certificates that Logstash can verify have been signed by your CA.

is it possible with ftp to append the file, that logstash / filebeat is not detecting a truncated logfile each time it is updated?

Most likely not.

so, is it possible to use client certificates with nxlog / logstash / filebeat? If so, that might be the solution