Hi there
we have an ELK STACK in our LAN env (internal environment)
we also have a DMZ env in the organization
we want to install beats (metric winlog file...) on every server in the DMZ env
and we want the output to get to the ELK in the LAN
i read that you can use redis for that ( and configure in the beat YAML a redis output)
but i also read that you can use reverse proxy - BTW - how can we set that up?
are there any other options?
what is the best one?
You probably have different options here. If you write to redis or kafka first, you can use Logstash from your LAN to access redis / kafka to fetch the data. If the reverse proxy setup with the elasticsearch output will work perhaps @steffens can give some more details?
I'd prefer a queueing system like redis/kafka to have a separate single point of contact between the subnetworks. But a reverse proxy should work as well. Advantage of queuing system is, if ES is not reachable, data can still be pushed to queue.
Given you're having 2 separate networks, some consideration about access rights to queueing system or reverse proxy should taken into account.
What if we install logstash on the proxy server (between the dmz and the lan) beats from dmz will write to this logstash (with persistent queue) and this logstash writes directly to ES in the lan (we just open one port between logstash and ES) ?
Is that option preferable?
With the proxy server being crucial for networks to operate correctly, I'd consider putting logstash on another machine within the protected network and simply forward/proxy the TCP connection.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.