Proxying beats outputs

Elastic Stack Beats
1

Hi community

I want to send elasticsearch.output through an nginx proxy.

If i set that nginx IP i get:
2017-03-28T10:50:51-04:00 ERR Connecting error publishing events (retrying): Failed to parse JSON response: invalid character '<' looking for beginning of value

if i set ES host actual IP all works ok.

Idea is to obfuscate ES endpoint from the remote machines where beats are running

Thanks!

2

Hi @Mihai-CMM,

Can you share your NGINX settings? Also, what's the output in NGINX log?

3

Hi Carlos,
Actually i think I've just managed to get it work. Not quite what I would like but i think it works like this. I needed to use proxy_url

  1. In metricbeat.yml I've added this lines:

output.elasticsearch:
# Array of hosts to connect to.
hosts: ["browser.shows.this.server.com"]
index: metricbeat-client1
proxy_url: "54.77.26.243/client1" # that's NGINX on AWS

In nginx i have this:

sudo cat /etc/nginx/conf.d/reverseproxy.conf
server {
listen 80;
server_name browser.shows.this.server.com;
location / {
proxy_set_header Host https://{my ELK domain} - west-1.es.amazonaws.com:443;
proxy_pass https://{my ELK domain}-west-1.es.amazonaws.com:443;
}

location /client1 {
proxy_set_header Host https://{my ELK domain} - west-1.es.amazonaws.com:443;
proxy_pass https://{my ELK domain}-west-1.es.amazonaws.com:443;

}

}

Thanks again for looking at this. Will probably follow with other topics on securing this deployment and how to leverage using elastic cloud not AWS elasticsearch