hello,
i want that my elastic agents (in company A) send logs to a reverse proxy in front of my fleet server (in company b) which Is connected directly to my elasticsearch cluster
is there any guide to follow ?
please help
Logs are sent to the output, not to the (or via) Fleet server.
hello lesio thank you for taking the time to answer
okaaay !! i was mistaken so the fleet server control the agents but the logs are sent directly to the Elasticsearch cluster (but in my case through the reverse proxy for both control (ngnix -> fleet) and data (ngnix-> elasticsearch) ) right ?
is there any guide I can follow to deploying this architecture ?
please suggest
Good Diagram here although does not show a proxy
You will need to look closely at the proxy directives on this page
1 Like
Hey
Did you find the solution ?
hello stephen thank so much for taking the time to answer,
sorry for the late reply, i was away from labbing these days
can you provide further information please about the configuration in fleet under kibana ui or an nginx conf file as an example, that would be extremely helpful
I found this guide too: Fleet managed Elastic Agent connectivity using a proxy server | Fleet and Elastic Agent Guide [8.15] | Elastic
should I do these steps to and add the proxy to fleet under kibana UI ?
also, should my conf file in nginx look like this ? :
server {
listen 443 ssl;
server_name company_a.com;
ssl_certificate /etc/nginx/ssl/fleetserver.crt;
ssl_certificate_key /etc/nginx/ssl/fleetserver.key;
location / {
proxy_pass <https://fleetserver.internal_IP:8220>;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
}
}
and the command to deploy and enroll the agent in fleet like this ? :
./elastic-agent install --url=https://company_a.com:443 --enrollment-token=YOUR_ENROLLMENT_TOKEN --proxy-url=https://proxy.example.com:8080 --certificate-authorities=/path/to/ca.crt
please suggest
unfortunatly not yet but i'm still trying once i find the solution amma write the steps here
if you have any suggestions feel free please
@Abdarrahmane Unfortunately, I am not a proxy specialist...
1 Like
no worries Stephen
thank you so much for always taking the time to answer I really appreciate it
amma try my best to find a solution for this and come back with the steps
one last question if you don't mind
usually with elastic agents, do companies who offer soc as a service use this approach of putting a reverse proxy in front of fleet-server or simply put fleet-server in their DMZ and secure it well or even put the fleet-server in the client infrastructure ?
Hard to say ...
Remember Fleet it Command and control...
The actual data does not flow through fleet.
See these diagrams, did you look at them?
Different ingest architectures
Detail of Fleet / Agent Control and Data Flow
1 Like
oh i see i forgot about that 
so the proxy approach is the best I guess it's flexible and secure (Site-to-Site VPN, NAT, Port Forwarding, DMZ setup) are not secure, complex or require additional configurations for all endpoints (On the other hand Dedicated Gateway Server is for advanced deployments)
the Proxy should ensure communication between elastic agent from a side and both elasticsearch and fleet from another side
thank you stephen for your guidance
I'll try my best to come up with the steps for the solution
and if someone has done this before and is reading this please suggest !
hello i finally made it work and here are the steps I followed
1- Installed Nginx on a server that can access both the internet and my internal network (I put it in my DMZ)
2- I obtained an SSL certificate for my domain using Let's Encrypt for example or another certificate authority.
(for the sake of this lab I used elasticsearch cert util)
./bin/elasticsearch-certutil cert \
--name proxy1 \
--ca-cert /usr/share/elasticsearch/converted_ca/cert.crt \
--ca-key /usr/share/elasticsearch/converted_ca/private.key \
--dns proxy1.homelab.lan \
--ip 192.168.2.36 \
--pem
3- I created a new Nginx configuration file sudo nano /etc/nginx/sites-available/elastic_reverse_proxy.conf with the following configuration:
#type all your elasticsearch data and master nodes or even better you coordinating_only nodes if you have them nginx will load balance over them
upstream elasticsearch {
server 192.168.1.14:9200;
server 192.168.1.15:9200;
server 192.168.1.16:9200;
keepalive 15; #change it according to the number of agents you are deploying using this proxy I guess
}
upstream fleet {
server 192.168.1.23:8220; # Fleet server IP and port
keepalive 15; #change it according to the number of agents you are deploying using this proxy I guess
}
server {
listen 443 ssl;
server_name elasticsearch-proxy;
ssl_certificate /etc/nginx/ssl/proxy1.crt;
ssl_certificate_key /etc/nginx/ssl/proxy1.key;
location /api {
proxy_pass https://fleet; # Forward to Fleet server
proxy_http_version 1.1;
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
proxy_ssl_verify off; # Disable SSL verification for testing
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location / {
proxy_pass https://elasticsearch;
proxy_http_version 1.1; # Use HTTP/1.1 to enable keep-alive
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
proxy_ssl_verify off; # Disable SSL verification
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
4- I enabled the new configuration:
sudo ln -s /etc/nginx/sites-available/fleet-server /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx
5- I configured my firewall to allow traffics on ports I used 443 ...
for the sake of this lab let's agree that 192.168.2.36 is my nginx proxy public IP (in production you should update your DNS to point fleet.companya.com to the IP address of your Nginx server...)
6- I then I went to one of my endpoint and installed the elastic agent using:
./elastic-agent install --url=https://192.168.2.36:443 --enrollment-token=my token --insecure
ofc replace 192.168.2.36:443 with your public IP and the enrollment token as well and use --certificate-authorities=/path/to/ca.crt instead of --insecure in production environments
7- I went to check fleet under kibana UI and found this issue:
the agent keeps updating and then goes offline without being healthy at all
so I went to check the logs and found the problem:
thanks to @leandrojmp in Fleet Policy different IP from the server - #3 by leandrojmp
I found that I should add the public IP of my proxy to the output of elasticsearch and fleet server under fleet --> settings like this :
I reenrolled the agent and waited for a few minutes (connection through the proxy is quite slow that's one of the biggest down sides of this approach idk if there is a solution for it (if someone knows please suggest a solution))
and here is the result:
here are some good resources:
Elasticsearch + Kibana behind NGINX reverse proxy with TLS · GitHub /
nginx proxy configuration for elasticsearch · GitHub /
Playing HTTP Tricks with Nginx | Elastic Blog /
for this approach I haven't tried yet I'll certainly do and come with some remarks: Fleet managed Elastic Agent connectivity using a proxy server | Fleet and Elastic Agent Guide [8.15] | Elastic
I hope this help
if you have any remarks or suggestion to improve the performance or security of my approach please feel free to suggest
thank you for taking the time to read !
1 Like