Health Check on FileBeats Instance from AWS NLB

Elastic Stack Beats
1

Hi All,

Could you please help with the below NLB configuration in AWS in-front of FileBeat Instance? When I tried to created the setup, the Target Group always shows unhealthy - Health checks failed.

Requirement:
1. Devices(over UDP) --> 2.NLB --> 3.FileBeat Server (EC2) --> AWS MSK --> LogStash --> Elastic Cloud Elastic Search

1. Devices:
All the n/w devices, like router switches will be configured to send their logs to NLB endpoint over UDP

2. NLB:
NLB should route the incoming UDP traffic to a ec2 instance where elastic filebeat agents are up and running.

3. Filebeat Server(EC2)
Elastic Filebeat is configured to listen to NLB endpoint.

NLB : LB-TST-SOC-LOC-FB-7bf90bf8d5c8f5f4.elb.eu-west-1.amazonaws.com
Port :TCP_UDP: 53

Input configuration in filebeats:
filebeat.inputs:

Below error recorded in the filebeat log while starting the filebeat agent:
2021-04-06T16:50:38.547+0100 ERROR udp/input.go:102 Error running harvester: listen udp 10.178.2.97:53: bind: cannot assign requested address

I could see the Target Group status as unhealthy - Health checks failed.

I have a query related to the FileBeat server running on the target.
The health checks are configured as TCP. Is the filebeat capable of handling TCP health checks originating from the NLB ?
Could you please help?

1 Like
2

I think that your problem is that the AWS NLB can't do a health check using UDP, so you will need to do the health check using TCP or HTTP/HTTPS.

You can do that adding a TCP input that will be used only for this healthcheck, than in your target group you configure the health check to use this port.

Or you can try this experimental feature that exposes the metrics http endpoint, and use this endpoint as the target for the health check.

3

Please don't create multiple topics for the same question, let's continue the discussion in your original thread AWS NLB config infront of FileBeats Instance

1 Like