Kibana tries to connect to 169.254.169.254:80

Elastic Stack Kibana
1

Hello elastic community,
any idea why Kibana tries to connect to 169.254.169.254:80 for first 5-6 minutes after I start it?
I noticed this with version 7.17.9, but I think it was like this at least for all 7.17.X versions.

2

Hi @nisow95612. I'm not aware of Kibana trying to connect to this address. But I see that the add_cloud_metadata Beats processor can reach out to this address.

See also these links.

3

Yes, this IP address seems to be a "cloud metadata URL" or something.
But I am not aware of any configuration that would cause kibana need it.

It is a very standard kibana server installed from the official repository:
deb https://artifacts.elastic.co/packages/7.x/apt stable main

There are no plugins installed:

# su kibana -s /bin/sh -c "bin/kibana-plugin list"
No plugins installed.

But whenever I systemctl restart elasticsearch, these connections show up.

Also, wouldn't Beats processor run on elasticsearch ingest nodes?
When I restart elasticserach nodes they dont try to connect to 169.254.169.254.

4

Ah, I am wrong. Kibana does use this endpoint for telemetry collection. I believe if you opt out of telemetry (i.e. telemetry.optIn: false in your "kibana.yml") it would stop these outbound calls.

1 Like
5

Well, thank you for the answer.
I was hoping that telemetry.sendUsageFrom: "browser" would be enough, but it isn't.
I tried disabling telemetry in Kibana advanced settings, but it wasn't enough.
I tried setting telemetry.optIn: false, but it wasn't enough.

I found that telemetry.optIn and telemetry.allowChangingOptInStatus cannot be both false at the same time, because Kibana will refuse to start with misleading error:
FATAL Error: [config validation of [telemetry].optIn]: expected value to equal [true]
I was worried it is a licensed feature, but it turns out that Telemetry settings for 7.17 are just different.

So I tried setting the 7.x-only telemetry.enabled: false, but it still has no effect.

PS: I am running the free/basic. I pushed for getting the Gold subscription, but it got discontinued.

1 Like
6

Oh, I didn't notice this line before. This is happening when you start Elasticsearch, not Kibana? If so, we should maybe move this to the Elasticsearch Discuss forum for a more appropriate audience?

1 Like
7

Thank you for spotting it. Unfortunately it is a typo on my side. I was already thinking about next paragraph where I mention this not happening when I restart elasticsearch.

To be clear:
When I systemctl restart kibana on the kibana server, the kibana server generates this traffic
When I systemctl restart elasticsearch on the elasticsearch servers, there is no such traffic.
When I systemctl restart elasticsearch on the kibana server, it gives "unit not found error"

8

Hmm, I'm not able to recreate this. Where are you seeing these requests being made? In the log file?

Ultimately, if you're not running Kibana on a cloud service VM like GCS, AWS, or Azure, these requests are to Link-local addresses meaning they are likely not accessing any external networks.

9

Yes. I would agree that request to a link-local address should not reach the internet. Unfortunately manufactures of all routers on the way turned out to disagree and exactly that happens: I am seeing these request on my internet uplink.

Since you repeatedly mention cloud it may be relevant that my Kibana server is a linux-KVM VM?

10

I confirmed that this connections attempt comes from the NodeJS/Kibana process:

# netstat -Wapnt | grep 169
tcp        0      1 10.x.y.z:45802     169.254.169.254:80      SYN_SENT    32440/node