Hi,
I set up ECK on GKE and spun up Elasticsearch, Kibana and Fleet server with ingress Nginx along with Letsencrypt certs. Everything works perfect expect agent is checking in every 10 mins. It is trying connect but fails then getting successful again.
16:39:54.702
elastic_agent
[elastic_agent][warn] Possible transient error during checkin with fleet-server, retrying
16:43:23.220
elastic_agent
[elastic_agent][error] Cannot checkin in with fleet-server, retrying
16:49:54.222
elastic_agent
[elastic_agent][error] Cannot checkin in with fleet-server, retrying
16:57:17.594
elastic_agent
[elastic_agent][error] Cannot checkin in with fleet-server, retrying
17:12:15.585
elastic_agent
[elastic_agent][error] Cannot checkin in with fleet-server, retrying
17:20:56.014
elastic_agent
[elastic_agent][error] Checkin request to fleet-server succeeded after 6 failures
17:20:57.041
elastic_agent
[elastic_agent][info] Settings action done, setting agent log level to debug
17:20:57.041
elastic_agent
[elastic_agent][debug] Successfully dispatched action: 'action_id: e55b3440-63b3-4e20-8c6d-bdd7ff3e887a, type: SETTINGS, log_level: debug'
17:20:57.057
elastic_agent
[elastic_agent][info] Updating running component model
17:20:57.063
elastic_agent
[elastic_agent][debug] Updating running component model
17:20:57.065
elastic_agent
[elastic_agent][debug] observed component update for endpoint service
17:20:57.076
elastic_agent
[elastic_agent][info] Unit state changed system/metrics-default-system/metrics-system-28f301a0-15f1-11ee-aff6-81ca3d114f61 (HEALTHY->CONFIGURING): Configuring
17:20:57.076
elastic_agent
[elastic_agent][info] Unit state changed system/metrics-default (HEALTHY->CONFIGURING): Configuring
17:20:57.076
elastic_agent
[elastic_agent][info] Unit state changed beat/metrics-monitoring-metrics-monitoring-beats (HEALTHY->CONFIGURING): Configuring
17:20:57.076
elastic_agent
[elastic_agent][info] Unit state changed beat/metrics-monitoring (HEALTHY->CONFIGURING): Configuring
17:20:57.076
elastic_agent
[elastic_agent][info] Unit state changed filestream-monitoring (HEALTHY->CONFIGURING): Configuring
17:20:57.076
elastic_agent
[elastic_agent][info] Unit state changed filestream-monitoring-filestream-monitoring-agent (HEALTHY->CONFIGURING): Configuring
17:20:57.076
elastic_agent
[elastic_agent][info] Unit state changed log-default (HEALTHY->CONFIGURING): Configuring
17:20:57.076
elastic_agent
[elastic_agent][info] Unit state changed log-default-logfile-system-28f301a0-15f1-11ee-aff6-81ca3d114f61 (HEALTHY->CONFIGURING): Configuring
17:20:57.076
elastic_agent
[elastic_agent][info] Unit state changed http/metrics-monitoring-metrics-monitoring-agent (HEALTHY->CONFIGURING): Configuring
17:20:57.076
elastic_agent
[elastic_agent][info] Unit state changed http/metrics-monitoring (HEALTHY->CONFIGURING): Configuring
17:20:57.211
elastic_agent
[elastic_agent][debug] FleetGateway calling Checkin API
17:20:57.211
elastic_agent
[elastic_agent][debug] Checking started
17:20:57.318
elastic_agent
[elastic_agent][debug] observed check-in for endpoint service: token:"12b10651-06e3-458f-bc48-509a4ffe344b" units:{id:"endpoint-default-b6bfdc00-15f2-11ee-9fef-6b22721aa321" config_state_idx:2 state:HEALTHY message:"Applied policy {b6bfdc00-15f2-11ee-9fef-6b22721aa321}" payload:{fields:{key:"error" value:{struct_value:{fields:{key:"code" value:{number_value:0}} fields:{key:"message" value:{string_value:"Success"}}}}}}} units:{id:"endpoint-default" type:OUTPUT config_state_idx:1 state:HEALTHY message:"Applied policy {b6bfdc00-15f2-11ee-9fef-6b22721aa321}" payload:{fields:{key:"error" value:{struct_value:{fields:{key:"code" value:{number_value:0}} fields:{key:"message" value:{string_value:"Success"}}}}}}} version_info:{name:"Endpoint" version:"8.8.1"} features:{source:{fields:{key:"agent" value:{struct_value:{fields:{key:"features" value:{struct_value:{fields:{key:"fqdn" value:{struct_value:{fields:{key:"enabled" value:{bool_value:false}}}}}}}}}}}} fqdn:{}} features_idx:1
17:20:57.431
elastic_agent
[elastic_agent][debug] using previously saved ack token: 8a87eee5-4e55-4207-a3d8-8bb42f5c0b9c
17:20:57.432
elastic_agent
[elastic_agent][debug] Request method: POST, path: /api/fleet/agents/c162c539-0273-4779-8ac0-1286eff3ec8e/checkin, reqID: 01H41Y302Q1K1MW0BR8D5NT0H3
17:20:57.432
elastic_agent
[elastic_agent][debug] Creating new request to request URL https://x.com:443/api/fleet/agents/c162c539-0273-4779-8ac0-1286eff3ec8e/checkin?
17:20:58.071
elastic_agent
[elastic_agent][info] Unit state changed log-default-logfile-system-28f301a0-15f1-11ee-aff6-81ca3d114f61 (CONFIGURING->HEALTHY): Healthy
17:20:58.071
elastic_agent
[elastic_agent][info] Unit state changed log-default (CONFIGURING->HEALTHY): Healthy
17:20:58.080
elastic_agent
[elastic_agent][info] Unit state changed http/metrics-monitoring-metrics-monitoring-agent (CONFIGURING->HEALTHY): Healthy
17:20:58.080
elastic_agent
[elastic_agent][info] Unit state changed http/metrics-monitoring (CONFIGURING->HEALTHY): Healthy
17:20:58.080
elastic_agent
[elastic_agent][info] Unit state changed filestream-monitoring (CONFIGURING->HEALTHY): Healthy
17:20:58.081
elastic_agent
[elastic_agent][info] Unit state changed filestream-monitoring-filestream-monitoring-agent (CONFIGURING->HEALTHY): Healthy
17:20:58.081
elastic_agent
[elastic_agent][info] Unit state changed beat/metrics-monitoring-metrics-monitoring-beats (CONFIGURING->HEALTHY): Healthy
17:20:58.081
elastic_agent
[elastic_agent][info] Unit state changed beat/metrics-monitoring (CONFIGURING->HEALTHY): Healthy
17:20:58.081
elastic_agent
[elastic_agent][info] Unit state changed system/metrics-default-system/metrics-system-28f301a0-15f1-11ee-aff6-81ca3d114f61 (CONFIGURING->HEALTHY): Healthy
17:20:58.081
elastic_agent
[elastic_agent][info] Unit state changed system/metrics-default (CONFIGURING->HEALTHY): Healthy
17:21:10.453
elastic_agent
[elastic_agent][debug] observed check-in for endpoint service: token:"12b10651-06e3-458f-bc48-509a4ffe344b" units:{id:"endpoint-default-b6bfdc00-15f2-11ee-9fef-6b22721aa321" config_state_idx:2 state:HEALTHY message:"Applied policy {b6bfdc00-15f2-11ee-9fef-6b22721aa321}" payload:{fields:{key:"error" value:{struct_value:{fields:{key:"code" value:{number_value:0}} fields:{key:"message" value:{string_value:"Success"}}}}}}} units:{id:"endpoint-default" type:OUTPUT config_state_idx:1 state:HEALTHY message:"Applied policy {b6bfdc00-15f2-11ee-9fef-6b22721aa321}" payload:{fields:{key:"error" value:{struct_value:{fields:{key:"code" value:{number_value:0}} fields:{key:"message" value:{string_value:"Success"}}}}}}} version_info:{name:"Endpoint" version:"8.8.1"} features:{source:{fields:{key:"agent" value:{struct_value:{fields:{key:"features" value:{struct_value:{fields:{key:"fqdn" value:{struct_value:{fields:{key:"enabled" value:{bool_value:false}}}}}}}}}}}} fqdn:{}} features_idx:1
17:21:30.454
elastic_agent
[elastic_agent][debug] observed check-in for endpoint service: token:"12b10651-06e3-458f-bc48-509a4ffe344b" units:{id:"endpoint-default-b6bfdc00-15f2-11ee-9fef-6b22721aa321" config_state_idx:2 state:HEALTHY message:"Applied policy {b6bfdc00-15f2-11ee-9fef-6b22721aa321}" payload:{fields:{key:"error" value:{struct_value:{fields:{key:"code" value:{number_value:0}} fields:{key:"message" value:{string_value:"Success"}}}}}}} units:{id:"endpoint-default" type:OUTPUT config_state_idx:1 state:HEALTHY message:"Applied policy {b6bfdc00-15f2-11ee-9fef-6b22721aa321}" payload:{fields:{key:"error" value:{struct_value:{fields:{key:"code" value:{number_value:0}} fields:{key:"message" value:{string_value:"Success"}}}}}}} version_info:{name:"Endpoint" version:"8.8.1"} features:{source:{fields:{key:"agent" value:{struct_value:{fields:{key:"features" value:{struct_value:{fields:{key:"fqdn" value:{struct_value:{fields:{key:"enabled" value:{bool_value:false}}}}}}}}}}}} fqdn:{}} features_idx:1