Elastic-Agent healthy but sends no logs

Elastic Stack Beats
,
1

Hi,

Im trying to use Elastic Agent on my Windows 10 machine. I have sucessfully enrolled the machine and can see it marked as healty in the agent overview. However there is no log data delivered from the agent and also no Data Streams. I also noticed that it temporarily goes to offline even though the machine is in use.

In the elastic-agent-json.log on the host I see multiple entries:

{"log.level":"error","@timestamp":"2021-11-20T14:51:38.663Z","log.origin":{"file.name":"fleet/fleet_gateway.go","file.line":205},"message":"Could not communicate with fleet-server Checking API will retry, error: status code: 400, fleet-server returned an error: BadRequest","ecs.version":"1.6.0"}

And in the fleet-agent log on kibana I see:

15:29:52.922 elastic_agent.fleet_server [elastic_agent.fleet_server][debug] must generate api key as default API key is not present
15:29:52.922 elastic_agent.fleet_server [elastic_agent.fleet_server][debug] Generating a new API key
15:29:52.923 elastic_agent.fleet_server [elastic_agent.fleet_server][error] fail generate output key

There seems to be a problem with the creation of API keys.
I have the option xpack.encryptedSavedObjects.encryptionKey set to a random string of 40 chars, and kibana has elasticsearch.username set to "kibana_system" which should have the permissions to create API keys.

Edit 1:
I just tried executing C:\"Program Files"\Elastic\Agent\elastic-agent.exe inspect and got the following message. I checked the troubleshooting guide but my problem persists.

Error: no fleet config retrieved yet
For help, please see our troubleshooting guide at https://www.elastic.co/guide/en/fleet/7.15/fleet-troubleshooting.html

Edit 2:
I was able to record the failing request with Burp

Request 
POST /api/fleet/agents/<agent_id>/checkin? HTTP/2
Host: <hostname>:8220
Authorization: ApiKey <key>
User-Agent: Elastic Agent v7.15.2
Content-Type: application/json
Accept: application/json
Kbn-Xsrf: 1
Content-Length: 1324
Accept-Encoding: gzip, deflate

{
  "status": "online",
  "events": [],
  "local_metadata": {
    "elastic": {
      "agent": {
        "id": "<agent_id>",
        "version": "7.15.2",
        "snapshot": false,
        "build.original": "7.15.2 (build: fd322dad6ceafec40c84df4d2a0694ea357d16cc at 2021-11-04 17:59:02 +0000 UTC)",
        "upgradeable": true,
        "log_level": "info"
      }
    },
    "host": {
      "architecture": "x86_64",
      "hostname": "<hostname>",
      "name": "<hostname>",
      "id": "<host_id>",
      "ip": [
        "fe80::8df6:2969:45f4:3946/64",
        "169.254.57.70/16",
        "fe80::e08c:2dca:3980:8111/64",
        "192.168.1.113/24",
        "fe80::4dca:61f4:efed:640a/64",
        "169.254.100.10/16",
        "fe80::905f:a887:65ef:ec71/64",
        "169.254.236.113/16",
        "fe80::5ba:f065:3d1c:8ca3/64",
        "169.254.140.163/16",
        "fe80::3588:c747:8300:a5eb/64",
        "169.254.165.235/16",
        "fe80::ac6b:7f28:a3b6:f99b/64",
        "192.168.116.1/24",
        "fe80::380c:80f8:7628:6d2d/64",
        "192.168.142.1/24",
        "fe80::9cd7:4b6e:295d:f8b7/64",
        "169.254.248.183/16",
        "::1/128",
        "127.0.0.1/8",
        "fe80::a96e:270b:b3e0:fa7d/64",
        "172.31.240.1/20"
      ],
      "mac": [
        "<mac_address>",
        "<mac_address>",
        "<mac_address>",
        "<mac_address>",
        "<mac_address>",
        "<mac_address>",
        "<mac_address>",
        "<mac_address>",
        "<mac_address>"
      ]
    },
    "os": {
      "family": "windows",
      "kernel": "10.0.19041.1348 (WinBuild.160101.0800)",
      "platform": "windows",
      "version": "10.0",
      "name": "Windows 10 Education",
      "full": "Windows 10 Education(10.0)"
    }
  }
}
Response 
HTTP/2 400 Bad Request
Content-Type: application/json; charset=utf-8
X-Content-Type-Options: nosniff
Content-Length: 39
Date: Sat, 20 Nov 2021 20:39:05 GMT

{"statusCode":400,"error":"BadRequest"}
2

Could you share some more details on how you setup / run Elastic Agent with fleet-server?

kibana_system has not sufficient permissions to create the api keys for fleet-server, you need to be logged in as superuser.

3

My problem was fixed when I added the EDR Integration to the Default Fleet Server Policy. Maybe the other agent wasn't able to create the datastreams itself.
It might also have been something completely different that fixed it in the end. :smiley:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.