Status: (FAILED) could not start output: failed to reload output: open : no such file or directory reading <nil> accessing 'elasticsearch'

Elastic Stack Elastic Agent
1

I have strangers error from my fleet server is Unhealthy and then I check the the elastic-agent status :

Screenshot 2025-05-08 at 21.36.15
Screenshot 2025-05-08 at 21.36.152458×1020 284 KB

Before that I've problem with x509 unknown authority error but it disappeared in elastic agent logs because I'm trying to solve it with copying the CA to the yaml reference to this documentation Configure SSL/TLS for self-managed Fleet Servers | Elastic Docs

I didn't know it is related or not but how to fix this? because there's no data coming on the Integration.

I installed the fleet-server elastic agent with this command :

sudo ./elastic-agent install --url=https://10.0.10.152:8220 \
  --fleet-server-es=https://10.0.10.152:9200 \
  --fleet-server-service-token=<token> \
  --fleet-server-policy=fleet-server-policy \
  --certificate-authorities=/opt/Elastic/Agent/ca.crt \
  --fleet-server-es-ca=/opt/Elastic/Agent/http_ca.crt \
  --fleet-server-cert=/opt/Elastic/Agent/fleet-server.crt \
  --fleet-server-cert-key=/opt/Elastic/Agent/fleet-server.key \
  --fleet-server-port=8220 \
  --install-servers
2

Hi @neofall Welcome to the community...

It seems like you have added additional integrations to the fleet policy... which is OK.. but can complicate things when just getting started...

I would first just get fleet running without a bunch of additional integrations.

Then you need to make sure the Elasticsearch output in

Kibana -> Fleet -> Setting -> Outputs

You probably need to add the CA there...

To encrypt traffic between Elastic Agents, Fleet Server, and Elasticsearch:

  1. Configure Fleet settings. These settings are applied to all Fleet-managed Elastic Agents.
  2. In Kibana, open the main menu, then click Management > Fleet > Settings.
  3. Under Fleet Server hosts, specify the URLs Elastic Agents will use to connect to Fleet Server. For example, https://192.0.2.1:8220, where 192.0.2.1 is the host IP where you will install Fleet Server.

Tip

For host settings, use the https protocol. DNS-based names are also allowed.

  1. Under Outputs, search for the default output, then click the Edit icon in the Action column.
  2. In the Hosts field, specify the Elasticsearch URLs where Elastic Agents will send data. For example, https://192.0.2.0:9200.
  3. Specify either a CA certificate or CA fingerprint to connect securely Elasticsearch:
  • If you have a valid HEX encoded SHA-256 CA trusted fingerprint from root CA, specify it in the Elasticsearch CA trusted fingerprint field. To learn more, refer to the Elasticsearch security documentation.
  • Otherwise, under Advanced YAML configuration, set ssl.certificate_authorities and specify the CA certificate to use to connect to Elasticsearch. You can specify a list of file paths (if the files are available), or embed a certificate directly in the YAML configuration. If you specify file paths, the certificates must be available on the hosts running the Elastic Agents.
3

Hi @stephenb , thank you for your reply,

I'm trying to reinstall the fleet server with new SSL/TLS and add the CA in Outputs then the fleet server elastic agent is Healthy

Screenshot 2025-05-09 at 21.02.56
Screenshot 2025-05-09 at 21.02.561920×1200 105 KB

but when i am trying to integrate my Trendmicro Vision one with the API and regional domain as mentioned in Trend Micro Vision One | Elastic Documentation , the data is not available. I'm trying to figured out in elastic agent fleet logs and same errors like x509 certificates unknown authorities :smiling_face_with_tear:

4

Logs from my Elastic agent fleet

Screenshot 2025-05-09 at 21.38.05
Screenshot 2025-05-09 at 21.38.052524×640 106 KB

5

My Healthy Fleet server but still error x509: certificate signed by unknown authority

Screenshot 2025-05-09 at 21.43.17
Screenshot 2025-05-09 at 21.43.172880×1800 370 KB

6

First please don't post images of text really hard to read search etc. some people can not event see them... please post the text

What I would try is this method .... see what happens

Pasted certificate example: Be carefull with the indents....

ssl:
  certificate_authorities:
  - |
    -----BEGIN CERTIFICATE-----
    MIIDSjCCAjKgAwIBAgIVAKlphSqJclcni3P83gVsirxzuDuwMA0GCSqGSIb3DQEB
    CwUAMDQxMjAwBgNVBAMTKUVsYXN0aWMgQ2VydGlmaWNhdGUgVG9vbCBBdXRvZ2Vu
    ZXJhdGVkIENBMB4XDTIxMDYxNzAxMzIyOVoXDTI0MDYxNjAxMzIyOVowNDEyMDAG
    A1UEAxMpRWxhc3RpYyBDZXJ0aWZpY2F0ZSBUb29sIEF1dG9nZW5lcmF0ZWQgQ0Ew
    ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDOFgtVri7Msy2iR33nLrVO
    /M/6IyF72kFXup1E67TzetI22avOxNlq+HZTpZoWGV1I4RgxiQeN12FLuxxhd9nm
    rxfZEqpuIjvo6fvU9ifC03WjXg1opgdEb6JqH93RHKw0PYimxhQfFcwrKxFseHUx
    DeUNQgHkMQhDZgIfNgr9H/1X6qSU4h4LemyobKY3HDKY6pGsuBzsF4iOCtIitE9p
    sagiWR21l1gW/lNaEW2ICKhJXbaqbE/pis45/yyPI4Q1Jd1VqZv744ejnZJnpAx9
    mYSE5RqssMeV6Wlmu1xWljOPeerOVIKUfHY38y8GZwk7TNYAMajratG2dj+v9eAV
    AgMBAAGjUzBRMB0GA1UdDgQWBBSCNCjkb66eVsIaa+AouwUsxU4b6zAfBgNVHSME
    GDAWgBSCNCjkb66eVsIaa+AouwUsxU4b6zAPBgNVHRMBAf8EBTADAQH/MA0GCSqG
    SIb3DQEBCwUAA4IBAQBVSbRObxPwYFk0nqF+THQDG/JfpAP/R6g+tagFIBkATLTu
    zeZ6oJggWNSfgcBviTpXc6i1AT3V3iqzq9KZ5rfm9ckeJmjBd9gAcyqaeF/YpWEb
    ZAtbxfgPLI3jK+Sn8S9fI/4djEUl6F/kARpq5ljYHt9BKlBDyL2sHymQcrDC3pTZ
    hEOM4cDbyKHgt/rjcNhPRn/q8g3dDhBdzjlNzaCNH/kmqWpot9AwmhhfPTcf1VRc
    gxdg0CTQvQvuceEvIYYYVGh/cIsIhV2AyiNBzV5jJw5ztQoVyWvdqn3B1YpMP8oK
    +nadUcactH4gbsX+oXRULNC7Cdd9bp2G7sQc+aZm
    -----END CERTIFICATE-----
7

Sorry about that @stephenb ,

I already replace and add my certificate like this :

ssl:
  certificate_authorities:
  - |
    -----BEGIN CERTIFICATE-----
    MIIDWjCCAkKgAwIBAgIVAO3ItXzqMVrrgXiO8SPQc7GpJDpqMA0GCSqGSIb3DQEB
    CwUAMDQxMjAwBgNVBAMTKUVsYXN0aWMgQ2VydGlmaWNhdGUgVG9vbCBBdXRvZ2Vu
    ZXJhdGVkIENBMB4XDTI1MDUwOTA3MjAzNloXDTI4MDUwODA3MjAzNlowNDEyMDAG
    A1UEAxMpRWxhc3RpYyBDZXJ0aWZpY2F0ZSBUb29sIEF1dG9nZW5lcmF0ZWQgQ0Ew
    ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCzDupwL3uhJx/pEYA0MGKS
    IUlnX8GzJFyM8uSpGYk3VtLlroi/cU8Fe+xY0CXxX++rGkP6l163eZV/VlqIOQlr
    BmJP6OHRJKXZ/vI99mcxuPyISj5gwy168zODWebmgdtkPoSYWiOc4DxUYU8WXVI5
    NAQNDgmq+u53xJ7IsNsZ3J2JRAQZwZJTegvTkoQbNL2ZN5Fvxg1ZF26N78sI8I3I
    mkSubPdRBjD6n4rAQGj+H2XhmUmsG79jLL6/lvI172VNOuhBteCZTSDq+ddr5ucb
    zrwz19qd2B2qqQSxBlq+F3QG8gSIrUYNXJhAOHas1MNTAjgvNwdetd+/cCjqJh9R
    AgMBAAGjYzBhMB0GA1UdDgQWBBTh2QiU6wxQZ76Z71XradEIQkdE5DAfBgNVHSME
    GDAWgBTh2QiU6wxQZ76Z71XradEIQkdE5DAPBgNVHRMBAf8EBTADAQH/MA4GA1Ud
    DwEB/wQEAwIBBjANBgkqhkiG9w0BAQsFAAOCAQEAos5Z7nvZ6hu13v3Dsmjz9D5g
    4VE9ycgX3fjwY5N7FiDjb9Hnu9YEBQT7rJk/cqCWvJXeFD5ekO/aC+4gmVz/D64p
    MK4GzNWnAIbr4r1zaa6ec/2V6SlNrseOzo2uZAgU4IwDYCgEj/F5kM9gJ70mZiq9
    YNFmSRegm2LjApFVZjAjNTB7nN+t0Ci5vBQsk8FSweGXWdkLsFaFsFzCHsU/qu0a
    9JUluZ0FFt8DXtq+8D0LVOZZ398Vb/Purz7PteBF2lRPFOH7WCd0nZGPj/O+4HqM
    Ut9PCHGt0Rbpq43Je06NBBDu8dECeXVKsf/Zg6dqss3OWveneoERqlZ2ebv9DQ==
    -----END CERTIFICATE-----

Then i get new error like this :

{"log.level":"info","@timestamp":"2025-05-09T16:14:50.666Z","message":"Attempting to reconnect to backoff(elasticsearch(https://localhost:9200)) with 7 reconnect attempt(s)","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"httpjson-default","type":"httpjson"},"log":{"source":"httpjson-default"},"log.logger":"publisher_pipeline_output","log.origin":{"file.line":140,"file.name":"pipeline/client_worker.go","function":"github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*netClientWorker).run"},"service.name":"filebeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2025-05-09T16:14:50.669Z","message":"Error dialing x509: certificate is not valid for any names, but wanted to match localhost","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"httpjson-default","type":"httpjson"},"log":{"source":"httpjson-default"},"log.logger":"esclientleg","log.origin":{"file.line":39,"file.name":"transport/logging.go","function":"github.com/elastic/elastic-agent-libs/transport/httpcommon.(*HTTPTransportSettings).RoundTripper.LoggingDialer.func2"},"service.name":"filebeat","network.transport":"tcp","server.address":"localhost:9200","ecs.version":"1.6.0","ecs.version":"1.6.0"}

hmm this is weird, I'm trying to check the elasticsearch with the CA and not from the kibana then it can be connected :

root@ip-10-0-10-152:~# curl --cacert /etc/elasticsearch/certs/ca.crt -u elastic:<password> https://10.0.10.152:9200
{
  "name" : "mysiem",
  "cluster_name" : "Prod-ElasticSIEM",
  "cluster_uuid" : "o8RMEwmoSVW6Z8uPmcLJeg",
  "version" : {
    "number" : "9.0.1",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "73f7594ea00db50aa7e941e151a5b3985f01e364",
    "build_date" : "2025-04-30T10:07:41.393025990Z",
    "build_snapshot" : false,
    "lucene_version" : "10.1.0",
    "minimum_wire_compatibility_version" : "8.18.0",
    "minimum_index_compatibility_version" : "8.0.0"
  },
  "tagline" : "You Know, for Search"
}
8

did you try putting this in hosts in the elasticsearch output instead of localhost