Elastic Agent is not getting configured properly

Elastic Stack Elastic Agent
1

Hi Team,

We have installed a standalone Elastic Agent on our machine to capture logs from the ForgeRock API. The logs are expected to be sent to a Kafka topic. While the Elastic Agent installation was successful , these logs are not being sent to the Kafka topic as expected.
There is not connectivity issue because we are able to CURL the ForgeRock logs locally on the same machine.

Below is the configuration used in the elastic-agent.yml file.
"inputs:

  • type: httpjson
    schedule: '@every 30m'
    config:
    url: "https://------.com/monitoring/logs?source=am-everything"
    method: GET
    headers:
    Authorization: "Bearer 2---:c1-----1"
    response_format: json
    processors:
    • decode_json_fields:
      fields: ["message"]
      target: ""
      overwrite_keys: true
      add_error_key: true
      use_output: kafka

outputs:
kafka:
type: kafka
hosts:
- "b-1.-----:9094"
- "b-2.-----:9094"
- "b-3.-----:9094"
topic: "test_app_topic"
ssl:
enabled: true
truststore_location: "/etc/pki/tls/certs/kafka.client.truststore.jks"
truststore_password: "----"

elasticsearch:
type: elasticsearch
hosts:
- "https://-----:9243/"
username: "elastic"
password: "-----"
ssl:
enabled: true
certificate_authorities: ["/etc/pki/tls/certs/ca-bundle.crt"]

monitoring:
enabled: true
logs: true
metrics: true
use_output: kafka

agent:
logging:
level: info
to_files: true
files:
path: "/var/log/elastic-agent/"
name: "elastic-agent"
keepfiles: 7
permissions: 0644
"

Additionally, we are encountering the following error logs in the elastic-agent.log file.

"{"log.level":"error","@timestamp":"2024-12-16T12:15:44.523Z","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/agent/application/coordinator.(*Coordinator).processVars","file.name":"coordinator/coordinator.go","file.line":1222},"message":"updating Coordinator variables: generating component model: failed to render components: failed to inject monitoring: failed to inject monitoring output: output "default" used for monitoring not found","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
".

Could anyone help on this?

Regards

2

have you run the

/opt/Elastic/Agent/elastic-agent status

/opt/Elastic/Agent/elastic-agent inspect

and taken a look?

3

I do not use Elastic Agent in standalone mode, but from the docs you need to have a output named default.

This is mentioned here.

Try to change the name of the kafka output from kafka to default.

2 Likes
4

Yes @leandrojmp great catch

A default output configuration is required.

5

Thank you so much @leandrojmp . It worked once we changed the output to default. But we are receiving the filebeat and metricbeat logs instead of the API logs.

7

It looks like your issue was resolved in the other thread API logs not coming after setting up Elastic Agent , Beats logs are coming - #14 by strawgate