Dec 1st, 2022: [EN] Elastic Agent 101

This article is also available in portuguese.

First things first, what is the Elastic Agent?

Elastic Agent is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, forward data from remote services or hardware, and more.

How do the Elastic Agent does that all? It delegates, its responsibility is to run, manage and configure the needed applications to fulfil the "tasks assigned to it", such as collecting logs, metrics or protecting a host.

A policy is the Elastic Agent's configuration, in the policy several integrations are added, then the Elastic Agent analyzes the policy to discover all the applications needed to fulfill the policy.

The Elastic Agent will take care of running, managing and configuration each of the needed applications. As an example, if we create a policy and add the APM Server, Custom Logs and System metrics integrations, the Elastic Agent will run and configure the APM server, Filebeat (to collect the logs) and Metricbeat (to collect system metrics).

Creating a policy, adding integrations and installing the Elastic Agent are well documented process, check the links if you want to know more about them.

What is less known are all the features the Elastic Agent, as a CLI, provides to investigate and diagnose problems happening with itself or any of the applications it runs.
We all know, sometimes things don't works as we expect and we need to discover what is happening. Here is where the CLI commands we'll discuss next come in hand.

First, the status command shows the status of every application the Elastic Agent is running:

elastic-agent status

Status: HEALTHY
Message: (no message)
Applications:
  * filebeat               (HEALTHY)
                           Running
  * metricbeat             (HEALTHY)
                           Running
  * filebeat_monitoring    (HEALTHY)
                           Running
  * metricbeat_monitoring  (HEALTHY)
                           Running

The _monitoring Beats, as the name suggests, monitor the Elastic Agent and its applications collecting metrics and their logs.

Next command is diagnostics, a lot more technical than status, it'll also show information about the applications running under the Elastic Agent:

elastic-agent diagnostics

elastic-agent  id: 65a5bc58-d3fe-414a-bc55-9b2bb69c85f5                version: 8.5.2
               build_commit: c13f9157c438fc60cfbb822b385ea91bc91193cc  build_time: 2022-11-17 21:16:12 +0000 UTC  snapshot_build: false
Applications:
  *  name: filebeat               route_key: default
     process: filebeat            id: bdd93dd6-47b8-41cb-a5eb-a4e73ca8d205          ephemeral_id: f9e7c229-73a4-40c0-ad5e-7d751071934b  elastic_license: true
     version: 8.5.2               commit: 1ebd0940bd56943642ea8d63d1fe8227f86e7435  build_time: 2022-11-15 20:38:43 +0000 UTC           binary_arch: amd64
     hostname: elastic-agent      username: root                                    user_id: 0                                          user_gid: 0
  *  name: metricbeat             route_key: default
     process: metricbeat          id: 6146ca43-d3dc-43fc-864c-16f7c718931f          ephemeral_id: c98424db-6ba9-4dab-b656-25965c82accc  elastic_license: true
     version: 8.5.2               commit: 1ebd0940bd56943642ea8d63d1fe8227f86e7435  build_time: 2022-11-15 20:38:34 +0000 UTC           binary_arch: amd64
     hostname: elastic-agent      username: root                                    user_id: 0                                          user_gid: 0
  *  name: filebeat_monitoring    route_key: default
     process: filebeat            id: dd6d9f4e-0fc7-413e-a829-232d8fb9222b          ephemeral_id: 43d99165-49a2-41ef-9a63-358f545bd5ec  elastic_license: true
     version: 8.5.2               commit: 1ebd0940bd56943642ea8d63d1fe8227f86e7435  build_time: 2022-11-15 20:38:43 +0000 UTC           binary_arch: amd64
     hostname: elastic-agent      username: root                                    user_id: 0                                          user_gid: 0
  *  name: metricbeat_monitoring  route_key: default
     process: metricbeat          id: 014efcdf-40c2-4aad-916d-4468dc67ad48          ephemeral_id: 0e01dae8-86d5-4717-9594-0228479ba5c8  elastic_license: true
     version: 8.5.2               commit: 1ebd0940bd56943642ea8d63d1fe8227f86e7435  build_time: 2022-11-15 20:38:34 +0000 UTC           binary_arch: amd64
     hostname: elastic-agent      username: root                                    user_id: 0                                          user_gid: 0

The diagnostics has got the sub-command collect, that well, collects pretty much everything about the Elastic Agent and the programs it's running. It gathers metadata, the policy, the individual configuration the Elastic Agent generates from the policy to each program it's running and the logs. This is by far the most useful command for investigations, mainly when who is analysing the data does not have access to the host where the Elastic Agent is running. It's one of the first things we ask our customers when they reach out for support regarding a problem with the Elastic Agent or any of the integrations.

The inspect command will show the current configuration. The output is huge as it shows everything being collected from the host, so here is a shorter version of its output:

elastic-agent inspect

agent:
  download:
    source_uri: https://artifacts.elastic.co/downloads/
  monitoring:
    enabled: true
    logs: true
    metrics: true
    namespace: default
    use_output: default
fleet:
  hosts:
  - https://my.fleet-server.co:443
id: f0beede0-6f1e-11ed-aaed-9bf77350c160
inputs:
- data_stream:
    namespace: default
  id: logfile-system-17831582-e0d5-48b5-a72c-405396085de7
  meta:
    package:
      name: system
      version: 1.20.4
  name: system-1
  package_policy_id: 17831582-e0d5-48b5-a72c-405396085de7
  revision: 1
  streams:
  - data_stream:
      dataset: system.syslog
      type: logs
    exclude_files:
    - .gz$
    id: logfile-system.syslog-17831582-e0d5-48b5-a72c-405396085de7
    ignore_older: 72h
    multiline:
      match: after
      pattern: ^\s
    paths:
    - /var/log/messages*
    - /var/log/syslog*
    processors:
    - add_locale: null
  type: logfile
  use_output: default
output_permissions:
  default:
    _elastic_agent_checks:
      cluster:
      - monitor
    _elastic_agent_monitoring:
      indices:
      - names:
        - logs-elastic_agent.apm_server-default
        privileges:
        - auto_configure
        - create_doc
      - names:
        - metrics-elastic_agent.apm_server-default
        privileges:
        - auto_configure
        - create_doc
outputs:
  default:
    api_key: <REDACTED>
    hosts:
    - https://my.ES.co:443
    type: elasticsearch
revision: 1

Last but not least, elastic-agent help will, as you probably already guessed, show you all the available commands within the Elastic Agent.

4 Likes

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.