Host, observer, monitor confusion

Elastic Stack Beats
1

The docs state that the "observer" is the one doing the monitoring and the host is the thing being monitored.

However the "host" fields are being populated with data about the node running the heartbeat process (aka observer).

So the processor "add_host_metadata" and "add_observer_metadata" seem to serve the same purpose?

I would like to add metadata to the monitored object... should there be an "add_monitor_metadata" processor?

example output: (observer.hostname and host.hostname are the same)

{
  "_index": "heartbeat-7.2.0-2019.07.03",
  "_type": "_doc",
  "_id": "SXnHtmsB1e855P3-YfDb",
  "_version": 1,
  "_score": null,
  "_source": {
    "ecs": {
      "version": "1.0.0"
    },
    "resolve": {
      "rtt": {
        "us": 1286
      },
      "ip": "10.41.210.227"
    },
    "error": {
      "type": "io",
      "message": "Get http://cam-010200-00-01.example.net:80: dial tcp 10.41.210.227:80: i/o timeout"
    },
    "monitor": {
      "check_group": "d36b8610-9d65-11e9-9c43-005056a9dc98",
      "type": "http",
      "name": "http_example_cameras",
      "status": "down",
      "ip": "10.41.210.227",
      "duration": {
        "us": 16001480
      },
      "id": "auto-http-0X254C92F24A1E394E-11e55715396e91fd"
    },
    "observer": {
      "geo": {
        "region_iso_code": "NY",
        "city_name": "New York",
        "name": "nyc-dc1-rack1",
        "location": "40.7128, -74.0060",
        "continent_name": "North America",
        "country_iso_code": "US",
        "region_name": "New York"
      },
      "hostname": "o-issc-004246"
    },
    "host": {
      "architecture": "x86_64",
      "os": {
        "version": "7.6 (Maipo)",
        "name": "Red Hat Enterprise Linux Server",
        "family": "redhat",
        "kernel": "3.10.0-957.21.3.el7.x86_64",
        "codename": "Maipo",
        "platform": "rhel"
      },
      "containerized": false,
      "name": "heartbeat_6.8",
      "ip": [
        "172.18.128.198",
        "fe80::250:56ff:fea9:dc98"
      ],
      "geo": {
        "region_iso_code": "ZH",
        "city_name": "Den Haag",
        "name": "Anna van Buerenplein",
        "country_iso_code": "NL",
        "continent_name": "Europa",
        "location": "52.08163, 4.32594",
        "region_name": "Zuid Holland"
      },
      "mac": [
        "00:50:56:a9:dc:98"
      ],
      "hostname": "o-issc-004246",
      "id": "4f84b0e31a18417cafc3e92e09924869"
    },
    "@timestamp": "2019-07-03T07:40:31.577Z",
    "summary": {
      "down": 1,
      "up": 0
    },
    "url": {
      "domain": "cam-010200-00-01.example.net",
      "full": "http://cam-010200-00-01.example.net:80",
      "port": 80,
      "scheme": "http"
    },
    "tags": [
      "heartbeat",
      "004246",
      "search",
      "example",
      "example_cameras",
      "cameras",
      "web-tier",
      "beats_input_raw_event"
    ],
    "agent": {
      "version": "7.2.0",
      "type": "heartbeat",
      "name": "heartbeat_6.8",
      "ephemeral_id": "498e98eb-80d7-490e-b3bb-6c9198981779",
      "hostname": "o-issc-004246",
      "id": "0bffa927-db55-4110-95b9-11fb870a7031"
    },
    "event": {
      "dataset": "uptime"
    },
    "@version": "1"
  },
  "fields": {
    "@timestamp": [
      "2019-07-03T07:40:31.577Z"
    ]
  },
  "highlight": {
    "url.domain": [
      "@kibana-highlighted-field@cam-010200-00-01.example.net@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1562139631577
  ]
}
2

You are correct that the processors do similar things (they populate slightly different fields per the ECS spec). Additionally, there's an open bug that we still populate host.name.

You can also use any custom fields you like with the add_fields processor. This is applied globally if defined in the root of the config, next to add_observer_metadata. It can also be applied per-monitor.

3

Thanks! I got confused about the seemingly strict definition of "observer" and "host" and the resulting behavior.

"add_fields" is working fine for me.

  processors:
  - add_fields:
      when:
        contains:
          url.domain: "400800"
      target: monitor.geo
      fields:
        name: Anna van Buerenplein
        location: 52.08163, 4.32594
        continent_name: Europa
        country_iso_code: NL
        region_name: Zuid Holland
        region_iso_code: ZH
        city_name: Den Haag
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.