HTTP JSON input with Split but keeping other fields

[bitnapper (thorsten)]

Hi,

as a learning project I set up a ELK stack to read the status of my Hue Homeautomation installation. I only need to read the sensordata at the moment but I want to keep the association to the bridge. I use filebeat to read from the Hue API and response.split.target: "body.sensors" to get every sensor state as a singel event. Along with the sensor data come a config object wich I'd like to add to every one of these sensor evets to parse it later in the pipeline. But I'm not sure wether the http json input can do that or not. The JSON is like

{
  "lights": { "1": {}, "3": {} },
  "config": { "name": "xxx", "mac": "xx:xx:xx:xx:xx" },
  "sensors": { "5": {}, "2": {} }
}

The filebeat input looks like this:

filebeat.inputs:
- type: httpjson
  config_version: 2
  response.split.type: "map"
  response.split.target: "body.sensors"
  interval: 10s
  request.url: "http://192.168.1.44/api/xxx/"
  pipeline: "philips_hue"
  processors:
    - decode_json_fields:
        max_depth: 3
        fields: ["message"]
        target: "json"
    - fingerprint:
        fields: ["message"]
        target_field: "@metadata._id"

[Wolfram_Haussig]

Hello Thorsten,

I have not tried it out myself but I guess you want to use response.split.keep_parent as described here:

If set to true, the fields from the parent document (at the same level as target ) will be kept. Otherwise a new document will be created using target as the root. Default: false .

Best regards
Wolfram