Ndjson parser doesn't expand keys if target is set

[Firewire2002]

Hi,

it seems that there is the same issue with the ndjson parser like in the decode_json_fields processor some time ago: Expand fields in `decode_json_fields` if target is set by kvch · Pull Request #32010 · elastic/beats · GitHub

dotted fields aren't expanded it the target option in ndjson parser is set. All works fine if the target option is missing.
Seems to be this code position: beats/json.go at main · elastic/beats · GitHub

Example Config:

- type: filestream
  id: my_dummy_log
  paths:
    - /tmp/my_dummy_log
  parsers:
    - ndjson:
        target: 'dummy'
        expand_keys: true

Example Log data

{ "my.test": 1 }

Expected event:

{ "dummy":  { "my": { "test": 1 } } }

But got event:
t:

{ "dummy":  { "my.test": 1 } }

Tried with

- type: filestream
  id: my_dummy_log
  paths:
    - /tmp/my_dummy_log
  parsers:
    - ndjson:
        expand_keys: true

Then I get:

{ "my": { "test": 1 } }

It would be nice to get the old processor bug fix also for the ndjson parser.

Regards,
Marcel

[carly.richmond]

Hi @Firewire2002,

We really appreciate your research on this and for sharing the issue. It's really helpful! I don't see an active issue that fits this problem.

Do you mind raising a GitHub bug issue on @elastic/beats for this one? Do DM me if you need help raising it.

Thanks again!

[Firewire2002]

Done: ndjson parser doesn't expand keys if target is set · Issue #35628 · elastic/beats · GitHub