Ingest pipeline - processor - can't reuse fieldname after rename of fieldname

Elastic Stack Elasticsearch
1

Hi

I got a logfeed coming from a black box (aka. i can't change a bit in datastructure) and arriving at me ES.

We use in all our logging the field "log" to structure our data to (after grok) like log.level, log.thread, ... that we parse from the "message" field

The blackbox logging field is already defined as "log", so before grokking this I first rename the field to "message" (using rename processor) and then grok it back to log.xxxx

problem is this doesn't seem to work, if I change the "log" naming to "logx" it works, so i assume ES has an issue with parsing it in the same pipeline

so

  • rename log -> message
  • grok message-> {LEVEL:log.level}
    FAILS

so

  • rename log -> message
  • grok message-> {LEVEL:logx.level}
    WORKS

I also added a remove of "log" between the rename & grok but that doesn't help either.

is this a restriction, of am I missing something ?

Help is appreciated

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.