I'm using Filebeat, plus hints based auto discovery, to get my Docker Swarmt container Apache logs passed through the built in Apache logs ingest pipelines.
The thing is, that several of my apache containers are also outputting logs from Symfony and/or Drupal.
Those logs, obviously, don't match any Apache grok pattern and cause grok parse failure messages.
To fix that, I have a final ingest pipeline configured. In it I detect if there is a grok parse error, if the doc is tagged for symfony or drupal, and then run it through my own grok pattern.
That works just fine, but I'm still left with the
error.message field claiming there is still a grok error.
I'd like to go ahead and just drop that field when my custom grok processor succeeds.
Where I'm stuck is that I'm not seeing an obvious way to detect if my grok processors have succeeded so I can run a remove processor on the
I could just remove the field all the time, but that would prevent me from finding the errors I need to fix.
I also tried adding a tag in the grok processor, but for some reason those tags don't actually get added to the tags field. (A long time ago I ran across this and someone explained it, but I can't recall what they said and didn't find the topic in a quick search...)
Anyone have any ideas?