Problem: Custom Filebeat Module to parse modified nginx log Grok Error

SOLVED

The secret sauce is Drum roll.

You have to delete the injest pipeline in elasticsearch in order for a modified one to be loaded. I had the right grok pattern the whole time. The issue was the very first time filebeat loaded default.json I still had the old nginx pattern in there.

Wow this really needs to have better documentation. Because it was NOT AT ALL obvious.

What I should have been doing from the start was running this command in devtools each and everytime I modified the ingest pipeline.
DELETE _ingest/pipeline/filebeat-6.3.0-adblock-access-default

I would like to thank everyone that helped here and over chat trying to narrow this down. I finally figured it out when I saw a hint at the issue in a video.

1 Like