I was wondering if there a way to accept all patterns excluding a specific regular expression?
Can u be more specific? Can u provide an example of what you're trying to do?
I can't share an example but I try to be more specific. I have setup a regex in the ingest-pipeline using the grok to look for a specific message. But I was misinformed, they want all data excluding that message format. So I am trying to see if there is a way to just accept everything that doesn't match that message and exclude the ones that do match the original pattern. I am wondering if it should be a conditional statement or maybe I can encapsulate the pattern with '!'?
I think i get it but are you saying that you want to drop any documents that don't match the GROK pattern (Or do)? or do something else based on the pattern?
Yes, The grok pattern that I have shouldn't be indexed. They kinda don't serve any purpose in our configuration. I think if they are drop that would be more ideal since it would reduce stress to our storage.
If you're using filebeat, you can use a drop processor with a regex.
You can also use an ingest pipeline with a drop processor and a condition using RegEx for that as well. (Be careful a bit with that)
That can be a little tricky.
And yes you could set a grok And then set a tag in that grok processor and then do the drop processor afterwards based on that tag.
So you have options? You just got to figure out what's the most efficient for your use case
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.