Is there a way to prevent specific events from being indexed? I don't mean specific fields, but conditionals for a field.
For example, we're ingesting netflow traffic data and I want to exclude all events that have both source and destination IPs from a specific range. While I know I can exclude these from search results, the point would be to cut down on the index sizes because netflow data is massive.
I'm guessing the best way might be with a pipeline processor, but honestly the docs for using pipelines is so arcane, I can't even get one to compile outside of something that's just descriptive. Even if I can get such a pipeline created, I don't know how it would get applied to the existing architecture.