I've got an ingest pipeline that receives very similar messages from a few different log files. These logfiles have delimiters ";" where some of the fields have no value, but the context of each delimited value does not change. Consider:
01/01/01 01:01:01;INFO;doing action;; 01/01/01 01:01:02;INFO;;dir2;dir2
f1=datetime, f2=severity, f3=message, f4=sourcedir, f5=destdir
but sometimes there is no "message", so when I grok extract field3 it could be an empty string.
Is there a clean way in my ingest pipeline to say remove any fields that are empty strings without setting many "remove" entries with "if"s attached?