Hello Team,
We are using combination grok, ruby ,date filters in our Logstash pipeline. and the input load is is around 20k records per minute.
We observed that sometimes the data which is not present in that record (but some other record has that value) is getting assigned to the field.
for example :
input : ""
filter : grok
ruby : process matched grok field and assign values to the fields
so sometime the matched grok pattern of record1 is processed and assigned to the field of record 1 and also getting assigned to the field of record2 so the value of matched grok field and processed field are different and we see _rubyeception in tags.
but if we try to process the records individually it works fine.
Not sure if this is happening because of the heavy input load or some cahing at logstash level.
Any pointers to resolve the issue are appreciated.
Thanks in advance for your help and support.
Logstash version : 7.4.1
Thank you,
Aditya