Compare two fields with different documents

Elastic Stack Logstash
1

Hi,
I want to compare two fields but not in the same document is that possible ?
for example :

if date.document1 == date.document2 and x.document1 == y.document2 {
    #DO somthing
}

Any help would be sincerely appreciate!
Thanks!

2

I will try to explain more I want for a new document I want to compare it with all the documents I have or something like that if i found date.document1 == date.document2 and x.document1 == y.document2 then #DO somthing

3

It is not clear what you want to do, can you provide more context and a real example?

What do you want to do after comparing it? Are the documents being ingested or they already exist in Elasticsearch?

You want to compare fields from different events in logstash? Or you want to compare documents that are in elasticsearch but using a Logstash pipeline?

4

i have two files i read these files using filebeat so i extract each log with multiline.pattern then i send the data to kafka so the logstash entry is kafka...

For file1, for example I have two fields "date,code", so I want to compare it with other documents, I mean the "documents in file2" if I found the date and code are the same in this document, then send me an e-mail. i am sending data from two files in the same index to compare it so this is what i mean with date.document1 == date.document2 and x.document1 == y.document2 then #send email

I hope you understand me

5

logstash.conf

filter {
if ("kafka-topic-file1" in [topic_name]) {
# i created a fields using grok with regular expression 
# example fields : date1, code1
}
if("kafka-topic-file2" in [topic_name]) {
# i created a fields using grok with regular expression 
# example fields : date2, code2
}
output {
if ([date1] == [date2] and [code1] == [code2]) {
# send e-mail ( i did it it works )
}

doesn't work because as I understand I can't compare two fields with different documents and I ask if there is a solution for this.

I hope you understand me @leandrojmp and Thanks <3

6

This is not possible with Logstash.

Every event is independent from each other, you can't compare different events.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.