Significant difference in size output because of an if statment (that shouldn't happend)


(Sahar Q) #1

Hi,
For some reason that i dont get the follow input creates twice less the size(27KB) on the created index in elastic:

output {
if "ArpTable" in [tags] {
elasticsearch {
hosts => ["x"]
action => "index"
index => "arp-table"
}
}
stdout {codec => rubydebug}
}

While if i delete the if statement the size of the index will increase twice (62KB):

output {
elasticsearch {
hosts => ["x"]
action => "index"
index => "arp-table"
}
stdout {codec => rubydebug}
}

what is the reason for it?

Edit: Solved - larger index had more shards


(Magnus B├Ąck) #2

Are you indexing the same amount of documents each time? Are the documents identical?


(Sahar Q) #3

yes there are. both contained 41 documents, and the same data was indexed


(Sahar Q) #4

After reChecking the index setting i found that the larger index was split into more shards than the other.
problem solved, sorry :slight_smile:


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.