@fbaligand
Thank you very much for your interest. I tried to implement your code but no joy just yet.
Here is what I have; hopefully it will be obvious where I am going wrong?
The (test) log file I am reading looks like this:
cycle 1
name: jellybeans
some: chocolate
cycle 2
name: bubblegum
name: gumdrops
name: candycanes
cycle 3
name: sherbet
name: candyfloss
cycle 4
name: mintdrops
name: icecream
My conf file is this:
input {
file {
path => "/Users/me/Downloads/logstash-2.3.1/source.log"
ignore_older => 0
}
}
filter {
if [message] =~ /.*cycle.*/ {
grok {
match => { "message" => ".*cycle\s(?<cycle>\d*)"}
}
}
if [cycle] {
aggregate {
task_id => "%{host}%{path}"
code => "map['cycle'] = event.get('cycle')"
}
} else {
aggregate {
task_id => "%{host}%{path}"
code => "event.set('cycle', map['cycle'])"
}
}
}
output {
stdout {
codec => rubydebug
}
}
Which I think is pretty much representative of the code/behaviour you suggested. I am not differentiating messages by [type] however as all my events are coming from the same log file so I cannot see where or why I would use different types.. I was thinking that I could just trigger the aggregate events based on what the event contains (a cycle value or not).
The output I get is basically this:
Aggregate exception occurred. Error: undefined method `set' for #<LogStash::Event:0x6173639f> ; Code: event.set('cycle', map['cycle']) ; Map: {} ; EventData: {"message"=>"name: candyfloss", "@version"=>"1", "@timestamp"=>"2017-02-04T20:32:10.648Z", "path"=>"/Users/me/Downloads/logstash-2.3.1/source.log", "host"=>"MBP"} {:level=>:error}
Aggregate exception occurred. Error: undefined method `get' for #<LogStash::Event:0x13833cc8> ; Code: map['cycle'] = event.get('cycle') ; Map: {} ; EventData: {"message"=>"cycle 4", "@version"=>"1", "@timestamp"=>"2017-02-04T20:32:10.648Z", "path"=>"/Users/me/Downloads/logstash-2.3.1/source.log", "host"=>"MBP", "cycle"=>"4"} {:level=>:error}
Aggregate exception occurred. Error: undefined method `set' for #<LogStash::Event:0x35a0a565> ; Code: event.set('cycle', map['cycle']) ; Map: {} ; EventData: {"message"=>"name: mintdrops", "@version"=>"1", "@timestamp"=>"2017-02-04T20:32:10.649Z", "path"=>"/Users/me/Downloads/logstash-2.3.1/source.log", "host"=>"MBP"} {:level=>:error}
Aggregate exception occurred. Error: undefined method `set' for #<LogStash::Event:0x74778748> ; Code: event.set('cycle', map['cycle']) ; Map: {} ; EventData: {"message"=>"name: icecream", "@version"=>"1", "@timestamp"=>"2017-02-04T20:32:10.649Z", "path"=>"/Users/me/Downloads/logstash-2.3.1/source.log", "host"=>"MBP"} {:level=>:error}
{
"message" => "cycle 1",
"@version" => "1",
"@timestamp" => "2017-02-04T20:32:10.645Z",
"path" => "/Users/me/Downloads/logstash-2.3.1/source.log",
"host" => "MBP",
"cycle" => "1",
"tags" => [
[0] "_aggregateexception"
]
}
{
"message" => "name: jellybeans",
"@version" => "1",
"@timestamp" => "2017-02-04T20:32:10.646Z",
"path" => "/Users/me/Downloads/logstash-2.3.1/source.log",
"host" => "MBP",
"tags" => [
[0] "_aggregateexception"
]
}
{
"message" => "name: chocolate",
"@version" => "1",
"@timestamp" => "2017-02-04T20:32:10.647Z",
"path" => "/Users/me/Downloads/logstash-2.3.1/source.log",
"host" => "MBP",
"tags" => [
[0] "_aggregateexception"
]
}
As I am sure you are aware I am looking for the current value of cycle to be present in all events.
Really appreciate this!
