ssasporta
(Sharon Sasporta)
November 14, 2017, 10:49am
1
My Logstash conf output looks like that:
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "logstash%{[fields][index]}%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}
I have aggregation in my filter.
In end of run, I have two Indexces created:
Contain the events
Contain the aggregations
The events index name: logstashcmserver2017.09.11
The aggregation index name : logstash%{[fields][index]}2017.11.14
Can't I control the aggregation index name? Should it be hard coded?
Thanks
Sharon.
Apparently %{[fields][index]}
is not set for aggregation records, so you should probably correct that.
ssasporta
(Sharon Sasporta)
November 14, 2017, 12:15pm
3
Thanks Christian.
Should I use an if statement in the output for the aggregation ?
Can you refer me to something similar?
I will try to do something and will share the results.
Thanks
Sharon.
You could use a conditional to set it if it is not already set, but that depends on whether you have other types of events that could be affected or not.
ssasporta
(Sharon Sasporta)
November 14, 2017, 1:31pm
5
Hi Christian,
I created a new output:
output {
if [aggregation] == "true" {
elasticsearch {
hosts => ["localhost:9200"]
index => "logstashaggregation%{[entity]}%{+YYYY.MM.dd HH:mm}"
}
}
else {
elasticsearch {
hosts => ["localhost:9200"]
index => "logstash%{[fields][index]}%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}
}
My aggregation filter looks like that:
aggregate {
task_id => "%{inputserver}_%{exceptiontype}_%{apiname}"
code => " map['totalProcessTime'] ||= 0;
map['counter'] ||= 0;
map['counter'] += 1;
map['totalProcessTime'] += event.get('cputimeinmillisec')
"
push_map_as_event_on_timeout => true
timeout => 100
timeout_tags => ['_aggregatetimeout']
timeout_code => "event.set('avgProcessTime' , ( event.get('totalProcessTime') / event.get('counter') ) );
event.set('aggregation' , true);
event.set('entity' , 'cmserver');
event.set('AggregationFields' , '%{task_id}');
"
}
The aggregated events looks like that:
Three issues:
In AggregationFields I want to see the task_id that I set in the aggregation
I see the aggregation field with value: true. Why the if in the output doesn't apply and we are in the 'else'
entity field contains cmserver. if the 'if' from issue 2 will work, will this value be in the index name?
Thanks
Sharon.
ssasporta
(Sharon Sasporta)
November 14, 2017, 1:49pm
6
issue 2 solved
issue 3 worked too.
Issue 1 still not.
timeout_code => "event.set('avgProcessTime' , ( event.get('totalProcessTime') / event.get('counter') ) );
event.set('aggregation' , 'true');
event.set('entity' , 'cmserver');
event.set('AggregationFields' , %{task_id});
Thanks
Sharon.
That sounds like a separate issue, so I would recommend opening a new thread for that. I don't know the aggregation filter very well so will unfortunately not be able to help.
1 Like
system
(system)
Closed
December 12, 2017, 2:06pm
9
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.