Beats-pipeline.conf Spikes CPU 100% w/2+ outputs

Hi,
Outside of the individual performance of the targets I haven't seen any examples like this. Doesn't seem to matter which output I use as long as I only use one so I don't feel target performance is the issue and I may have a yml format problem with what is posted below.

I don't see how I could solve this in a pipeline either.

Q: Are multiple output supported in a single pipeline or could someone add an example or reference one in Documentation?

Thanks !

input {
beats {
port => 5044
}
}

output{
elasticsearch {
hosts => "elasticsearch.svr:9200"
sniffing => false
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
} # End elasticsearch

  if [type]=="wineventlog" and [level] == "Error" or [level] == "Warning" {
    http {
    user=>"apiusername"
    password=>"#########"
    http_method=>"post"
    format=>"message"
    content_type => "application/json"
    url=>"http://monitoring.svr/api/alert"
    #message =>'{"message":"%{name} %{log_name}"}'
    message =>'{"aligned_resource":"/device/5013","message":"%{host} %{log_name} %{level} EventID:%{event_id} %{message}"}'
    } # End http
  } # End if

  if [type]=="wineventlog" {
    sumologic {
    url => "https://endpoint1.collection.us2.sumologic.com/""
    compress => true
    compress_encoding => "gzip"
    source_name => "%{host}"
    source_category => "windows"
    format => "%{@timestamp} %{@json}"
    } # End Sumo
  } # End if

} # End Out

You can have multiple outputs in the same pipeline, with the caveat that if one output isn't able to process events it'll stall the whole pipeline and prevent events from being sent to the rest of the outputs.

Issue seems to have been fixed by tuning the SumoLogic output.
automatic_retries => 10
validate_after_inactivity => 3
request_timeout => 10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.