Hi,
CPU usage spiked up to 3000% with the load of the box being around 400. There was no config change. This is also causing messages to pile up in the queues
This is my setup:
rsyslog -> logstash -> rabbitmq -> logstash -> elasticsearch
This is the filter:
grok {
match => [ "message", "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:time_extra}\s+%{DATA:hostname}\s+%{DATA:type_extra}:\s+%{SYSLOGTIMESTAMP:time}\s+[%{DATA:tomcat_instance}]\s+%{DATA:log_type}\s+%{DATA:servlet}\s+-\s+(uuid%{DATA:uuid}|)?%{IP:ip_address}|%{DATA:origin}|%{DATA:db_server}|%{DATA:path}|%{DATA:call_type}|%{NUMBER:TotalTime:float}|%{NUMBER:CpuTime:float}|%{NUMBER:DbTime:float}|%{NUMBER:NetworkTime:float}(|%{NUMBER:KEY_1}|%{NUMBER:VALUE_1})?(|%{NUMBER:KEY_2}|%{NUMBER:VALUE_2})?(|%{NUMBER:KEY_3}|%{NUMBER:VALUE_3})?(|%{NUMBER:KEY_4}|%{NUMBER:VALUE_4})?(|%{NUMBER:KEY_5}|%{NUMBER:VALUE_5})?(|%{NUMBER:KEY_6}|%{NUMBER:VALUE_6})?(|%{NUMBER:KEY_7}|%{NUMBER:VALUE_7})?(|%{NUMBER:KEY_8}|%{NUMBER:VALUE_8})?(|%{NUMBER:KEY_9}|%{NUMBER:VALUE_9})?(|%{NUMBER:KEY_10}|%{NUMBER:VALUE_10})?(|%{NUMBER:KEY_11}|%{NUMBER:VALUE_11})?(|%{NUMBER:KEY_12}|%{NUMBER:VALUE_12})?(|%{NUMBER:KEY_13}|%{NUMBER:VALUE_13})?(|%{NUMBER:KEY_14}|%{NUMBER:VALUE_14})?(|%{NUMBER:KEY_15}|%{NUMBER:VALUE_15})?(|%{NUMBER:KEY_16}|%{NUMBER:VALUE_16})?(|%{NUMBER:KEY_17}|%{NUMBER:VALUE_17})?(|%{NUMBER:KEY_18}|%{NUMBER:VALUE_18})?(|%{NUMBER:KEY_19}|%{NUMBER:VALUE_19})?(|%{NUMBER:KEY_20}|%{NUMBER:VALUE_20})?(|%{NUMBER:KEY_21}|%{NUMBER:VALUE_21})?(|%{NUMBER:KEY_22}|%{NUMBER:VALUE_22})?(|%{NUMBER:KEY_23}|%{NUMBER:VALUE_23})?(|%{NUMBER:KEY_24}|%{NUMBER:VALUE_24})?(|%{NUMBER:KEY_25}|%{NUMBER:VALUE_25})?(|%{NUMBER:KEY_26}|%{NUMBER:VALUE_26})?(|%{NUMBER:KEY_27}|%{NUMBER:VALUE_27})?(|%{NUMBER:KEY_28}|%{NUMBER:VALUE_28})?(|%{NUMBER:KEY_29}|%{NUMBER:VALUE_29})?(|%{NUMBER:KEY_30}|%{NUMBER:VALUE_30})?(|%{NUMBER:KEY_31}|%{NUMBER:VALUE_31})?(|%{NUMBER:KEY_32}|%{NUMBER:VALUE_32})?(|%{NUMBER:KEY_33}|%{NUMBER:VALUE_33})?(|%{NUMBER:KEY_34}|%{NUMBER:VALUE_34})?(|%{NUMBER:KEY_35}|%{NUMBER:VALUE_35})?(|%{NUMBER:KEY_36}|%{NUMBER:VALUE_36})?(|%{NUMBER:KEY_37}|%{NUMBER:VALUE_37})?(|%{NUMBER:KEY_38}|%{NUMBER:VALUE_38})?(|%{NUMBER:KEY_39}|%{NUMBER:VALUE_39})?(|%{NUMBER:KEY_40}|%{NUMBER:VALUE_40})?(|%{WORD:tag_1}|%{WORD:tag_data}|%{WORD:tag_2}|%{WORD:first_tag_data}|%{WORD:tag_3}|%{WORD:or_of_test})?" ]
remove_field => [ "time_extra", "type_extra" ]
The CPU usage was with a single instance of logstash running. My production version of logstash is 1.4. Tried with the latest version too and saw the same thing happening.
Can someone help with this?