Logstash use 100%+ cpu with simple beat input

[hckim (Hichul Kim)]

Hi
I am setting up filebeat -> logstash -> rabbitMQ ->elasticsearch
some how logstash simple config is using 100%+ cpu

is this normal ? or am I missing some thing

filebeat ( V1.3.1)
logstash ( V2.4.1)
cpu : AMD Opteron(tm) Processor 6376 32core
ram: 128G

logstash config ---------

input {
beats {
port => 5044
ssl => true
ssl_certificate => "/cert/cert.crt"
ssl_key => "/cert/cert.key"
ssl_key_passphrase => "passphrase"
}
} #end of input

filter {
} #end of Filter

output {
}#end of output

LS_HEAP_SIZE=4G

ps -ef|grep logstash
conse 16587 1 99 12:27 pts/0 00:28:01 /app/jdk1.8.0_51/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Xmx4G -Xss2048k -Djffi.boot.library.path=/app/logstash/vendor/jruby/lib/jni -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/app/logstash/heapdump.hprof -Xbootclasspath/a:/app/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/app/logstash/vendor/jruby -Djruby.lib=/app/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main --1.9 /app/logstash/lib/bootstrap/environment.rb logstash/runner.rb agent -f /app/logstash/config/aa.conf --verbose -l /data/logstash/logstash.log

and most of cpu are used by beat input

top -Hp 16587
26473 conse 20 0 11.729g 1.017g 20488 S 70.1 0.8 3:10.07 [main]<beats
16632 conse 20 0 11.729g 1.017g 20488 S 4.0 0.8 1:02.98 java
16631 conse 20 0 11.729g 1.017g 20488 S 2.0 0.8 0:19.29 java
16611 conse 20 0 11.729g 1.017g 20488 S 1.7 0.8 0:11.49 java
16613 conse 20 0 11.729g 1.017g 20488 S 1.7 0.8 0:11.74 java

filebeat config --------
############################# Shipper #########################################

shipper:
name: "172.16.100.203"
tags: ["epsLog"]

############################# Filebeat
filebeat:

spool_size, idle_timeout is for network

spool_size: 4089
idle_timeout: 5s

registry_file: /data/beat/filebeat1.point

prospectors:
-
paths:
- /manager/log/*
exclude_lines: ["^#"]
exclude_files: [ "dnscount.log", "pe.log", "stdout.log", "communication.log", "dns-passivedns.log", "pcr.log", "stderr.log" ]
fields_under_root: true
backoff: 1s
scan_frequency: 500ms
harvester_buffer_size: 6000
tail_files: true

############################# Output
output:

Logstash as output

logstash:
# The Logstash hosts
hosts: [ "5.5.5.5:5044" ]
# Number of workers per Logstash host.
worker: 2

# Optional load balance the events between the Logstash hosts
loadbalance: false

# Optional index name. The default index name depends on the each beat.
# For Packetbeat, the default is set to packetbeat, for Topbeat
# top topbeat and for Filebeat to filebeat.
index: logs

compression_level: 0
bulk_max_size: 2048

tls:
  # List of root certificates for HTTPS server verifications
  certificate_authorities: ["/cert/cert.crt"] 
  insecure: true
  min_version: '1.2'

file:
path: "/data/beat"
filename: eps.file
rotate_every_kb: 50000 #=50M
number_of_files: 7

############################# Logging

There are three options for the log ouput: syslog, file, stderr.

Under Windos systems, the log files are per default sent to the file output,

under all other system per default to syslog.

logging:

to_syslog: false

to_files: true

files:
path: /data/beat
name: eps.log
rotateeverybytes: 50485760 # = 50MB
keepfiles: 7
level: info

from other server filebeat is sending about 5000 +- event per second
2017-02-02T12:42:50+09:00 INFO Events sent: 4089
2017-02-02T12:42:50+09:00 INFO Registry file updated. 60 states written.
2017-02-02T12:42:51+09:00 INFO Events sent: 4089
2017-02-02T12:42:51+09:00 INFO Registry file updated. 60 states written.
2017-02-02T12:42:51+09:00 INFO Events sent: 4089
2017-02-02T12:42:51+09:00 INFO Registry file updated. 60 states written.
2017-02-02T12:42:52+09:00 INFO Events sent: 4089
2017-02-02T12:42:52+09:00 INFO Registry file updated. 60 states written.
2017-02-02T12:42:53+09:00 INFO Events sent: 4089
2017-02-02T12:42:53+09:00 INFO Registry file updated. 60 states written.
2017-02-02T12:42:53+09:00 INFO Events sent: 4089
2017-02-02T12:42:53+09:00 INFO Registry file updated. 60 states written.