log file excerpt:
2017-11-06 11:54:43,333; [LOG_LEVEL=ALWAYS, CMPNT_NM=com.fmr.ifeb.alayer.cache.ehcache.ApplicationCacheImpl, MESSAGE=Initialized cache named 'oscarJdbcDaxCache']
filebeat.yml
---
filebeat.prospectors:
-
document_type: springlog
input_type: log
#multiline.match: after
#multiline.negate: true
#multiline.pattern: "^\\[[0-9]{4}-[0-9]{2}-[0-9]{2}"
paths:
- "C:\\Users\\a617744\\Newdata\\data6.log"
#tail_files: true
logging.level: debug
output.logstash:
hosts:
- "localhost:5044"
logstash config file:
input {
beats {
port => 5044
}
}
filter {
mutate{
gsub=>["message","\r",""]
}
grok {
id => "myspringlogfilter"
match => { "message" => [ "(?m)^%{TIMESTAMP_ISO8601:timestamp}; [LOG_LEVEL=%{LOGLEVEL:log-level}, CMPNT_NM=%{NOTSPACE:component}, MESSAGE=%{QUOTEDSTRING:restmessage}]"]}
overwrite => ["message"]
}
}
output {
elasticsearch {
hosts => "localhost:9200"
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
#index => "filebeat"
document_type => "%{[@metadata][type]}"
}
stdout {
codec => rubydebug
}
logstash log file looks like:
[2017-12-08T20:11:12,522][DEBUG][io.netty.handler.ssl.CipherSuiteConverter] Cipher suite mapping: TLS_PSK_WITH_RC4_128_SHA => PSK-RC4-SHA
[2017-12-08T20:11:12,522][DEBUG][io.netty.handler.ssl.CipherSuiteConverter] Cipher suite mapping: SSL_PSK_WITH_RC4_128_SHA => PSK-RC4-SHA
[2017-12-08T20:11:12,522][DEBUG][io.netty.handler.ssl.CipherSuiteConverter] Cipher suite mapping: TLS_RSA_WITH_RC4_128_MD5 => RC4-MD5
[2017-12-08T20:11:12,522][DEBUG][io.netty.handler.ssl.CipherSuiteConverter] Cipher suite mapping: SSL_RSA_WITH_RC4_128_MD5 => RC4-MD5
[2017-12-08T20:11:12,532][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2017-12-08T20:11:12,552][DEBUG][io.netty.channel.MultithreadEventLoopGroup] -Dio.netty.eventLoopThreads: 4
[2017-12-08T20:11:12,645][DEBUG][io.netty.channel.nio.NioEventLoop] -Dio.netty.noKeySetOptimization: false
[2017-12-08T20:11:12,647][DEBUG][io.netty.channel.nio.NioEventLoop] -Dio.netty.selectorAutoRebuildThreshold: 512
[2017-12-08T20:11:12,685][INFO ][logstash.pipeline ] Pipeline started {"pipeline.id"=>"main"}
[2017-12-08T20:11:12,713][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2017-12-08T20:11:12,688][DEBUG][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x35a4651d@C:/Users/a617744/logstash-6.0.0/logstash-core/lib/logstash/pipeline.rb:290 run>"}
[2017-12-08T20:11:12,911][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2017-12-08T20:11:12,916][DEBUG][io.netty.channel.DefaultChannelId] -Dio.netty.processId: 8956 (auto-detected)
[2017-12-08T20:11:13,042][DEBUG][io.netty.util.NetUtil ] Loopback interface: lo (Software Loopback Interface 1, 127.0.0.1)
[2017-12-08T20:11:13,044][DEBUG][io.netty.util.NetUtil ] \proc\sys\net\core\somaxconn: 200 (non-existent)
[2017-12-08T20:11:13,094][DEBUG][io.netty.channel.DefaultChannelId] -Dio.netty.machineId: 00:50:56:ff:fe:b4:1d:75 (auto-detected)
[2017-12-08T20:11:17,692][DEBUG][logstash.pipeline ] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#<Thread:0x35a4651d@C:/Users/a617744/logstash-6.0.0/logstash-core/lib/logstash/pipeline.rb:290 sleep>"}
[2017-12-08T20:11:22,699][DEBUG][logstash.pipeline ] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#<Thread:0x35a4651d@C:/Users/a617744/logstash-6.0.0/logstash-core/lib/logstash/pipeline.rb:290 sleep>"}