Kalyan_MB
(Kalyan Mb)
April 16, 2019, 5:42am
1
Hi,
i have the log format as below,
log file:
pid: 1131 start: 0 stacksize: 0 breaksize: 9 command: tuned arguments: /usr/bin/python -Es /usr/sbin/tuned -l -P
pid: 1163 start: 0 stacksize: 0 breaksize: 11 command: redis-server arguments: /usr/bin/redis-server *
filter in logstash.conf file
else if [log_type] == "pro_logs" {
dissect { mapping => {'message' => '%{pro_id->} %{start->} %{stack_size->} %{breaksize_val->} %{command->} %{msg->}'}}
}
}
is this filter correct, because i am not able to see the proper parsed output in kibana for this log file.
Badger
April 16, 2019, 12:09pm
2
No, it is not correct. Try
dissect { mapping => { "message" => "pid: %{pid} start: %{start} stacksize: %{stacksize} breaksize: %{breaksize} command: %{command} arguments: %{arguments}" } }
Kalyan_MB
(Kalyan Mb)
April 17, 2019, 2:24pm
3
Thanks @Badger the solution didn't solve my problem though, i am using below filter with grok RE.
grok { match => { "message" => "%{DATA}: %{INT:pid_value} %{DATA}: %{INT:start} %{DATA}: %{INT:stacksize} %{DATA}: %{INT:breaksize} %{DATA}: %{WORD:command_val} %{DATA}: %{WORD:argument_val} %{GREEDYDATA:args}" }
filebeat.yml:
input_type: log
paths:
#- /home/vankata/190_APS_QUALIFICATION/kalyan_elk_logs/new_working_dir/Process_heap*.txt
/home/vankata/190_APS_QUALIFICATION/kalyan_elk_logs/new_working_dir/test_dir/Process_heap*.txt
encoding: utf-8
fields_under_root: true
document_type: log
fields:
service_name: pro
app_name: cfx_perf_logs
log_type: pro_logs
Process_heap_09_09_23_40.txt Process_heap_09_09_07_40.txt these two files present in the log directory.
file diff between above files..
Problems that i am facing with above configurations
1._grokparsefailure for both the file inputs initially.
2.If the log directory consists more than 50 files, index pattern is not done properly.
Eg:
this doesnt have index for icscf where as input file has an entry for the same.
[root@perfDatastore new_working_dir]# cat Process_heap_09_09_21_40.txt |grep icscf | more
pid: 25925 start: 0 stacksize: 0 breaksize: 17 command: icscf arguments: IMS_ICSCF01 -n IMS_CMPROXY
pid: 25926 start: 0 stacksize: 0 breaksize: 19 command: icscf arguments: IMS_ICSCF02 -n IMS_CMPROXY
pid: 25927 start: 0 stacksize: 0 breaksize: 17 command: icscf arguments: IMS_ICSCF03 -n IMS_CMPROXY
system
(system)
Closed
May 15, 2019, 2:24pm
4
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.