Creating index pattern


(Kalyan Mb) #1

Hi,

i have the log format as below,
log file:

pid: 1131 start: 0 stacksize: 0 breaksize: 9 command: tuned arguments: /usr/bin/python -Es /usr/sbin/tuned -l -P
pid: 1163 start: 0 stacksize: 0 breaksize: 11 command: redis-server arguments: /usr/bin/redis-server *:100:

filter in logstash.conf file


else if [log_type] == "pro_logs" {
dissect { mapping => {'message' => '%{pro_id->} %{start->} %{stack_size->} %{breaksize_val->} %{command->} %{msg->}'}}
}
}

is this filter correct, because i am not able to see the proper parsed output in kibana for this log file.


#2

No, it is not correct. Try

dissect { mapping => { "message" => "pid: %{pid} start: %{start} stacksize: %{stacksize} breaksize: %{breaksize} command: %{command} arguments: %{arguments}" } }

(Kalyan Mb) #3

Thanks @Badger the solution didn't solve my problem though, i am using below filter with grok RE.

grok { match => { "message" => "%{DATA}: %{INT:pid_value} %{DATA}: %{INT:start} %{DATA}: %{INT:stacksize} %{DATA}: %{INT:breaksize} %{DATA}: %{WORD:command_val} %{DATA}: %{WORD:argument_val} %{GREEDYDATA:args}" }

filebeat.yml:

  • input_type: log
    paths:
    #- /home/vankata/190_APS_QUALIFICATION/kalyan_elk_logs/new_working_dir/Process_heap*.txt
    • /home/vankata/190_APS_QUALIFICATION/kalyan_elk_logs/new_working_dir/test_dir/Process_heap*.txt
      encoding: utf-8
      fields_under_root: true
      document_type: log
      fields:
      service_name: pro
      app_name: cfx_perf_logs
      log_type: pro_logs

Process_heap_09_09_23_40.txt Process_heap_09_09_07_40.txt these two files present in the log directory.

file diff between above files..

Problems that i am facing with above configurations
1._grokparsefailure for both the file inputs initially.


2.If the log directory consists more than 50 files, index pattern is node done properly.
Eg:
this doesnt have index for icscf where as input file has an entry for the same.
[root@perfDatastore new_working_dir]# cat Process_heap_09_09_21_40.txt |grep icscf | more
pid: 25925 start: 0 stacksize: 0 breaksize: 17 command: icscf arguments: IMS_ICSCF01 -n IMS_CMPROXY
pid: 25926 start: 0 stacksize: 0 breaksize: 19 command: icscf arguments: IMS_ICSCF02 -n IMS_CMPROXY
pid: 25927 start: 0 stacksize: 0 breaksize: 17 command: icscf arguments: IMS_ICSCF03 -n IMS_CMPROXY