Grokparsefailure and type problems in logstash configuration file

[Mathis_Gourc]

Hi everybody,

I have several problems with my configuration file. My goal is to parse three types of logs (for the moment). Here they are :

[29/05/2020 07:41:51.354] - ih912865 - 10.107.119.121 - 93 - Transaction 7635 COMPLETED 318 ms wait time 3183 ms
[29/05/2020 10:30:01.318] - Process status database sync - us1salx08167.corpnet2.com:8400(#52279) (load 0 grace period 5 minutes) : current date 2020/02/02 21:30:01 update date 2020/02/02 21:29:58 old state OK new state OK
   31730  31626  464 10980020     52:25 /plw/modules/bin/Lx86_64/opx2-intranet.exe -I /plw/modules/bin/Lx86_64/opx2-intranet.dxl -H /plw/modules/bin/Lx86_64 -L /plw/PLW_PROD/modules/preload-intranet.ini -- plw-sysconsole -port 8400 -logdir /plw/PLW_PROD/httpdocs/admin/log/ -slaves 2

Two of these logs can be in slave files named intranet-2020-06-25-8401.log or intranet-2020-06-25-8400.log the last one is in a master file named intranet-2020-06-25-8402.log
For my tests I simplified the architecture of my log files, so I have a Log-test folder in which I put a slave file and a master file.

In these files I only put the corresponding logs and a different log to be able to see how to manage this case.

Here is the content of a "slave" :

[29/05/2020 07:41:51.354] - ih912865 - 10.107.199.125 - 93 - Transaction 7635 COMPLETED 318 ms wait time 3183 ms
[29/05/2020 10:30:01.318] - Process status database sync - us1salx08167.corpnet2.com:8400(#52279) (load 0 grace period 5 minutes) : current date 2020/02/02 21:30:01 update date 2020/02/02 21:29:58 old state OK new state OK
[29/05/2020 13:49:20.635] - Main process - Transaction SYSTEM 105238-12 SQL done 1 ms

Here is the content of a "master" :

   31730  31626  464 10980020     52:25 /plw/modules/bin/Lx86_64/opx2-intranet.exe -I /plw/modules/bin/Lx86_64/opx2-intranet.dxl -H /plw/modules/bin/Lx86_64 -L /plw/PLW_PROD/modules/preload-intranet.ini -- plw-sysconsole -port 8400 -logdir /plw/PLW_PROD/httpdocs/admin/log/ -slaves 2
[26/06/2020 21:38:01.386] - Main process - Starting HTTP service on port 8402 (socket #<MULTIVALENT stream socket waiting for connection at */8402 @ #x1022d2ddbb2>)

Now that you have a better understanding of my environment and my purpose, here's the problem. When I launch my logstash configuration, I retrieve my data in kibana. But kibana shows me that each log has been treated as coming from a slave file while I also have a log coming from a master file which doesn't have the same processing.

For a better understanding here is my configuration file :

input {
	file { 
		path => "/home/mathis/Documents/**/intranet*.log"
		exclude =>"*8402.log"
		sincedb_path => '/dev/null'
		start_position => beginning
		type => "slave"
	}
	file { 
		path => "/home/mathis/Documents/**/intranet*8402.log"
		sincedb_path => '/dev/null'
		type => "master"
	}
}
filter {
	if [type] == "slave" {
		grok {
			match => { "message" => ["\[%{DATESTAMP:eventtime}\] \- %{USERNAME:user} \- %{IPV4:clientip} \- %{NUMBER} \- %{WORD} %{NUMBER:exectime} %{WORD} %{NUMBER:time} %{GREEDYDATA:data} %{NUMBER:waittime}","\[%{DATESTAMP:eventtime}\] \- Process status database sync \- %{WORD}\.%{WORD}\.%{WORD}\:%{NUMBER:slavenumb}\(\#%{NUMBER}\) \(load %{NUMBER:nbutilisateur} grace period 5 minutes\) %{GREEDYDATA}"] }
			remove_field => "message"
		}
	
	date {
        		match => [ "eventtime", "dd/MM/YYYY HH:mm:ss.SSS" ]
			target => "@timestamp"
 		}
	}
	if [type] == "master" {
		grok {
        		match => {"message" => ["%{NUMBER}%{SPACE}%{NUMBER}%{SPACE}%{NUMBER}%{SPACE}%{NUMBER}%{SPACE}(?<starttime>((?!<[0-9])%{HOUR}:)?%{MINUTE}(?::%{SECOND})(?![0-9]))"]}
        		remove_field => "message"
		}
    		date {
        		match => [ "starttime", "HH:mm:ss","mm:ss" ]
    		}
	}
		
	
}
output {
	elasticsearch {
		hosts => "127.0.0.1:9200"
		index => "logstash-local3-%{+YYYY.MM.dd}"
	}
}

And now this is what kibana shows me:

As you can see, the type field is slave for all logs but we can also observe that the logs of the slave file "intranet-2020-06-25-8401.log" are correctly parsed and that the line of added log that does not interest me has the field tags _grokparsefailure (the middle line in the picture).

The other problem is that the other logs (the first two lines on the image) are from a slave file (which is not true) according to kibana, so I guess they are processed in my first grok which would explain why they also have the _grokparsefailure tags field.

So I guess there are several errors in my input and filter part. I've been searching for a long time and doing a lot of testing, could you help me fix my config file please?