Hi shyamari,
Sample logstash file patterns
grok{
match => {"message" => "\[%{WORD:logLevel}\] %{WORD:logType}->\|datetime:%{NOTSPACE:time}\+0000\|hostname:%{HOSTNAME:hostname}/%{IPV4:ip}\(%{IPV4}\)\|threadId:%{NOTSPACE:threadId}\|userId:%{WORD:userId}\|id:%{NOTSPACE:id}\|applicationName:%{NOTSPACE:applicationName}\|className:%{NOTSPACE:className}\|logMessage:SERVICE_DEPENDENCY: CALL-COMPLETED From %{NOTSPACE:fromService} To %{NOTSPACE:toService} on Uri %{URI:targetUri} with latency %{NUMBER:latency}"}
add_tag => ["service_dependency"]
}
if "_grokparsefailure" in [tags]{
grok{
remove_tag => ["_grokparsefailure"]
match => {"message" => "\[%{WORD:logLevel}\] %{WORD:logType}->\|datetime:%{NOTSPACE:time}\+0000\|hostname:%{HOSTNAME:hostname}/%{IPV4:ip}\(%{IPV4}\)\|threadId:%{NOTSPACE:threadId}\|userId:%{WORD:userId}\|id:%{NOTSPACE:id}\|applicationName:%{NOTSPACE:applicationName}\|className:%{NOTSPACE:className}\|logMessage:%{GREEDYDATA:logMessage}"}
add_tag => ["applog"]
}
}
if "_grokparsefailure" in [tags]{
grok{
remove_tag => ["_grokparsefailure"]
match => {"message" => "\[%{WORD:logLevel}\] %{WORD:logType}->\|datetime:%{NOTSPACE:time}\+0000\|hostname:%{HOSTNAME:hostname}/%{IPV4:ip}\(%{IPV4}\)\|threadId:%{NOTSPACE:threadId}\|id:%{NOTSPACE:id}\|className:%{NOTSPACE:className}\|logMessage:%{GREEDYDATA:logMessage}"}
add_tag => ["applog"]
}
}
if "_grokparsefailure" in [tags]{
grok{
remove_tag => ["_grokparsefailure"]
match => {"message" => "\[%{WORD:logLevel}\] %{WORD:logType}->\|datetime:%{NOTSPACE:time}\+0000\|hostname:%{HOSTNAME:hostname}/%{IPV4:ip}\(%{IPV4}\)\|threadId:%{NOTSPACE:threadId}\|userId:%{WORD:userId}\|id:%{NOTSPACE:id}\|applicationName:%{NOTSPACE:applicationName}\|className:%{NOTSPACE:className}"}
add_tag => ["applog"]
}
}
like this i am having so many patterns , because every log message comes with different fields.