Grok filter - issues with spaces and special charecters

Elastic Stack Logstash
1

Hi ,

I am having issues with skipping spaces and special characters which i am testing . I am trying to parse

INPUT :- [2017-05-15 00:00:07,397] :|: INFO :|: dubprdsfe33.dub.jabodo.com

I am using following

input {
file {
path => "/apache/logs/download.log"
start_position => "beginning"
}
}
filter {
grok {
match => { "message" => "[%{DATESTAMP:timestamp}]%{SPACE}:|:%{SPACE}%{WORD:severity}%{SPACE}"%{WORD:hostname}}
}
}
output {
stdout { codec => rubydebug }
}

Can someone advise on this.

2

Did you forget the second set of :|: after "INFO"?

3

Now its working with
filter {
grok {
match => { "message" => "%{DATESTAMP:timestamp}]%{SPACE}:|:%{SPACE}%{WORD:severity}%{SPACE}:|:%{SPACE}%{WORD:hostname}"}
}
}

with input [2017-05-15 00:00:07,397] :|: INFO :|: dubprdsfe33.dub.jabodo.com

Thanks,

Now i am trying to parse another field in the log --9e346ae5ad894ce882d271d7ecfba227

INPUT :-[2017-05-13 13:24:35,152] :|: INFO :|: dubprdfe33.dub.jabodo.com :|:9e346ae5ad894ce882d271d7ecfba227

How can i define the last field ( 9e346ae5ad894ce882d271d7ecfba227)

Nikhil

4

I'm assuming it is randomly created so you can use %{NOTSPACE}

5

can you please guide me how i can use %{NOTSPACE} or will dreedydata will work ??

%{SPACE}:|:%{SPACE}%{GREEDYDATA:class}

6

Greedydata should also work, but NOTSPACE just pulls every character until there is a space. So it would be %{SPACE}:|:%{SPACE}%{WORD:hostname}%{SPACE}:|:%{NOTSPACE:UniqueID}.

7

I tried using %{SPACE}:|:%{SPACE}%{GREEDYDATA:class}

and - %{NOTSPACE:coinid_key}
grok {
match => { "message" => "%{DATESTAMP:timestamp}\]%{SPACE}:|:%{SPACE}%{WORD:level}%{SPACE}:|:%{SPACE}%{WORD:hostname}%{SPACE}:|:%{NOTSPACE:coinid_key}"}
}

for input - [2017-05-15 00:00:07,751] :|: INFO :|: dubprdsfe33.dub.jabodo.com :|: 26baee2d30164c2083e380b40b2a3b15

but not working .

OUTPUT

{
"path" => "/apache/logs/download.log",
"@timestamp" => 2017-05-15T19:27:46.876Z,
"@version" => "1",
"host" => "ip-172-31-4-117",
"message" => "[2017-05-15 00:00:07,751] :|: INFO :|: dubprdsfe33.dub.jabodo.com :|: 26baee2d30164c2083e380b40b2a3b15 ",
"tags" => [
[0] "_grokparsefailure"
]
}

Can you please advise

8

Maybe it has to do with this bracket

9

Date and timestamp are always coming in bellow format
[2017-05-15 00:00:07,397]
[2017-05-15 00:00:07,564]
[2017-05-15 00:00:07,877]
[2017-05-15 00:00:07,966]
[2017-05-15 00:00:07,988]

when i use it without GREEDYDATA and NOTSPACE . It works fine . But if i add it ..then it breaks .

Is there any other way i can define --%{DATESTAMP:timestamp}] ---other than this .
NOTE- i am not using [ at the start to skip [ . If i use then it wot work

Thanks

10

I think the problem is with===== dubprdsfe33.dub.jabodo.com

How can i take this as a whole field...consider as hostname

11

Solved with -%{USERNAME:hostname}

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.