How to filter Data with Spaces in Logstash

Elastic Stack Logstash
1

Hello Folks, i need some Help here to read this Example Data in my Text editor with its Spaces in Logstash.
Example Data in my Text editor as follows:

                 Hello            World

         Hello back

Please notice the Spaces between the worlds and the Spaces between the Start in each row, and the empty Line between those two Sentences.

How can i let Logstash read this Example .
i tried with follows , but i could not let Logstash jump to a new Line with Space at the Beginning.

input{{ ...}}

filter{
grok { match => { "message" => "%{SPACE}%{WORD:hello}%{SPACE}%{WORD:world}(?[\r\n]+)%{SPACE}{WORD:hi}%{SPACE}%{WORD:you}"}
}}

output{{.....}}

Can anybody help me please to correct my filter ?
thx.

2

HI samyo,
Can you try with this pattern,

\s%{WORD:data1}\s+%{WORD:data2}

3

Please let me know if you need any further info.

4

Hi Daniel37 and Ganesh2303:

i tried this \s%{WORD:data1}\s+%{WORD:data2} and it did not work correctly .
i tried this \s+%{WORD:data1}\s+%{WORD:data2} it works for this follows example:

                                 Hello World

i noticed that \s+%{WORD:data1}\s+%{WORD:data2} can read the first empty line in my text editor and than some whiteSpaces until the "Hello" and the second \s+ let me just add one whitespace between each other word nothing else.
it does not allowed me to jump to next empty Line , or to read the Example above .

i need to read the Example above correctly with all whitespaces , and all empty Lines.

Can you please help me , i even tried with this
match => { "message" => "\s+%{WORD:data1}\s+\r%{WORD:data2}"}
in order to try to jump to next empty Line.

i tried this too
match => { "message" => "\s+%{WORD:data1}\s+\n%{WORD:data2}"}
Nothing works until now.

5

Hi Ganesh2303:
i tried your
\s%{WORD:data1}\s+%{WORD:data2}
and it did not work correctly.
it could not read empty Line or jump into after empty Line with Whitespaces.

with this:
\s+%{WORD:data1}\s+%{WORD:data2}
i could read just
Hello World

6

Yeah this is your message so it will retrieve the data without those whitespaces right

7

i need to let logstash jump after the empty Line and get the rest of my above example "Hello back" .
as soon as Logstash see an empty Line after "Hello World" , it will return nothing after that.
That is the Issue.
Do you have any suggestion?

8

So you are saying that 2 line are single message am i right

9

i did not understand correctly what you meant with "2 Lines are single message" ?
i need to let Logstash read my "Hello back", that means after reading the "Hello World" , it has to jump after any empty Line he sees and get the Rest "Hello Back".

10

Ok, Can you try this and let me know,

\s%{WORD:data1}\s+%{WORD:data2}\n\s+%{WORD:data3}\s%{WORD:data4}

11

Logstash is line to line read process, if your message is single line no issues but if you want to process 1 or 2 line message we need to configure according to that message.

12

it did not work , the "\n" did not help to jump after an empty Line.

14

As per my understand,

hello world is one line,
empty space is one line,
hello back is one line am i right

15

example
example.jpg1008×630 45.5 KB

16

example%202
example 2.jpg1008×630 34.9 KB

17

Try with drop command if the line is empty those line need to skip

18

that is my Filter , an it does not work, i tried to use drop comand too.

filter {

grok {match => { "message" => "\s%{WORD:data1}\s+%{WORD:data2}\n\s+%{WORD:data3}\s%{WORD:data4}"}
}
if [message] =~ /^\s*$/ {
drop { }
}
}

19

Sorry for Late answer, cause this tool here blocked me for 20 Hour cause of reaching my maximum text level. so i am just able to modify my old Comments in order to ask for help.
i tried to put drop Command like follows, but still could not read the Example above correctly,
any Suggestions.
filter {
if [message] =~ /^\s*$/ {
drop { }
}
grok {match => { "message" => "\s%{WORD:data1}\s+%{WORD:data2}\n\s+%{WORD:data3}\s%{WORD:data4}"}
}
}

20

Use your drop filter first and then use filter condition, because intially your filter works first and then only it will goes for drop condition as per your configuration.

21

Hello Ganesh33, sorry for late Response,
so i did do what you said about drop Command at Beginning of my filter,
filter {
if [message] =~ /^\s*$/ {
drop { }
}
grok {match => { "message" => "\s%{WORD:data1}\s+%{WORD:data2}\n\s+%{WORD:data3}\s%{WORD:data4}"}
}
}

it is still does not work to tell Logstash to jump over an empty Line , and jump the whitespaces until reach the "Hello Back" sentence and read it. as you can see in the above Picture.
the "\n\s" did not do anything to let Logstash reach the "Hello Back" sentence.

Any Suggestion?