Eliminate excesive blanks or select fields for position

Elastic Stack Logstash
1

Hello,

I'd like it if someone could help me with the following problem.

I'm sending a series of logs to logstash/elastic.

The problem is that my logs are separated by blanks, and sometimes after the first field, there can be more than one blank. If I set separator => " ", many empty fields are created because I have many blanks in a row.
I want to replace excessive blanks with a single blank.
I would also like to select the fields by position since they always appear in the same position.
I've attached a log sample.

Thanks

Captura de pantalla 2025-05-21 173456
Captura de pantalla 2025-05-21 1734561244×113 62 KB

2

You could use a gsub in the mutate filter to replace one or more blank space by just one blank space or a different separator.

Can you share a plain text sample of your messages so this can be tested?

3

Ok, thanks

EXW1.POTHYEW.APOTHYE.D250521.T112630.N01     000000011904 000049 LOYTHT 20250521 112654 VBLE 000496 000000000 POR.POTHYEW.R0049.APOTHYE.F250521.H1126.N01
EXW1.POTHYEW.APOTHYE.D250521.T112634.N01     000000011904 000049 OUJYHG 20250521 112654 VBLE 000496 000000000 POR.POTHYEW.R0049.APOTHYE.F250521.H1126.N02
FICF34.ENV                                   000000001040 000078 JUGHGF 20250521 134151 FIJO 000260 000000000 POR.POTHYEW.R0078.ATPVF34.F250521.H1342.N01
TL.SIHTP100.IUYRH43E.R008910.SKUKIV1         000000008720 000182 HSHHDU 20250521 010009 FIJO 000080 000000000 POR.POTHYEW.R0182.AOPYEV1.F250521.H0101.N01
4

I can think of three ways to do this

    grok { match => { "message" => "^%{NOTSPACE:field1}\s+%{NOTSPACE:field2}\s+%{NOTSPACE:field3}\s+%{NOTSPACE:field4}\s+%{NOTSPACE:field5}\s+%{NOTSPACE:field6}\s+%{NOTSPACE:field7}\s+%{NOTSPACE:field8}\s+%{NOTSPACE:field9}\s+%{NOTSPACE:field10}" } }

or

    mutate { gsub => [ "message", "\s+", " " ] }
    csv { source => "message" separator => " " }

or

    mutate { gsub => [ "message", "\s+", " " ] }
    dissect { mapping => { "message" => "%{field1} %{field2} %{field3} %{field4} %{field5} %{field6} %{field7} %{field8} %{field9} %{field10}" } }

If you had fixed width fields with no spaces like

EXW1.POTHYEW.APOTHYE.D250521.T112634.N01    000000011904000049OUJYHG20250521112654VBLE000496000000000POR.POTHYEW.R0049.APOTHYE.F250521.H1126.N02
FICF34.ENV                                  000000001040000078JUGHGF20250521134151FIJO000260000000000POR.POTHYEW.R0078.ATPVF34.F250521.H1342.N01

You could use

grok { match => { "message" => "^(?<field1>.{44})(?<field2>.{12})(?<field3>.{6})(?<field4>.{6})(?<field5>.{8})(?<field6>.{6})(?<field7>.{4})(?<field8>.{6})(?<field9>.{9})(?<field10>.{42})" } }
2 Likes
5

Great, thank you so much.
I'll try it out.

Best regards