How to strip white space from all fields in logstash

Elastic Stack Logstash
1

I am creating fields using grok match with position and resulting json fields contain white spaces.

I need to trim the white space from all fields .

input { stdin { } }

filter {
  grok {
   
       match => { "message" => "(?<COMP_VER_NO>^.{6})(?<f1>.{2})(?<f2>.{20})(?<FILLER_CM>.{8})(?<f3>.{1})(?<f4>.{1})(?<f5>.{18})(?<f6>.{1})(?<f7>.{20})(?<f8>.{3})(?<f9>.{3})(?<f10>.{8})(?<a1>.{6})(?<CRD_FI_ID>.{5})(?<f1>.{19})(?<a2>.{2})(?<CLIENT_NM>.{30})(?<a3>.{3})(?<a4>.{5})(?<a5>.{8})(?<a6>.{5})(?<aa7>.{1})(?<a8>.{8})(?<a9>.{8})" }

}

#I want to trim all fields created by above grok


}

output {

  stdout { codec => rubydebug }
}

how can I trim all the fields in resulting json ?

thanks for reading and helping.

2

Check this might help

https://discuss.elastic.co/t/how-to-trim-zero-or-more-leading-and-trailing-spaces-from-key-value-pairs/117933/7

3

You have two fields called f1 which results in f1 being an array. I renamed the second one to be f11. Then this works:

    ruby {
        code => '
            event.to_hash.each { | k, v|
                if v.is_a? String
                    event.set(k, v.strip)
                end
            }
        '
    }

@elasticforme the technique used in the post you linked to does not work because it relies on removing spaces around field delimiting characters. Rahul's data is not delimited, it is all fixed width -- see how every single grok field has a fixed width set using {n}

1 Like
4

I am trying to understand this. is it saying

event.to_hash.each {|k, v| -> get each key-value pair
if v.is_a? string -> if value is a string then
event.set(k, v.strip) -> set key as is, and strip space from value

is this correct assumption?

5

I dont have any delimiter as badger informed. I can only use position based regex to match my fields within grok pattern

6

Yes, you are reading it correctly. It strips leading and trailing (but not embedded) spaces from the value.

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.