How to create filter to handle spaces & commas in street addresses & person names?

[hc-jgregory (Jason Gregory)]

Sample text from log (comes in a segment without quotes enclosing field and values as expected if true JSON):

{Field1=1111 NE 1ST AVE,Field2=ATLANTA, GA 30309-1111,Field3=JOHN ALLEN DOE}

        filter {
            grok {
                    match => { "message" => "%{TIMESTAMP_ISO8601:log_timestamp}%{SPACE}%{GREEDYDATA:log_header}\{%{GREEDYDATA:json_detail}\}%{GREEDYDATA:log_footer}" }
            }
            date {
                    match => ["log_timestamp","yyyy-MM-dd HH:mm:ss"]
            }
            kv {
                    source => "json_detail"
                    trim_key => "<>\[\]\{\},"
                    trim_value => "<>\[\]\{\},"
                    allow_duplicate_values => false
            }
            mutate {
                    remove_field => ["message","log_timestamp","log_header","log_footer"]
            }
    }

Thanks!

Jason

[Robo]

If the json_detail has always the same format, maybe you can use dissect filter here:

filter {
    dissect {
      mapping => {
        #separated fields
        "message" => "Field1=%{address},Field2=%{city},Field3=%{name}"
        #one field
        "message" => "Field1=%{contact},Field2=%{+contact},Field3=%{+contact}"
      }
    }
  }