Issue with Weird spaces with Filebeat in windows Log

[enrique.villar (Enrique Lorenzo Villar Mavila)]

Good day. I try to index some windows (MSSQL) logs. This logs are in plain text.

Some logs are multiline

So, in Filebeat is configure:

- type: log
  enabled: true
  paths:
    - D:\logs\*.log #example path
  multiline.pattern: '^<Header|^\S*?!'
  multiline.negate: true
  multiline.match: after
  tags: ["reporting_indra"]
output.logstash:
    hosts: ["elk:5044"]
# The rest options are by default.

In logstash I have a filter to mutate tags and a output if to asign index "reporting_indra" to tag "reporting_indra".

So. I can see the logs in Kibana.

But, if I search a word..

If I filter by value I get this:

So, if I replace the characters for another "word" works, but always with that symbols in beetwen.

I want to search directly with a normal word. Maybe I need to filter again?

Thanks so much.

[ChrsMark (Chris Mark)]

Hi! Not sure what these symbols are but Library seems to not be a single term since it's in the same string with other terms: Library!WindowService.... Maybe this is something you need to solve by splitting these terms?

C.

[enrique.villar (Enrique Lorenzo Villar Mavila)]

Thank you, Chris.

That's right. ´!´ is a separator.

I will indagate how that logs are formated, but I prefer don't touch the origin since is a regular txt file.

[ChrsMark (Chris Mark)]

Hey! You can maybe ship the logs to Logstash and manually split them there using grok patterns so as to avoid changing your source.

C.

[enrique.villar (Enrique Lorenzo Villar Mavila)]

This is my logstash conf:

input {
  beats {
    port => 5044
  }
}

filter {
    if "beats_input_codec_plain_applied" in [tags] {
        mutate {
            remove_tag => ["beats_input_codec_plain_applied"]
        }
    }
    if "rs_indra" in [tags] {
        grok {
           match => { "message" => "%{WORD:evento}!%{WORD:origen}!%{WORD:codigo}!%{DATE_US:fecha}-%{TIME:hora}:: %{GREEDYDATA:mensaje}" }
        }
    }
}
output {
        if "nginx-proxy-reverso" in [tags] {
                elasticsearch {
                        hosts => "localhost:9200"
                        manage_template => false
                        index => "nginx-proxy-reverso2-"
                        document_type => "%{[@metadata][type]}"
                }
        }
        else if "rs_indra" in [tags] {
               elasticsearch {
                        hosts => "localhost:9200"
                        manage_template => false
                        index => "rs_indra"
                        document_type => "%{[@metadata][type]}"
                }
        }
        else {
                elasticsearch {
                        hosts => "localhost:9200"
                        manage_template => false
#                        index => "nginx-proxy-reverso-%{[@metadata][beat]}-%{+Y                                                                                                                                                             YYY.MM.dd}"
                        index => "file-beat2-"
#                        setup.template.name: "nginx-proxy-reverso"
#                        setup.tamplate.pattern: "nginx-proxy-reverso-"
                        document_type => "%{[@metadata][type]}"
                }
        }
}

but I get this:

[ChrsMark (Chris Mark)]

If the grok pattern is able to successfully parse the messages then I expect that evento field should have the value of library.

Do you see any errors regarding the parsing?