File beat multiline is not working for XML type of files

Elastic Stack Beats
1

Hi,

I am using logstash 2.2.1 ,logstash-input beats-2.1.3 and filebeat 1.1.2 . I have some XML type of log files. I have written beats config. I am facing the following issue

Issue 1: The xml log file multiline events are not getting combined into single event as expected.

Below is my beats yml file configuration.

> filebeat:
>   prospectors:
>     -
>       paths:
>          - /logs/mylogs/2015*/*.xml
>       document_type: server_log
>       registry_file: /myarea/config/mylogs/.filebeat      
>   multiline:
    pattern: "^<error"
>     negate: true
>     match: after
> output: 
>   logstash:     
>      hosts: ["localhost:11689"]
> console:
>     pretty: true

My sample XML log will be in below format

<error id="1qas79" host="hhy789">
<snapshot>
<variable name="a">
  <item string="sss"> </item>
</variable>
</snapshot>
</error>
2

For regexes better use single quotes: pattern: '^<error'.

See this regex sample code and press run. Every line beginning with false should indicate a new multiline-event

Content is 2 xml events as mentioned in variable content, and regular expression in pattern. Try by replacing content with a few events/lines from your original logs and see if pattern works ok.

Is there a chance of having whitespace before <error>?

1 Like
3

Thanks for your guidance. Let me try your suggestion and let you know the result.

There is no whitespace before my tag <error>

4

Hi Steffen, I have tried the pattern: '^<error'. and I also include one more proerty in my beats YML file. input_type: xml. After this my issue got resolved.

Thank you steffen

5

@rajkamalkool6 input_type: xml is not a valid input type. Not sure how that helped?

6

you sure logs are send correctly? There is no input_type: xml. Valid values for input_type are log and stdin only.

7

No after removing input_type: xml its working, by default its taking as log only.

8

input_type: xml doesn't exists. input_type is kind of the 'plugin'-type to use. As xml does not exist it will fall back for log.

9

Yes you are right