Multiline is not working when i am sending logs to logstash through filebeat?


(Yaswanth ) #1

Hi,

Logs:

[2017-05-25 01:00:00,647][ERROR][marvel.agent.exporter.local] local exporter [default_local] -  ...
[2017-05-18 00:00:06,339][DEBUG][action.admin.indices.create] [data-2] [data-may-2017,data-apr-..

When i am sending logs from filebeat to logstash through multiline codec in the filebeat configuration it is not working .

My filebeat config:

- input_type: log

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - F:\elasticsearch-2.4.0\elasticsearch-2.4.0\logs\elasticsearch_index_search_slowlog.log
    #- c:\programdata\elasticsearch\logs\*
 
  multiline.pattern: ^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]
  multiline.negate: true
  multiline.match: "after"

output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

logging:
  to_files: true
  files:
    path: C:\Program Files\filebeat\logs
  level: debug
  selectors: ["*"]

Output: My both logs are merged together and coming

Logstash: When i used the same multiline in logstash it is working fine . I can see my logs are separated.

Whether the problem is in my filebeat configuration?

Thanks


(Christian Dahlqvist) #2

You seem to be using grok syntax in the Filebeat config, which I suspect is why it is not working. Have a look at the example in the documentation to see what it should look like. Change this and remove the multiline codec from the beats input and see if that helps.


(Yaswanth ) #3

Thanks..

But when i used below pattern it is working fine in logstash.[quote="Yaswanth, post:1, topic:88170"]
^[%{TIMESTAMP_ISO8601:TIMESTAMP}]
[/quote]

Did it differ that much in Filebeat?

Thanks


(Christian Dahlqvist) #4

As far as I know Filebeat does not support grok syntax, so patterns will need to be different as outlined in the documentation.


(Yaswanth ) #5

Ohh..Thanks @Christian_Dahlqvist


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.