Multiple Log type

Elastic Stack Beats
1

Hi Team,

Hope you all are doing great.

I have a question regarding the different log type in filebeat. I have configured two prospectors and used different log type but at my kibana dashboard it shows only doc, while i have not set any log type as doc.

here is my filebeat configuartion.

  • type: log
    enabled: true
    paths:

    • /home/tomcat/builds/FCSKY/logs/application.log

    fields:
    #document_type: App_log
    type: App_log

tried both of them but none worked.

multiline.pattern: '^(([0-9]{4}-[0-9]{2}-[0-9]{2})|([a-zA-z]{3} [0-9]{2}, [0-9]{4} [0-9]{2}:[0-9]{2}:[0-9]{2} [AM|PM])|([0-9]{2}-[a-zA-z]{3}-[0-9]{4})|([a-zA-z]{3} [a-zA-z]{3} [0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}))'
multiline.negate: true
multiline.match: after

  • type: log
    enabled: true
    paths:

    • /home/tomcat/builds/FCSKY/logs/db.log

    fields:
    #document_type: db_log
    type: db_log

tried both of them but none worked.

multiline.pattern: '^(([0-9]{4}-[0-9]{2}-[0-9]{2})|([a-zA-z]{3} [0-9]{2}, [0-9]{4} [0-9]{2}:[0-9]{2}:[0-9]{2} [AM|PM])|([0-9]{2}-[a-zA-z]{3}-[0-9]{4})|([a-zA-z]{3} [a-zA-z]{3} [0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}))'
multiline.negate: true
multiline.match: after

In my logstash config i'm using this pattern.

if [type] == "db_log" {
grok {
match => [ "message", "%{TIMESTAMP_ISO8601:timetamp %{NOTSPACE} %{INT:line} %{NOTSPACE} %{LOGLEVEL:loglevel} (?:- Tenant Name :slight_smile: %{WORD:TENANT_NAME} (?:GlobalConnectionPool - Active Connections :)%{WORD:Active_Connections} %{NOTSPACE} (?:Idle Connections :slight_smile: %{WORD:Idle_Connection}" ]
}

Please help.
Thanks

2

Can you post some sample data? Are you referring to _type or type. _type should be doc, but if you have a custom type field it should not be.

3

It worked by adding
fields_under_root: true
fields:
type: db_log

Thanks man.

4

NP, glad to hear it!

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.