How to push two inputs data as a single event(docment) in logstash.ym


(Kabali12345) #1

Hi...all
How can i push two inputs data as a single event(docment) in logstash.yml
for example

input{
    file{
        type => "dummylog"
        path => [/var/log/messages]
    }
  file{
        type => "log"
        path => [/var/log/kernel.logs]
    }

}

output {
   elasticsearch { host => localhost } stdout { } }
}

let /var/log/messages has 100 events and
let /var/log/kernel.logs has 100 events
and how can i push /var/log/messages first line and /var/log/kernel.logs first line as single event(document) into elasticsearch
will that possible.....to do?
thank you


Very slow movement of data
(Jimmy Kuang) #2

In the output you can specify two if statements and point them to the same index oneEvent.

For example:


input {
    file {
            type => "technical"
            path => "/home/technical/log"
    }
    file {
            type => "business"
            path => "/home/business/log"
    }
} 
output {
    if [type] == "technical" {
            # output to elasticsearch index oneEvent
    }
    if [type] == "business" {
            # output to elasticsearch index oneEvent
    }

#3

I used the same way but it didn't worked out. input is not accepting 2 file parameters.


(Jimmy Kuang) #4

This works for me. See my new example below:

input {

 file {
sincedb_path => "/tmp/sincedb"
    path => "/tmp/tech.log"
    type => "technical"  # a type to identify those logs (will need this later)
    start_position => "beginning"
}

    file {
 sincedb_path => "/tmp/sincedb2"
            type => "business"
            path => "/tmp/bus.log"
start_position => "beginning"
    }
}

output {

            if [type] == "technical" {
            # output to elasticsearch index oneEvent
        elasticsearch {
                index => "hitwo"
                document_type =>"tech"
        }
    }
    if [type] == "business" {
            # output to elasticsearch index oneEvent
   elasticsearch {
                index => "hitwo"
                document_type =>"bus"
    }
}
}

(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.