How to raw logs to logstash default index and filtered log to another index


(Shamim Akhtar) #1

Hi All,
How can i configure logstash which can send all logs to default logstash-xxxx index and filtered log { i want only specific fields} will go to analytics index.

But I also want to have all raw logs in logstash index. Kindly help me.

Thanks
Shamim Akhtar

input {

This will allow for health check from Marathon

http {
port => 31210
type => "elb-healthcheck"
}

Default port is 12201/udp

gelf { port => 31212 }

}

filter {

some filter

}

output {

// here i want filterd log
elasticsearch {
hosts => ["host:port"]
index => "business-%{+YYYY.MM.dd}"
document_type => "analytics"
}
// here i want all raw log
elasticsearch {
hosts => ["host:port"]
index => "logstash-%{+YYYY.MM.dd}"
document_type => "log"
}

}


(Mark Walkom) #2

You can do that using this method - https://www.elastic.co/guide/en/logstash/current/config-examples.html#using-conditionals


(Shamim Akhtar) #3

@warkolm :

Thanks for your reply.

if i do condition and output to different index. filtered log goes to business-index but raw log for login activity then does not go to default index.

what i want to do all login activity filtered log will go to business index and everything without filtered log will go to default logstash-xxx index

Thanks for your support.


(Magnus B├Ąck) #4
if ... {
  elasticsearch {
    index => "business"
    ...
  }
}
elasticsearch {
  ...
}

Replace the first ... with whatever condition that matches login activity.


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.