Unable to parse multiple format logs

here is my full configuration:

input {
file {
type => "apache"
path => "C:/Users/Mahe/Desktop/intership/apache.log"
}
file {
type => "csvf"
path => "C:/data/cars.csv"
}
}
filter {
if [type] == "apache" {
grok {
match => [
"message" , "%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra_fields}",
"message" , "%{COMMONAPACHELOG}+%{GREEDYDATA:extra_fields}"
]
overwrite => [ "message" ]
}
mutate {
convert => ["response", "integer"]
convert => ["bytes", "integer"]
convert => ["responsetime", "float"]
}
geoip {
source => "clientip"
target => "geoip"
add_tag => [ "apache-geoip" ]
}
date {
match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
remove_field => [ "timestamp" ]
}
useragent {
source => "agent"
}
}
if [type] == "csvf" {
csv{
separator => ","
columns=>["maker","model","mileage","manufacture_year","engine_displacement","engine_power","body_type","color_slug","stk_year","transmission","door_count","seat_count","fuel_type","date_created","date_last_seen","price_eur"]
}
mutate{convert=>["mileage","integer"]}
mutate{convert=>["price_eur","float"]}
mutate{convert=>["engine_power","integer"]}
mutate{convert=>["door_count","integer"]}
mutate{convert=>["seat_count","integer"]}
}
}
output {
if [type] == "apache" {
elasticsearch {
hosts => "localhost"
index=>"apa"
document_type=>"apacsvd"
}
stdout{}
}
if [type] == "csvf" {
elasticsearch {
hosts => "localhost"
index=>"csvf"
document_type=>"apacsvd"
}
stdout{}

}

}
however when i try to parse them individually it works just fine

Okay, but in what way is it not working now? What is the symptom?

the pipeline gets started but logs are not parsed.

How are you starting Logstash when it's not working? How are you starting Logstash when it is working? Have you tried bumping up Logstash's loglevel and looked in its log for clues?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.