Filter help please!

Elastic Stack Logstash
1

I am banging my head on something and I am hoping that someone can help me out. I am also open to better ways to do what I am doing if anyone has any ideas. Currently I get logs from multiple systems that I am trying to store in different indexes. I currently want to use 4 different indexes, coresystem, auxsystem, test_sys, unknown. I was using indexes, coresystem, unknown fine, however when I added 3 things broke and I am just getting unknown for the new one I added. After adding test_sys the data did not start to go to the new index, it is still going to the unknown index.

filter {
if [host] in ["10.10.252.99", "10.145.10.133", "10.145.25.7"] {
mutate {
add_tag => ["coresystem"]
add_field => {
"system_type" => "coresystem"
}
}
}
}

filter {
if [host] == ["10.15.28.185"] {
mutate {
add_tag => ["auxsystem"]
add_field => {
"system_type" => "auxsystem"
}
}
}

}

filter {
if "test_sys" in [message] {
mutate {
add_tag => ["test_sys"]
add_field => {
"system_type" => "test"
}
replace => { "host" => "TestSys" }
}
}
}

filter {
if [tags] == {
mutate {
add_tag => ["unknown"]
add_field => {
"system_type" => "unknown"
}
}
}
}

output {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "%{system_type}-%{+YYYY.MM.dd}"
}
}

Thanks for the help!

2

I will have done similier thing. I will put everything in to one filter.
filter {
if { }
else if { }
else if { }
else { }
}# end of filter section

3

I was using else if with the same result, I can try again though.

4

Something like this

filter {
if [host] in ["10.10.252.99", "10.145.10.133", "10.145.25.7"] {
mutate {
add_tag => ["coresystem"]
add_field => {
"system_type" => "coresystem"
}
}
}
if [host] == ["10.15.28.185"] {
mutate {
add_tag => ["auxsystem"]
add_field => {
"system_type" => "auxsystem"
}
}
}
if "test_sys" in [message] {
mutate {
add_tag => ["test_sys"]
add_field => {
"system_type" => "test"
}
replace => { "host" => "TestSys" }
}
}

if [tags] ==  {
     mutate {
         add_tag => ["unknown"]
          add_field => {
                 "system_type" => "unknown"
           }
      }
}

}

5 
### job_status will be 0,1,2,3,4,5 lets put some meaning to it
   if [job_status] == "0" {
      mutate {
         add_field => { "status" => "queued" }
      } #Mutate
   }else if [job_status] == "1" {
      mutate {
         add_field => { "status" => "active" }
      } #Mutate
   } else if [job_status] == "2" {
      mutate {
         add_field => { "status" => "waiting_resource" }
      } #Mutate
   } else if [job_status] == "3" {
      mutate {
        add_field => { "status" => "success" }
      } #Mutate
   } else if [job_status] == "4" {
        mutate {
           add_field => { "status" => "suspended" }
        } #Mutate
     } else if [job_status] == "5" {
          mutate {
             add_field => { "status" => "incomplete" }
          } #Mutate
       }
   else {
      mutate {
          add_field => { "status" => "unknown" }
      } #Mutate
   }

Here is example that I am using and it is working fine.

6

I have changed them back to else if statements and I am waiting for some data to hit the logs now.

7

So I went back and it is working now, now sure why...Must have had a strange typo, thanks!

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.