Unable to segregate messages from two Input files

[Blason (R)]

Hi Team,

I posted this message on stack but not getting any replies. Can someone please help? I need help in seggregrating messages from my two different conf files. I am bit confused about ingestion
Here is my first file getting input from

input {
        file {
        path => ["/var/log/ransomware.json"]
#       tags => "ransomware"
        start_position => "beginning"
        id => "ransomware"
        }
        }
filter {
        json {
                source => "message"
                remove_field => ["message"]
                       }
}

While other file

input {
  file {
    path => "/var/log/compromise.txt"
    start_position => "beginning"
    id => "compromise"
        }
  }

filter {
  grok {
    match => { "message" => "%{WORD:threatactor},%{WORD:country},\(%{WORD:country2}\),%{URI:url},%{IPV4:ip},%{GREEDYDATA:timestamp}" }
        remove_field => [ "message", "path", "@version", "host", "country2" ]
        }
    mutate {
        add_field => { "tag" => "deface_portals" }
        }
    date {
        match => [ "timestamp", "dd/MM/yyyy" ]
        target => "@timestamp"
        }
    geoip {
        source => "ip"
        }
        }

And here is the output mentioned in first conf file

output {
        if [deface_portals] {
  elasticsearch {
        hosts => ["https://10.122.0.11:9200"]
        ssl => true
        manage_template => true
        ssl_certificate_verification => false
        user => "xxxxx"
        password => "xxxxxx"
        cacert => "/etc/logstash/ca.crt"
        index => "compromise-%{+YYYY.MM.dd}"
                }
        } else {
  elasticsearch {
        hosts => ["https://10.122.0.11:9200"]
        ssl => true
        manage_template => true
        ssl_certificate_verification => false
        user => "xxxxxx"
        password => "xxxxxx"
        cacert => "/etc/logstash/ca.crt"
        index => "ransomware-%{+YYYY.MM.dd}"
                }
        }
        }

Even after that my messages from compromise hosts are getting ingested in ransomwatch not sure why

[Rios]

I guess you are using 2 input.conf, 2 filter.conf and output.conf.
Easiest way is to add type=>"ransomware"and type=>"compromise" in every input.conf

input {
        file {
        path => ["/var/log/ransomware.json"]
        start_position => "beginning"
        type => "ransomware"
        id => "ransomware"
        }
}
filter {
  if [type] == "ransomware"{
    json { 
      source => "message"
     remove_field => ["message"]
     }
 } 
else if [type] == "compromise"{ 
...
 }
}
output {
  if [type] == "ransomware"{
    elasticsearch {
        hosts => ["https://10.122.0.11:9200"]
        ssl => true
        manage_template => true
        ssl_certificate_verification => false
        user => "xxxxxx"
        password => "xxxxxx"
        cacert => "/etc/logstash/ca.crt"
        index => "ransomware-%{+YYYY.MM.dd}"
    }
  }
 else if [type] == "compromise"{ 
  if [type] == "ransomware"{ ...
  }
 }
}

You can make single ransomware.conf and compromise.conf with input, filter, output. There is few useful samples here.

[Blason (R)]

Let me try doing that

[leandrojmp (Leandro Pereira)]

As @Rios explained you need to have conditionals in both your filters and outputs.

But the best approach in this case is to configure logstash to use multiple pipelines, this way your two pipelines are entirely segregated from each other without the need of using conditionals.