How to add tags and add filter?

Elastic Stack Logstash
1

Hi Guys,

I have below kind of information and looking assistance from community for creating logstash filter and add tag like "malware"

So that I am planning to start netflow on my devices and index the data and filter the data basis on tags "malware"

Can someone please tell me how do I put up logstatsh.

1.30.21.82 => IPADDR
1.30.57.188 => IPADDR
1.30.210.103 => IPADDR
1.30.218.38 => IPADDR
1.30.218.39 => IPADDR
1.30.218.40 => IPADDR
1.30.218.42 => IPADDR
xyz.com => DOMAIN
abc.com=> DOMAIN
kyz.net=> DOMAIN
fsfwessrsrssrsrrssrs324f = MD5
w2sgw4r45rh6ey56t564 =MD5

2

It's not clear what you want to accomplish. Please show an example event before and after the kind of filter you're looking for.

3

Hi there,

Her is the workflow -

  1. Well I have simple file which has a records like specified above. Rather has IP addresses, domains, URLs and MD5 hashes.
  2. I want to parse it in a Customized index like malicious-*
  3. Then create dashboard on IP addresses GEO hashes and of course multiple dashboards.
4

Hi Blason,

This is a pretty simple use case. I encourage you to read the documentation for ingesting documents and visualizing using ELK.

The workflow would be as follows:

Logstash => Elasticsearch => Kibana

Here is a sample conf file of logstash for reference:

input {
  file {
        path => "/opt/malware/*.csv"
        start_position => "beginning"
        sincedb_path => "/dev/null"
        tags => ["malware"]
  }
}

filter {
if "malware" in [tags] {
    csv {
      separator => ","
      columns => ["alertType","alertid","product","release","fileHash","dvchost","sname","dvc","locations","malware_type","timeoccured"]
    }
    date {
        locale => "en"
        match => ["timeoccurred", "YYYY-MM-dd'T'HH:mm:ss'Z'"]
        timezone => "Etc/GMT"
        target => "occurred"
        add_field => { "debug" => "timestampMatched"}
    }
   mutate {
     add_field => {
        "[@metadata][document_id]" => "%{alertid}"
     }
   }
}
}

output {
if "malware" in [tags] {
    stdout {
            codec => "rubydebug"
    }
    elasticsearch {
    action => "index"
    hosts => ["http://localhost:9200"]
    document_id => "%{[@metadata][document_id]}"
    index => "malware"
    }
}
}

To create geo location from ip, please refer this blog:

GeoIP in the Elastic Stack - Elasticsearch, Logstash, Ingest API | Elastic Blog

Once you have achieved this, creating visualizations is straight forward.

2 Likes
5

And I do need to have template for this right?I am sorry since I am pretty novice asking such a basic questions. even for geo tagging I guess I need to have template, correct me if I am wrong?

6

IP numbers and geopoints won't be autodetected by ES so in those cases you need to set the desired mapping explicitly, preferably via an index template.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.