Help with Netflow Configuration

Elastic Stack Logstash
1

Hello,

First, I have this error : "[2017-03-21T11:36:33,543][WARN ][logstash.codecs.netflow ] No matching template for flow id 260" I read a lot of topics about that but I don't know how to solve this.

Then, I tried to use the convert function but it didn't worked.

My config:

input {
udp {
port => 2055
codec => netflow {
versions => [9]
netflow_definitions => "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-netflow-3.4.0/lib/logstash/codecs/netflow/netflow.yaml"
}
type => netflow
tags => ["netflow"]
}
}

filter {
mutate { convert => [ "netflow.protocol" , "string" ]
}
translate {
field => "[netflow][protocol]"
destination => "[netflow][protocol]"
override => "true"
dictionary => [ "6", "TCP",
"17", "UDP",
"1", "ICMP",
"47", "GRE",
"50", "ESP",
"58", "IPv6-ICMP"
]
}

    }

output {
if "netflow" in [tags] {
elasticsearch {
hosts => ["localhost:9200"]
index => "logstash_netflow9-%{+YYYY.MM.dd}"
template_name => "netflow"
}

stdout { codec => rubydebug }
}

}

Thanks for your help.

2

Normally, you get that error when you first start sending netflow data to a newly-initialised Logstash instance.

This is because Netflow v9 is template-based.

Normally you just need to wait a little while and then this error will stop once the template appears in the stream, allowing the fields to be correctly decoded. You can check the config of your router or firewall to see how frequently the template is sent.

But if you are waiting for a while and still nothing, post some more info e.g. what logstash version you have, what version of the netflow codec plugin...

3

Okok for the error, I did not understand that it was normal but yes after a while I had already all my data.

However, regarding the mutate function, I want to use it to convert some field to string and use the tranlate function to display the name of the protocol according to the port. But I have an error specifying that my fields are number despite the mutate fuction:

filter {
mutate { convert => [ "netflow.protocol" , "string" ]
}
translate {
field => "[netflow][protocol]"
destination => "[netflow][protocol]"
override => "true"
dictionary => [ "6", "TCP",
"17", "UDP",
"1", "ICMP",
"47", "GRE",
"50", "ESP",
"58", "IPv6-ICMP"
]
}
}

4

Instead of netflow.protocol try [netflow][protocol] in the mutate filter.

Rob

5

Same error:

"[2017-03-22T14:04:26,683][WARN ][logstash.outputs.elasticsearch] Failed action. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash_netflow9-2017.03.22", :_type=>"netflow", :_routing=>nil}, 2017-03-22T13:04:27.000Z IPADDRESS %{message}], :response=>{"index"=>{"_index"=>"logstash_netflow9-2017.03.22", "_type"=>"netflow", "_id"=>"REDACTED", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [netflow.protocol]", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string: "UDP""}}}}}"

Config:

filter {
mutate { convert => { "[netflow][protocol]" => "string" }
}
translate {
field => "[netflow][protocol]"
destination => "[netflow][protocol]"
override => "true"
dictionary => [ "6", "TCP",
"17", "UDP",
"1", "ICMP",
"47", "GRE",
"50", "ESP",
"58", "IPv6-ICMP"]
}
}

6

Same too with mutate { convert => [ "[netflow][protocol]", "string" ] }

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.