Netflow v9 not being recieved over UDP

Elastic Stack Logstash
1

Having a very strange problem I'm hoping the logstash/Elastic community will have some guidance on. I'm trying to get logstash 5.6.1 to process Netflow v9 records, then send to a remote Elasticsearch instance, also v5.6.1. I can see UDP traffic arriving on port 2055, and I think I've got everything configured right, but not matter what I do, I never see any traffic emitted from the host running logstash, going over to the host running elasticsearch.

logstash.yml: (most options are default, commented lines not shown)

path.config: /etc/logstash/conf.d
log.level: trace
path.logs: /var/log/logstash

conf.d/netflow.conf:

input {
        udp {
                port => 2055
                host => "10.10.22.50"
                codec => netflow {
                        versions => [5, 9]
                        ipfix_definitions => "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-netflow-3.5.2/lib/logstash/codecs/netflow/ipfix.yaml"
#                       netflow_definitions => "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-netflow-3.5.2/lib/logstash/codecs/netflow/netflow.yaml"
                }
        }
}

output {
        stdout {
                codec => "rubydebug"
        }
        file {
                path => "/tmp/netflow.txt"
        }
        elasticsearch {
                index => "logstash-netflow-%{+YYYY.MM.dd}"
                hosts => ["10.10.22.51:9200"]
        }
}

Starting logstash with systemctl start logstash, I see the pipeline start up:

[2017-09-20T15:51:21,051][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//10.10.22.51:9200"]}
[2017-09-20T15:51:21,053][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-09-20T15:51:21,060][INFO ][logstash.pipeline        ] Pipeline main started
[2017-09-20T15:51:21,063][INFO ][logstash.inputs.udp      ] Starting UDP listener {:address=>"10.10.22.50:2055"}

However the input plugin never gets any hits, from the output of curl -XGET 'localhost:9600/_node/stats?pretty=true':

"plugins" : {
  "inputs" : [ {
    "id" : "e38c973d176d7df29ff9fef5c88d29701fd1ad97-2",
    "events" : {
      "out" : 0,
      "queue_push_duration_in_millis" : 0
    },
    "name" : "udp"
  } ],

There are no iptables rules and firewalld is not running. Any ideas?

2

Found your topic after opening mine. I have the same issue after updating from 5.4 to 5.6.

Lets hope somebody can help...

3

Good news, my netflow started working after updating to logstash-5.6.2-1. I'm running this on CentOS 7, using the RPMs provided in the Elastic repo at https://artifacts.elastic.co/packages/5.x/yum

As I made no changes my config, I'm almost certain it was the RPM update that did the trick.

4

Tried updating to 6.5.2 but no luck. Still nothing coming on.

5

Did you try with logstash plugin? Mine is working properly with logstatsh plugin from cli; I was facing same issue but when I started from cli it started showing all the records.

6

Nope, I just updated the RPM. Doing a rpm -ql logstash | grep netflow, I can see 182 files in that RPM are related to netflow. As far as I can tell, the netflow plugin is bundled in the RPM, and it was updated to, hence why it started working.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.