Can't get Kibana to work for graphing Netflow data

Hi all

I'm a complete n00b at the ELK stack and I feel like I'm out of my depth.

Basically I want to be able to send Netflow data from my DD-WRT router to logstash and then off to elasticsearch and Kibana for some nice graphs.

I read through the Logstash and elasticsearch guides to try to get a grip on the basics and I managed to get data in from the router but for the life of me I can't figure out why Kibana can't graph anything.

Anyways, here's some configs.

This is what I have for my logstash config:

input {
	udp {
		port => 2055
		codec => netflow

output {
	elasticsearch {
		hosts => [ "localhost:9200" ]
		index => "ddwrt-%{+YYYY.MM.dd}"

When I output to stdout instead of elasticsearch then I see records like these coming in from the router:

      "@version" => "1",
          "host" => "",
       "netflow" => {
                    "dst_as" => 0,
                   "in_pkts" => 9,
            "first_switched" => "2017-09-16T19:03:50.999Z",
             "ipv4_next_hop" => "",
               "l4_src_port" => 52953,
        "sampling_algorithm" => 0,
                  "in_bytes" => 2283,
                  "protocol" => 17,
                 "tcp_flags" => 0,
               "l4_dst_port" => 1900,
                    "src_as" => 0,
               "output_snmp" => 0,
                  "dst_mask" => 0,
             "ipv4_dst_addr" => "",
                   "src_tos" => 0,
                  "src_mask" => 0,
                   "version" => 5,
              "flow_seq_num" => 412486,
              "flow_records" => 1,
             "ipv4_src_addr" => "",
               "engine_type" => 0,
                 "engine_id" => 0,
                "input_snmp" => 1,
             "last_switched" => "2017-09-16T19:03:50.999Z",
         "sampling_interval" => 0
    "@timestamp" => 2017-09-16T19:04:05.000Z

Seems correct so far?

For elasticsearch, when I navigate to http://localhost:9200/_cat/indices?v then I see this:

health status index                     uuid                   pri rep docs.count docs.deleted store.size
yellow open   ddwrt-2017.09.16          k1GAKtrRRVKtdhl-YPxxDw   5   1       1515            0    841.6kb        841.6kb
yellow open   .kibana                   k4GoPKJWSWyrjx5qWCZV9A   1   1          2            0      8.3kb          8.3kb

Again, seems correct and the index is there.

But when I tell Kibana to use the pattern ddwrt*, I can choose the @timestamp field for the time but I can't get any graphs or anything to appear.

Under the 'Discover' tab, no fields show up unless I uncheck "Hide Missing Fields". And then when I click a field, they all say this:

This field is present in your elasticsearch mapping but not in any documents in the search results. You may still be able to visualize or search on it.

What am I missing or doing wrong?

FYI we’ve renamed ELK to the Elastic Stack, otherwise Beats feels left out :wink:

What version are you running? Do you have the right timeframe in the picker, top right?

Hi Mark

Should've renamed it BELK then. :joy:

I'm running the latest version.

It was the timeframe issue. I just started sending Netflow data via the router and I could see logstash and elasticsearch getting the data, so I had a few minutes worth of data. But if I selected "Last 15 mins" or "Last hour" nothing showed up. However, selecting "Today" worked and all the documents showed up.

Why is that? Shouldn't the former 2 options just show the stuff that was logged within those time frames?

It's based on the @timestamp field.
I don't know how the netflow codec works in detail, but it may be doing something with timezone conversion.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.