# Mapping Netflow Data (NSEL)

**URL:** <https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316>\
**Category:** Elasticsearch\
**Created:** [August 16, 2017, 8:54pm UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316 "2017-08-16T20:54:55Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![cdomansky](https://avatars.discourse-cdn.com/v4/letter/c/e9bcb4/32.png) [@cdomansky](https://discuss.elastic.co/u/cdomansky)\
**Post date:** [August 16, 2017, 8:54pm UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/1 "2017-08-16T20:54:55Z")

</div>

Am new to the Elastic Stack and my 1st project with it is to analyze Netflow data from a Cisco ASA which actually pushes out NSEL as opposed to regular Netflow [v5,9] packets.

The problem that I am running into is that the index pulled into Kibana doesn't seem classify the 'type' within each property correctly.

For example, flow\_seq\_number, I set the type as a 'long', yet Kibana is showing the type as a 'number'  
Each IP address property is loading into Kibana as type 'string'

I included the mappings below and would appreciate any thoughts on how to fix this.

Regards.

curl -XPUT localhost:9200/\_template/logstash\_netflow-9995 -d '{  
"template" : "logstash\_netflow-9995\*",  
"settings": {  
"index.refresh\_interval": "5s"  
},  
"mappings" : {  
"_default_" : {  
"\_all" : {"enabled" : false},

```
    "properties" : {
      "@version": { "index": "analyzed", "type": "integer" },
      "@timestamp": { "index": "analyzed", "type": "date" },
      "host": {"index": "analyzed", "type": "ip"},
      
      "netflow": {
        "dynamic": true,
        "type": "object",

        "properties": {
        
          "version": { 
            "index": "not_analyzed", 
            "type": "integer" 
          },

          "flow_seq_num": {
            "index": "not_analyzed",
            "type": "long"
          },

          "flowset_id": {
            "index": "not_analyzed",
            "type": "long"
          },

          "nf_f_conn_id": {
            "index": "not_analyzed",
            "type": "long"
          },

          "nf_f_src_addr_ipv4": {
            "index": "analyzed",
            "type": "ip"
          },

          "nf_f_src_port": {
            "index": "analyzed",
            "type": "long"
          },

          "nf_f_src_intd_id": {
            "index": "not_analyzed",
            "type": "long"
          },

          "nf_f_dst_addr_ipv4": {
            "index": "analyzed",
            "type": "ip"
          },

          "nf_f_dst_port": {
            "index": "analyzed",
            "type": "long"
          },

          "nf_f_dst_intf_id": {
            "index": "not_analyzed",
            "type": "long"
          },              

          "nf_f_protocol": {
            "index": "not_analyzed",
            "type": "integer"
          },

          "nf_f_ingress_acl_id": {
            "index": "not_analyzed",
            "type": "string"
          },

          "nf_f_egress_acl_id": {
            "type": "string",
            "index": "not_analyzed"
          },

          "nf_f_fwd_flow_delta_bytes": {
            "type": "long",
            "index": "analyzed"
          },

          "nf_f_rev_flow_delta_bytes": {
            "type": "long",
            "index": "analyzed"
          },

          "nf_f_icmp_code": {
            "type": "long",
            "index": "analyzed"
          },

          "nf_f_icmp_type": {
            "type": "long",
            "index": "analyzed"
          },

          "nf_f_event_time_msec": {
            "type": "string",
            "index": "analyzed"
          },

          "nf_f_flow_create_time_msec": {
            "type": "date",
            "index": "analyzed"
          },

          "nf_f_xlate_dst_addr_ipv4": {
            "type": "ip",
            "index": "analyzed"
          },

          "nf_f_xlate_dst_port": {
            "type": "long",
            "index": "analyzed"
          },

          "nf_f_xlate_src_addr_ipv4": {
            "type": "ip",
            "index": "analyzed"
          },

          "nf_f_xlate_src_port": {
            "type": "long",
            "index": "analyzed"
          }              
        }
      }
    }
  }
}

```

}'

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 16, 2017, 9:39pm UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/2 "2017-08-16T21:39:55Z")

</div>

What version are you on, what does the mapping that is applied look like?

---

<div class="post-metadata">

**Author:** ![cdomansky](https://avatars.discourse-cdn.com/v4/letter/c/e9bcb4/32.png) [@cdomansky](https://discuss.elastic.co/u/cdomansky)\
**Post date:** [August 17, 2017, 2:19pm UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/3 "2017-08-17T14:19:57Z")

</div>

Running 5.5.1

And here is what the mapping looks like once it is applied.

curl -XGET 'localhost:9200/logstash\_netflow-2017.08.17/\_mapping/?pretty'  
{  
"logstash\_netflow-2017.08.17" : {  
"mappings" : {  
"logs" : {  
"properties" : {  
"@timestamp" : {  
"type" : "date"  
},  
"@version" : {  
"type" : "text",  
"fields" : {  
"keyword" : {  
"type" : "keyword",  
"ignore\_above" : 256  
}  
}  
},  
"host" : {  
"type" : "text",  
"fields" : {  
"keyword" : {  
"type" : "keyword",  
"ignore\_above" : 256  
}  
}  
},  
"netflow" : {  
"properties" : {  
"flow\_seq\_num" : {  
"type" : "long"  
},  
"flowset\_id" : {  
"type" : "long"  
},  
"fw\_event" : {  
"type" : "long"  
},  
"in\_permanent\_bytes" : {  
"type" : "long"  
},  
"nf\_f\_conn\_id" : {  
"type" : "long"  
},  
"nf\_f\_csrc\_intf\_id" : {  
"type" : "long"  
},  
"nf\_f\_dst\_addr\_ipv4" : {  
"type" : "text",  
"fields" : {  
"keyword" : {  
"type" : "keyword",  
"ignore\_above" : 256  
}  
}  
},  
"nf\_f\_dst\_intf\_id" : {  
"type" : "long"  
},  
"nf\_f\_dst\_port" : {  
"type" : "long"  
},  
"nf\_f\_egress\_acl\_id" : {  
"type" : "text",  
"fields" : {  
"keyword" : {  
"type" : "keyword",  
"ignore\_above" : 256  
}  
}  
},  
"nf\_f\_event\_time\_msec" : {  
"type" : "long"  
},  
"nf\_f\_flow\_create\_time\_msec" : {  
"type" : "long"  
},  
"nf\_f\_fw\_ext\_event" : {  
"type" : "long"  
},  
"nf\_f\_icmp\_code" : {  
"type" : "long"  
},  
"nf\_f\_icmp\_type" : {  
"type" : "long"  
},  
"nf\_f\_ingress\_acl\_id" : {  
"type" : "text",  
"fields" : {  
"keyword" : {  
"type" : "keyword",  
"ignore\_above" : 256  
}  
}  
},  
"nf\_f\_protocol" : {  
"type" : "long"  
},  
"nf\_f\_src\_addr\_ipv4" : {  
"type" : "text",  
"fields" : {  
"keyword" : {  
"type" : "keyword",  
"ignore\_above" : 256  
}  
}  
},  
"nf\_f\_src\_port" : {  
"type" : "long"  
},  
"nf\_f\_username" : {  
"type" : "text",  
"fields" : {  
"keyword" : {  
"type" : "keyword",  
"ignore\_above" : 256  
}  
}  
},  
"nf\_f\_xlate\_dst\_addr\_ipv4" : {  
"type" : "text",  
"fields" : {  
"keyword" : {  
"type" : "keyword",  
"ignore\_above" : 256  
}  
}  
},  
"nf\_f\_xlate\_dst\_port" : {  
"type" : "long"  
},  
"nf\_f\_xlate\_src\_addr\_ipv4" : {  
"type" : "text",  
"fields" : {  
"keyword" : {  
"type" : "keyword",  
"ignore\_above" : 256  
}  
}  
},  
"nf\_f\_xlate\_src\_port" : {  
"type" : "long"  
},  
"version" : {  
"type" : "long"  
}  
}  
}  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 18, 2017, 2:20am UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/4 "2017-08-18T02:20:03Z")

</div>

I can't see `flow_seq_number` in either mapping?

---

<div class="post-metadata">

**Author:** ![cdomansky](https://avatars.discourse-cdn.com/v4/letter/c/e9bcb4/32.png) [@cdomansky](https://discuss.elastic.co/u/cdomansky)\
**Post date:** [August 21, 2017, 1:27pm UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/5 "2017-08-21T13:27:13Z")

</div>

Sorry Mark, I do not understand what it was that you were asking.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 21, 2017, 11:20pm UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/6 "2017-08-21T23:20:37Z")

</div>

The `flow_seq_number` you mention in the first post is not visible in any of the mappings you have posted.

---

<div class="post-metadata">

**Author:** ![cdomansky](https://avatars.discourse-cdn.com/v4/letter/c/e9bcb4/32.png) [@cdomansky](https://discuss.elastic.co/u/cdomansky)\
**Post date:** [August 22, 2017, 1:43pm UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/7 "2017-08-22T13:43:15Z")

</div>

Unless I am looking at it wrong, after retrieving the mapping, 'flow\_seq\_num' shows to be one of the 1st properties of listed under netflow.

“netflow” : {  
“properties” : {  
“flow\_seq\_num” : {  
“type” : “long”  
},

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 26, 2017, 4:49am UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/8 "2017-08-26T04:49:15Z")

</div>

Oh, see I was searching for `flow_seq_number`, when it's `flow_seq_num`. Looks like we had wires crossed! 🙂

Kibana does a bit of hand waving here and depicts any long/float/numeric as a number, so as long as the mapping is correct then you have nothing to be worried about. The docs should call this out, so if you really want to create an issue then please do, otherwise I will 🙂

The IP should show as one though, but looking at (eg) the `nf_f_src_addr_ipv4` field, it's showing as a text field. Are you able to edit your posts and wrap the json on code tags, makes it easier to read and debug 🙂

---

<div class="post-metadata">

**Author:** ![cdomansky](https://avatars.discourse-cdn.com/v4/letter/c/e9bcb4/32.png) [@cdomansky](https://discuss.elastic.co/u/cdomansky)\
**Post date:** [August 29, 2017, 6:46pm UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/9 "2017-08-29T18:46:58Z")

</div>

Hoping this turns out and it is a little easier to read.

```
"logstash_netflow-2017.08.29" : {
 "mappings" : {
  "netflow" : {
    "properties" : {
      "@timestamp" : {
        "type" : "date"
      },
      "@version" : {
        "type" : "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
          }
        }
      },
      "host" : {
        "type" : "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
          }
        }
      },
      "netflow" : {
        "properties" : {
          "flow_seq_num" : {
            "type" : "long"
          },
          "flowset_id" : {
            "type" : "long"
          },
          "fw_event" : {
            "type" : "long"
          },
          "in_permanent_bytes" : {
            "type" : "long"
          },
          "nf_f_conn_id" : {
            "type" : "long"
          },
          "nf_f_csrc_intf_id" : {
            "type" : "long"
          },
          "nf_f_dst_addr_ipv4" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "nf_f_dst_intf_id" : {
            "type" : "long"
          },
          "nf_f_dst_port" : {
            "type" : "long"
          },
          "nf_f_egress_acl_id" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "nf_f_event_time_msec" : {
            "type" : "long"
          },
          "nf_f_flow_create_time_msec" : {
            "type" : "long"
          },
          "nf_f_fw_ext_event" : {
            "type" : "long"
          },
          "nf_f_icmp_code" : {
            "type" : "long"
          },
          "nf_f_icmp_type" : {
            "type" : "long"
          },
          "nf_f_ingress_acl_id" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "nf_f_protocol" : {
            "type" : "long"
          },
          "nf_f_src_addr_ipv4" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "nf_f_src_port" : {
            "type" : "long"
          },
          "nf_f_username" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "nf_f_xlate_dst_addr_ipv4" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "nf_f_xlate_dst_port" : {
            "type" : "long"
          },
          "nf_f_xlate_src_addr_ipv4" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "nf_f_xlate_src_port" : {
            "type" : "long"
          },
          "version" : {
            "type" : "long"
          }
        }
      },
      "type" : {
        "type" : "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
          }
        }
      }
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 30, 2017, 10:38am UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/10 "2017-08-30T10:38:28Z")

</div>

It is, thanks!

Looking at that one then you definitely have the IP fields mapped as text+keyword. Can you show the template for it, also formatted?

---

<div class="post-metadata">

**Author:** ![cdomansky](https://avatars.discourse-cdn.com/v4/letter/c/e9bcb4/32.png) [@cdomansky](https://discuss.elastic.co/u/cdomansky)\
**Post date:** [August 30, 2017, 2:14pm UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/11 "2017-08-30T14:14:20Z")

</div>

Here is the template that was used

curl -XPUT localhost:9200/\_template/logstash\_netflow- -d

```
'{
  "template" : "logstash_netflow-",
  "settings": {
  "index.refresh_interval": "5s",
  "number_of_shards": "1",
  "number_of_replicas": "0"
},
"mappings" : {
  "_default_" : {
    "_all" : {"enabled" : true},

    "properties" : {
      "@version": { "index": "analyzed", "type": "integer" },
      "@timestamp": { "index": "analyzed", "type": "date" },
      "host": {"index": "analyzed", "type": "ip"},
      
      "netflow": {
        "dynamic": false,
        "type": "object",

        "properties": {
        
          "version": { 
            "index": "not_analyzed", 
            "type": "integer" 
          },

          "flow_seq_num": {
            "index": "not_analyzed",
            "type": "long"
          },

          "flowset_id": {
            "index": "not_analyzed",
            "type": "long"
          },

          "nf_f_conn_id": {
            "index": "not_analyzed",
            "type": "long"
          },

          "nf_f_src_addr_ipv4": {
            "index": "analyzed",
            "type": "ip"
          },

          "nf_f_src_port": {
            "index": "analyzed",
            "type": "long"
          },

          "nf_f_src_intd_id": {
            "index": "not_analyzed",
            "type": "long"
          },

          "nf_f_dst_addr_ipv4": {
            "index": "analyzed",
            "type": "ip"
          },

          "nf_f_dst_port": {
            "index": "analyzed",
            "type": "long"
          },

          "nf_f_dst_intf_id": {
            "index": "not_analyzed",
            "type": "long"
          },              

          "nf_f_protocol": {
            "index": "not_analyzed",
            "type": "integer"
          },

          "nf_f_ingress_acl_id": {
            "index": "not_analyzed",
            "type": "string"
          },

          "nf_f_egress_acl_id": {
            "type": "string",
            "index": "not_analyzed"
          },

          "nf_f_fwd_flow_delta_bytes": {
            "type": "long",
            "index": "analyzed"
          },

          "nf_f_rev_flow_delta_bytes": {
            "type": "long",
            "index": "analyzed"
          },

          "nf_f_icmp_code": {
            "type": "long",
            "index": "analyzed"
          },

          "nf_f_icmp_type": {
            "type": "long",
            "index": "analyzed"
          },

          "nf_f_event_time_msec": {
            "type": "string",
            "index": "analyzed"
          },

          "nf_f_flow_create_time_msec": {
            "type": "date",
            "index": "analyzed"
          },

          "nf_f_xlate_dst_addr_ipv4": {
            "type": "ip",
            "index": "analyzed"
          },

          "nf_f_xlate_dst_port": {
            "type": "long",
            "index": "analyzed"
          },

          "nf_f_xlate_src_addr_ipv4": {
            "type": "ip",
            "index": "analyzed"
          },

          "nf_f_xlate_src_port": {
            "type": "long",
            "index": "analyzed"
          }              
        }
      }
    }
  }
}
}'
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 27, 2017, 2:14pm UTC](https://discuss.elastic.co/t/mapping-netflow-data-nsel/97316/12 "2017-09-27T14:14:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.