# Netflow visualizations Kibana 4 - Limited fields?
**URL:** https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240
**Category:** Kibana
**Created:** [April 4, 2016, 12:57pm UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240 "2016-04-04T12:57:22Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)
#### Post date: [April 4, 2016, 12:57pm UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240/1 "2016-04-04T12:57:22Z")
Hi,
I'm running a test environment where I have ELK set up and receiving syslogs and netflow from a router. Everything is coming in well and I am trying to make some visualizations.
I feel like the fields I'm given on netflow are very limited and I can't even select them when creating visualizations. Am I missing something?
[ ](http://i.imgur.com/pZi07z6.png)
An example of a visualization that I'm trying to make is a simple pie chart with SRC ports or SRC ips like here [http://www.neteye-blog.com/wp-content/uploads/2016/02/Kibana4-dasboards.png](http://www.neteye-blog.com/wp-content/uploads/2016/02/Kibana4-dasboards.png).
Is it because my test environment is very limited and simply can't provide those fields for me to work with?
---
### Author:  [@Court](https://discuss.elastic.co/u/Court)
#### Post date: [April 4, 2016, 3:15pm UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240/2 "2016-04-04T15:15:38Z")
Are all of the fields you're trying to visualize nested under "netflow."? If so, can you flatten that data out when you index it in elasticsearch?
Also, Kibana automatically determines and caches the mappings for an index at index pattern creation time. If you think there are fields in the index that are not showing up at all in Kibana, hit the "refresh mappings" button on the index pattern in the settings tab.
---
### Author:  [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)
#### Post date: [April 5, 2016, 7:32am UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240/3 "2016-04-05T07:32:41Z")
Hi, appreciate your reply.
I used the command `curl 'localhost:9200/_cat/indices?v'` to check for all indexes on Elasticsearch and it returned me a bunch of them.
I noticed that there were two called `logstash-netflow9-2016.04.04` and the other `logstash-netflow9-2016.04.05`. These were probably made by the config file as I mentioned an index with date in the end at the output section. Is this correct?
You mentioned that I should flatten out the data, how can I do that?
My goal in the end would be having the fields `ipv4_src_addr, ipv4_dst_addr, l4_src_port, l4_dst_port` and more just like in [http://blogs.cisco.com/security/step-by-step-setup-of-elk-for-netflow-analytics](http://blogs.cisco.com/security/step-by-step-setup-of-elk-for-netflow-analytics) (As reference, I use a Juniper router and did not follow this)
---
### Author:  [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)
#### Post date: [April 5, 2016, 7:54am UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240/4 "2016-04-05T07:54:18Z")
Found out the following, when I use `curl -XGET localhost:9200/_template/logstash_netflow9?pretty` to take a look at the template I notice the new fields showing up on Available Fields on the left after a refresh of Kibana, but they dissapear shortly after again.
This is what the template looks like: [http://pastebin.com/FvVWwuZS](http://pastebin.com/FvVWwuZS)
Do I need to tell something that this is the template I was to be using indefinitely for netflow and have all incoming flows being shown up as it?
---
### Author:  [@anhlqn](https://discuss.elastic.co/u/anhlqn)
#### Post date: [April 5, 2016, 7:29pm UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240/5 "2016-04-05T19:29:54Z")
It looks to me that not all data are sent from Logstash to ELK. You should turn on debuging view in Logstash config first to see if all necessary Netflow fields reach Logstash and get processed correctly. \> [https://www.elastic.co/guide/en/logstash/current/plugins-outputs-stdout.html](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-stdout.html)
I wrote a series about processing Netflow with ELK: [http://www.secureict.info/2015/11/process-netflow-with-nprobe-and.html](http://www.secureict.info/2015/11/process-netflow-with-nprobe-and.html). Hope this will help you.
---
### Author:  [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)
#### Post date: [April 6, 2016, 9:54am UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240/6 "2016-04-06T09:54:08Z")
This is what my configuration looks like now after taking your advice
```
input {
tcp {
port => 2222
codec => netflow
type => "netflow"
}
udp {
port => 2222
codec => netflow
type => "netflow"
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => ["localhost:9200"]
index => "netflow-%{+YYYY.MM.dd}"
}
}
```
When checking Kibana I noticed a new entry showing netflow with all the new fields. It was from DHCP I believe as port 67 and 68 were involved.
Thought I got it working until I refreshed again a few minutes later and it was all gone. Available fields as well as the entry. There's still tons of other netflow entries like the one in my original post.
Am I missing something? Do these only happen on occasion and if so, will the fields only show up during it? After all I'm testing with a single router that doesn't deal with too much traffic right now.
---
### Author:  [@anhlqn](https://discuss.elastic.co/u/anhlqn)
#### Post date: [April 6, 2016, 4:24pm UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240/7 "2016-04-06T16:24:41Z")
Could you tell me what version of Elasticsearch you are using?
Since you have seen a few entries with all the necessary fields, I assume that Logstash works fine decoding Netflow data. If your entries are in Elasticsearch, they shouldn't disappear for no reason. One possibility that causes such weird behavior is the timestamp of the entry in ES. I got similar issue that data disappeared when I refreshed Kibana Discover.
Could you run the followings and post the outputs:
```auto
curl -XGET 'localhsot:9200/logstash-netflow9-2016.04.05/_mapping'
curl -XGET 'localhsot:9200/logstash-netflow9-2016.04.05/_search'
```
Also, did you see any errors in ES log at indexing time?
---
### Author:  [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)
#### Post date: [April 7, 2016, 7:01am UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240/8 "2016-04-07T07:01:51Z")
Thanks for your reply.
I am using Elasticsearch 2.2.1
\_mapping output:
[http://pastebin.com/HAafFjDL](http://pastebin.com/HAafFjDL)
\_search output:
[http://pastebin.com/BEPZDZ7n](http://pastebin.com/BEPZDZ7n)
I have not noticed any errors in the logs from the last day or so, I'll take another look to make sure when I can.
---
### Author:  [@anhlqn](https://discuss.elastic.co/u/anhlqn)
#### Post date: [April 7, 2016, 7:41am UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240/9 "2016-04-07T07:41:15Z")
````auto
"@timestamp":{
"type":"date",
"index":"analyzed",
"format":"strict_date_optional_time||epoch_millis"
```
This mapping may be causing you trouble. I'm not sure if having `"index": "analyzed"` in a date data type is appropriate, someone else can confirm this.
One way to try is to delete the index pattern in Kibana, recreate it but use either **first_switched** or **last_switched** as the timestamp field. If you can see the events normally, then it's your mapping issue.
````
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:56pm UTC](https://discuss.elastic.co/t/netflow-visualizations-kibana-4-limited-fields/46240/11 "2017-07-06T13:56:43Z")