# Field missing in Elasticsearch/kibana

**URL:** <https://discuss.elastic.co/t/field-missing-in-elasticsearch-kibana/60381>\
**Category:** Logstash\
**Created:** [September 13, 2016, 1:16pm UTC](https://discuss.elastic.co/t/field-missing-in-elasticsearch-kibana/60381 "2016-09-13T13:16:48Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![trenzalore](https://avatars.discourse-cdn.com/v4/letter/t/ac91a4/32.png) [@trenzalore](https://discuss.elastic.co/u/trenzalore)\
**Post date:** [September 13, 2016, 1:16pm UTC](https://discuss.elastic.co/t/field-missing-in-elasticsearch-kibana/60381/1 "2016-09-13T13:16:48Z")

</div>

Hi !

So I made a huge logstash config for my wifi hotspots.  
I managed to create nearly all the filters but I have a problem with one of them.

I receive this log:

`<134>1 1473769601.251850936 APVDA001 flows allow src=172.27.7.238 dst=64.233.166.188 mac=4C:34:88:01:2B:5B protocol=tcp sport=55267 dport=5228`

And I put this grok filter:

`%{SYSLOG5424PRI}%{INT} %{NUMBER:timestamp_unix} %{WORD:borne} flows %{WORD:flow_type} src=%{IP:ip_source} dst=%{IP:ip_dest} mac=%{MAC:mac} protocol=%{WORD:protocol} sport=%{NUMBER:sport} dport=%{NUMBER:dport}`

It works well with a grok debugger but some fields do not appear in Kibana: the fields ip\_source, ip\_dest, mac, protocol, sport and dport.

Why ? It parses well the others !

Thanks

---

<div class="post-metadata">

**Author:** ![Alex\_6](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_6/32/1330_2.png) [@Alex\_6](https://discuss.elastic.co/u/Alex_6)\
**Post date:** [September 13, 2016, 2:41pm UTC](https://discuss.elastic.co/t/field-missing-in-elasticsearch-kibana/60381/2 "2016-09-13T14:41:43Z")

</div>

Have you confirmed that Logstash is extracting the fields properly? So it's just Kibana which is not displaying them?

---

<div class="post-metadata">

**Author:** ![trenzalore](https://avatars.discourse-cdn.com/v4/letter/t/ac91a4/32.png) [@trenzalore](https://discuss.elastic.co/u/trenzalore)\
**Post date:** [September 13, 2016, 2:49pm UTC](https://discuss.elastic.co/t/field-missing-in-elasticsearch-kibana/60381/3 "2016-09-13T14:49:30Z")

</div>

Well logstash extracts borne and flow\_type so I think that it extracts fields properly.

How do I check if it works well ?

If logtash works well, then yes, it's a problem with Kibana (or maybe with elasticsearch ?).  
On kibana I have the field borne and the field flow\_type so why not the others ?

---

<div class="post-metadata">

**Author:** ![Ashutosh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashutosh/32/11051_2.png) [@Ashutosh](https://discuss.elastic.co/u/Ashutosh)\
**Post date:** [September 14, 2016, 6:26am UTC](https://discuss.elastic.co/t/field-missing-in-elasticsearch-kibana/60381/4 "2016-09-14T06:26:47Z")

</div>

Did you try to refresh the field lists in kibana?  
In kibana, Go to settings \>\> Indices  
Select which index you are having trouble with and use the Yellow colored 'Refresh' button to refresh the fields list.

Check if this helps.

---

<div class="post-metadata">

**Author:** ![trenzalore](https://avatars.discourse-cdn.com/v4/letter/t/ac91a4/32.png) [@trenzalore](https://discuss.elastic.co/u/trenzalore)\
**Post date:** [September 14, 2016, 7:25am UTC](https://discuss.elastic.co/t/field-missing-in-elasticsearch-kibana/60381/5 "2016-09-14T07:25:12Z")

</div>

Yes I did it and I still have the problem...

I'll try a logtash config with just the grok for this log and see if it works.

EDIT: with just this grok pattern, it works. I have all the fields.  
Maybe another grok pattern matches first ? (I checked my config file and I see nothing..)

EDIT 2: Here are the two config files I use, I hope it'll help (the cisco one is huge)

10-cisco.conf:

> <https://gist.github.com/newclem/4d93cc5d1ebbfa8471ebe551c0154ad8>

10-dhcp.conf:

> <https://gist.github.com/newclem/974794b23c8cd5ff51514cd28af05c0f>

EDIT 3: I FOUND THE ERROR.

In fact there is a sense of priority in grok patterns. I commented 2 patterns and it works again.  
Those 2 patterns had the same begin pattern and nothinf after.  
Now I'll see how to have those 2 patterns back. I don't want any grokparse failure

EDIT 4:  
Everything work thanks 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:38am UTC](https://discuss.elastic.co/t/field-missing-in-elasticsearch-kibana/60381/6 "2017-07-06T04:38:33Z")

</div>


