# Count occurences of a field excludig a pattern

**URL:** https://discuss.elastic.co/t/count-occurences-of-a-field-excludig-a-pattern/65054
**Category:** Kibana
**Created:** [November 4, 2016, 7:25pm UTC](https://discuss.elastic.co/t/count-occurences-of-a-field-excludig-a-pattern/65054 "2016-11-04T19:25:15Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![espogian](https://avatars.discourse-cdn.com/v4/letter/e/4491bb/32.png) [@espogian](https://discuss.elastic.co/u/espogian)
#### Post date: [November 4, 2016, 7:25pm UTC](https://discuss.elastic.co/t/count-occurences-of-a-field-excludig-a-pattern/65054/1 "2016-11-04T19:25:15Z")

</div>

Hi, having some occurences of a field like this (it's part of a SIP message):

```
<sip:john.doe@kibana.co>;tag=446463957-1474248204740
<sip:mark.grey@elastic.co>;tag=48768q74638634376
<sip:rose.mary@logstash.co>;tag=098974309852376086387

```

Is it possible to create a Visualization in Kibana which counts the occurences of values in this field but excluding part of the message, for example everyting before @ and everything after \>. In my example I would like to obtain this Data Table:

```
               Count
kibana.co 1
elastic.co 1
logstash.co 1

```

I've tried to use the Include or Exclude pattern, but I think I'm not using that feature properly or I'm getting something wrong.

Thanks 🙂

---

<div class="post-metadata">

### Author: ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)
#### Post date: [November 4, 2016, 7:41pm UTC](https://discuss.elastic.co/t/count-occurences-of-a-field-excludig-a-pattern/65054/2 "2016-11-04T19:41:33Z")

</div>

Typically this sort of tokenization is done at ingest time. If you are using Logstash to ingest your data into Elasticsearch, you could use the [grok filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html) to parse out the different tokens from the message and index them as separate fields in Elasticsearch. Then visualizing them in Kibana becomes not only easy but also very fast, using the Terms aggregation.

So if this performing tokenization at ingest time is an option for you, I would highly recommend doing that.

---

<div class="post-metadata">

### Author: ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)
#### Post date: [November 4, 2016, 7:49pm UTC](https://discuss.elastic.co/t/count-occurences-of-a-field-excludig-a-pattern/65054/3 "2016-11-04T19:49:20Z")

</div>

If you are on Elasticsearch (and Kibana) 5.0, you have a couple of other options:

If you don't use Logstash, you could look into using Elasticsearch's new [Ingest node](https://www.elastic.co/guide/en/elasticsearch/reference/5.0/ingest.html) to perform parsing/tokenization of the message field into separate fields in Elasticsearch itself right before indexing. This is similar to the Logstash option I suggested in the previous comment in that the tokenization happens before indexing.

That being said you could create a scripted field in Kibana, say `domain`, and use Elasticsearch's new [Painless scripting language](https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-painless.html) to parse out the domain names into a separate field at query time. This will be slower than doing the parsing/tokenization at index time as described in the Logstash or Ingest Node approaches earlier, though.

---

<div class="post-metadata">

### Author: ![espogian](https://avatars.discourse-cdn.com/v4/letter/e/4491bb/32.png) [@espogian](https://discuss.elastic.co/u/espogian)
#### Post date: [November 4, 2016, 9:02pm UTC](https://discuss.elastic.co/t/count-occurences-of-a-field-excludig-a-pattern/65054/4 "2016-11-04T21:02:05Z")

</div>

Hello @shaunak,

Thanks for your interesting replies! I'm able to manipulate the input via Logstash, so I'm following your first advice!  
However, I found interesting also your second post 🙂

Thank you!

---

<div class="post-metadata">

### Author: ![espogian](https://avatars.discourse-cdn.com/v4/letter/e/4491bb/32.png) [@espogian](https://discuss.elastic.co/u/espogian)
#### Post date: [November 4, 2016, 9:03pm UTC](https://discuss.elastic.co/t/count-occurences-of-a-field-excludig-a-pattern/65054/5 "2016-11-04T21:03:24Z")

</div>

Also, is there some detailed documentation about the Include/Exclude pattern in Kibana? I still can't get what are they used for 😊

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:34pm UTC](https://discuss.elastic.co/t/count-occurences-of-a-field-excludig-a-pattern/65054/6 "2017-07-06T13:34:25Z")

</div>


