# Grok "?" operator

**URL:** <https://discuss.elastic.co/t/grok-operator/68653>\
**Category:** Logstash\
**Created:** [December 11, 2016, 9:38pm UTC](https://discuss.elastic.co/t/grok-operator/68653 "2016-12-11T21:38:33Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\
**Post date:** [December 11, 2016, 9:38pm UTC](https://discuss.elastic.co/t/grok-operator/68653/1 "2016-12-11T21:38:33Z")

</div>

Hello.

I am using a grok filter on certain logs that give me a problem.  
A small part of them come out with one less field than the others.  
example given :

> input log 1  
> `num=10 event=attack source=mach1`

> input log 2  
> `num=10 source=mach2`

As of now i need to check with filebeats for ["event" in msg] but with other small diffirences in the mass ammount of logs there will be a huge if/case or w/e to make the right use of the filter plus the creation of many diffirent filters.

Conclusion: I need to make use of the "?" operator as used in regular expressions but i fail to use it right.  
Basically i would need something like:  
`number=%{NUMBER:num} (event=%{WORD:event} )?source=%{WORD:source}`

But it doesn't work like this and i can't find the right way to use it in grok.What is the right syntax for something like this so i won't need to make use of if condition in filebeat?Or actually is it even possble?  
One thing i could think of would be something close to `( | event=%{})` but i aint sure how to implement it the right way.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 11, 2016, 9:55pm UTC](https://discuss.elastic.co/t/grok-operator/68653/2 "2016-12-11T21:55:08Z")

</div>

I would recommend you have a look at the [kv filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html), which is designed to parse lists of key-value pairs that can appear in any order to only sometimes be present.

---

<div class="post-metadata">

**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\
**Post date:** [December 11, 2016, 10:53pm UTC](https://discuss.elastic.co/t/grok-operator/68653/3 "2016-12-11T22:53:49Z")

</div>

Thanks for your reply.  
1st if someone can give me an answer i would appreciate it a lot.I would like to learn how to do it.  
2nd is the kv filter good performance wise compared to making my own regexes+patterns for everything?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 12, 2016, 1:59pm UTC](https://discuss.elastic.co/t/grok-operator/68653/4 "2016-12-12T13:59:13Z")

</div>

@sancroth It seems to me that this question would better fit into the Logstash forum?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [December 12, 2016, 2:02pm UTC](https://discuss.elastic.co/t/grok-operator/68653/5 "2016-12-12T14:02:43Z")

</div>

I think logstash/ingest node use oniguruma regex engine. The grok pattern will be compiled down to a regex pattern using named capture groups. See logstash grok docs on [Custom Patterns](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#_custom_patterns) for using a capture group directly.

---

<div class="post-metadata">

**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\
**Post date:** [December 14, 2016, 4:47pm UTC](https://discuss.elastic.co/t/grok-operator/68653/6 "2016-12-14T16:47:41Z")

</div>

For some reason i thought i posted it there :o Terribly sorry.If a mod can swap places it would be nice 🙂 (looking at you ruf 😃 )

---

<div class="post-metadata">

**Author:** ![sancroth](https://avatars.discourse-cdn.com/v4/letter/s/838e76/32.png) [@sancroth](https://discuss.elastic.co/u/sancroth)\
**Post date:** [December 14, 2016, 4:50pm UTC](https://discuss.elastic.co/t/grok-operator/68653/7 "2016-12-14T16:50:29Z")

</div>

And just for the information and the help you given me,  
I had to manipulate fortigate logs.I went with KV filter as suggested after clearing the first part of the logs with regexes.  
Only thing i haven't figured out yet is if there is a way to change data types with KV since it automatically matches the delimeter of choice and creates the fields.  
With grok you could use %{NUMBER:num:int} and BOOM.Still looking in to it.

Thanks for everyones help!Really appreciated!

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 16, 2016, 7:35am UTC](https://discuss.elastic.co/t/grok-operator/68653/8 "2016-12-16T07:35:18Z")

</div>

I just moved the topic to logstash.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 1, 2017, 9:38pm UTC](https://discuss.elastic.co/t/grok-operator/68653/9 "2017-01-01T21:38:58Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.
