# Is posible to have iptional field with regex as grok optionals fields?

**URL:** <https://discuss.elastic.co/t/is-posible-to-have-iptional-field-with-regex-as-grok-optionals-fields/290645>\
**Category:** Logstash\
**Created:** [December 1, 2021, 9:52am UTC](https://discuss.elastic.co/t/is-posible-to-have-iptional-field-with-regex-as-grok-optionals-fields/290645 "2021-12-01T09:52:31Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\
**Post date:** [December 1, 2021, 9:52am UTC](https://discuss.elastic.co/t/is-posible-to-have-iptional-field-with-regex-as-grok-optionals-fields/290645/1 "2021-12-01T09:52:31Z")

</div>

Hello!

i want to know if a could set a field as optional in regular expresions as i do with grok

for example:

(%{DATA:Message})?

Thank you!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [December 1, 2021, 7:23pm UTC](https://discuss.elastic.co/t/is-posible-to-have-iptional-field-with-regex-as-grok-optionals-fields/290645/2 "2021-12-01T19:23:05Z")

</div>

Yes, you can use exactly that syntax to match zero or more %{DATA:Message} patterns. Note that DATA can match zero characters, so sometimes it will not consume anything.

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [December 1, 2021, 7:24pm UTC](https://discuss.elastic.co/t/is-posible-to-have-iptional-field-with-regex-as-grok-optionals-fields/290645/3 "2021-12-01T19:24:43Z")

</div>

Absolutely!  
This pattern:

```auto
%{WORD}? %{NUMBER} %{WORD}?

```

Will match:

```auto
ABC 123 XZY

```

```auto
{
  "WORD": [
    [
      "ABC",
      "XZY"
    ]
  ],
  "NUMBER": [
    [
      "123"
    ]
  ],
  "BASE10NUM": [
    [
      "123"
    ]
  ]
}

```

or it will match

```auto
ABC 123 

```

```auto
{
  "WORD": [
    [
      "ABC",
      null
    ]
  ],
  "NUMBER": [
    [
      "123"
    ]
  ],
  "BASE10NUM": [
    [
      "123"
    ]
  ]
}

```

---

<div class="post-metadata">

**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\
**Post date:** [December 7, 2021, 11:42am UTC](https://discuss.elastic.co/t/is-posible-to-have-iptional-field-with-regex-as-grok-optionals-fields/290645/4 "2021-12-07T11:42:26Z")

</div>

can i do it with texts ?

my log follows this format:

```auto
2021-12-07 09:59:33,940 INFO [xxxxxxxxxxxxxxxxxxxxxxxx][YYYYYYYYYYYYYYYYYY] Parámetros enviados a eeeee:
	action: efdwfwf_ewfwef_ewf
	action2 : wefwefwe:_FWwefwe
	type : feAASDF
	log : logfile

```

and this parameter with the format name : value could not appear.

so im trying something like:

`(action: %{USERNAME:Action})?%{DATA}(action2 : %{USERNAME:Action2})?%{DATA}(type: %{USERNAME:Type})?`

how can i do that?

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [December 7, 2021, 2:40pm UTC](https://discuss.elastic.co/t/is-posible-to-have-iptional-field-with-regex-as-grok-optionals-fields/290645/5 "2021-12-07T14:40:51Z")

</div>

This looks like a multiline log format so be sure you configure your input to also handle multiline messages, however afterwards I got this pattern to work with the grokdebugger

```auto
%{SPACE}(action: %{USERNAME:Action})?%{SPACE}(action2 : %{NOTSPACE:Action2})?%{SPACE}(type : %{USERNAME:Type})?%{SPACE}(log : %{NOTSPACE:log})?

```

```auto
{
  "SPACE": [
    [
      "\t",
      "\n\t",
      "\n\t",
      "\n\t"
    ]
  ],
  "Action": [
    [
      "efdwfwf_ewfwef_ewf"
    ]
  ],
  "Action2": [
    [
      "wefwefwe:_FWwefwe"
    ]
  ],
  "Type": [
    [
      "feAASDF"
    ]
  ],
  "log": [
    [
      "logfile"
    ]
  ]
}

```

---

<div class="post-metadata">

**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\
**Post date:** [December 9, 2021, 8:18am UTC](https://discuss.elastic.co/t/is-posible-to-have-iptional-field-with-regex-as-grok-optionals-fields/290645/6 "2021-12-09T08:18:25Z")

</div>

that's what i assumed i would have to do but when i make it optional it disappears.

Real example:

```auto
2021-12-09 08:50:03,938 INFO [WebContainer : x] I.xxxxxDatosPersonales [AfphLog.java:209] [09/12/2021 08:50:03][1YYYYYYYYY.XXXXXXDatosPersonales] Parámetros enviados a ZZZZZ:
    accion : ZZZZ
    accionProsa : null
    tipoServicio : YYYYYY
    log : Importass.XXXXXXXX

```

this grok works

`(?m)%{TIMESTAMP_ISO8601:Fecha} %{LOGLEVEL:LogLevel} \[WebContainer : %{NUMBER:WebContainer}] %{DATA:Entidad} \[AfphLog.java:%{NUMBER:AfphLogJava}]%{GREEDYDATA}accion : %{USERNAME:Accion}`

result:

```auto
{
  "Fecha": "2021-12-09 08:50:03,938",
  "WebContainer": "x",
  "AfphLogJava": "209",
  "Accion": "ZZZZ",
  "LogLevel": "INFO",
  "Entidad": "1YYYYYYYYY.XXXXXXDatosPersonales"
}

```

but if i put the grok filter like this:

`(?m)%{TIMESTAMP_ISO8601:Fecha} %{LOGLEVEL:LogLevel} \[WebContainer : %{NUMBER:WebContainer}] %{DATA:Entidad} \[AfphLog.java:%{NUMBER:AfphLogJava}]%{GREEDYDATA}(accion : %{USERNAME:Accion})?`

the accion field dissapears:

```auto
{
  "Fecha": "2021-12-09 08:50:03,938",
  "WebContainer": "7",
  "AfphLogJava": "209",
  "LogLevel": "INFO",
  "Entidad": "I.AFPH03DatosPersonales"
}

```

why the optional field dissapears?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 6, 2022, 8:19am UTC](https://discuss.elastic.co/t/is-posible-to-have-iptional-field-with-regex-as-grok-optionals-fields/290645/7 "2022-01-06T08:19:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.