# Grok can't create optional fields

**URL:** <https://discuss.elastic.co/t/grok-cant-create-optional-fields/251631>\
**Category:** Logstash\
**Created:** [October 10, 2020, 7:20am UTC](https://discuss.elastic.co/t/grok-cant-create-optional-fields/251631 "2020-10-10T07:20:40Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bhavin\_Varsur](https://avatars.discourse-cdn.com/v4/letter/b/ecccb3/32.png) [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\
**Post date:** [October 10, 2020, 7:20am UTC](https://discuss.elastic.co/t/grok-cant-create-optional-fields/251631/1 "2020-10-10T07:20:41Z")

</div>

I'm trying to parse logs in logstash with use of grok. In error log there are two more optional fields like requestUrl and requestMethod. but when i use grok for parsing logs to logstash.these from the error log grok can't create these two fields. here is my grok pattern :

```auto
    if [fields][log_type] == "finance" 
        {
           if [level] in ["Error", "Fatal"] 
            {
                grok {
                    match=> ["message","%{DATESTAMP:time} \[%{WORD:processId}\] %{LOGLEVEL:level} %{USERNAME:logger} %{USER:user} %{IPV4:clientIp} %{URI:requestUrl} %{USER:requestMethod} %{GREEDYDATA:message}"]
                    overwrite => ["message"]
                    }
                    mutate { gsub => ["message", "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{4} ",""]}
                    mutate { gsub => ["message", "%{level}",""]}
                    mutate { gsub => ["message", "%{logger}",""]}
                    mutate { gsub => ["message", "%{clientIp}",""]}
            }
            else
            {
                grok {
                    match=> ["message","%{DATESTAMP:time} \[%{WORD:processId}\] %{LOGLEVEL:level} %{USERNAME:logger} %{USER:user} %{IPV4:clientIp} %{GREEDYDATA:message}" ]
                    overwrite => ["message"]
                    }
                    mutate { gsub => ["message", "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{4} ",""]}
                    mutate { gsub => ["message", "%{level}",""]}
                    mutate { gsub => ["message", "%{logger}",""]}
                    mutate { gsub => ["message", "%{clientIp}",""]}
            }
       }  

```

here are my log sample :

```auto
2020-10-10 12:25:07.0891 [35476] DEBUG Program bhavin 192.168.43.244 init main 
2020-10-10 12:25:09.8357 [35476] WARN HttpsRedirectionMiddleware bhavin 192.168.43.244 Failed to determine the https port for redirect. 
2020-10-10 12:25:10.7106 [35476] ERROR DeveloperExceptionPageMiddleware bhavin 192.168.43.244 http://bhavin/Transactions POST An unhandled exception has occurred while executing the request.System.InvalidOperationException

```

here is the image of kibana view :

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/d/2dbcfff5fde6ed869b749cc6b235f68c353b5001.png)

you can see other fields are created but these two are not created

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 7, 2020, 7:20am UTC](https://discuss.elastic.co/t/grok-cant-create-optional-fields/251631/2 "2020-11-07T07:20:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.