# Field doesn't create when field value is empty

**URL:** <https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839>\
**Category:** Logstash\
**Created:** [March 7, 2018, 7:36am UTC](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839 "2018-03-07T07:36:14Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [March 7, 2018, 7:36am UTC](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839/1 "2018-03-07T07:36:14Z")

</div>

hi elastic team,

i would like to ask one thing.

when i check my log in `https://grokdebug.herokuapp.com/`, i got expect result

my test log is

> 2018-03-07 10:13:56,256 (null) 24 WARN : TEST.Controllers.PaymentController : zlbjkexpwcb5pai1zrsexpgp ::: TEST : {"version":"7.2","request\_timestamp":"2018-03-07 10:13:36","sid":"764764000000962","ref\_id":"","desc":"TESTTEST"} : 5932958ms

> {  
> "log\_timestamp": [  
> [  
> "2018-03-07 10:13:56,256"  
> ]  
> ],  
> "threadno": [  
> [  
> "24"  
> ]  
> ],  
> "log\_level": [  
> [  
> "WARN"  
> ]  
> ],  
> "session\_id": [  
> [  
> "zlbjkexpwcb5pai1zrsexpgp"  
> ]  
> ],  
> "sid": [  
> [  
> "764764000000962"  
> ]  
> ],  
> "ref\_id": [  
> [  
> """  
> ]  
> ]  
> }

but when i ship my log to logstash, **ref\_id** field is missing when its value is empty.

how do i get **ref\_id** field if it's value is empty or not????

i'm running logstash 6.1.1 and i use grok filter

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 7, 2018, 1:55pm UTC](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839/2 "2018-03-07T13:55:36Z")

</div>

What does your configuration look like?

---

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [March 8, 2018, 10:29am UTC](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839/3 "2018-03-08T10:29:52Z")

</div>

sorry i forgot to paste it.

here it is

> input {  
> stdin {  
> type =\> "true"  
> }   
> }
> 
> filter {  
> grok {   
> match =\> { "message" =\> ["^(?\<log\_timestamp\>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{1,3})._?(?\d+)._?%{WORD:log\_level}._?TEST._?\s+:\s+(?\<session\_id\>._?)\s{0,}:::._?"request\_timestamp":"(?._?)","sid":"(?._?)","ref\_id":"(?\<ref\_id\>._?)","desc":"(?._?)""]}   
> add\_field =\> { "api\_code" =\> "test" }  
> }
> 
> if ![ref\_id] {  
> mutate { add\_field =\> { "ref\_id" =\> "-" } }  
> }
> 
> }
> 
> output {  
> stdout {  
> codec =\> rubydebug  
> }   
> }

right now i add ![ref\_id]. this way ok for now. but i have no idea it is best practice or not.  
plz, advice me. thank you 🙂

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 8, 2018, 1:43pm UTC](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839/4 "2018-03-08T13:43:05Z")

</div>

Don't parse JSON with a grok filter. Use a json filter.

---

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [March 9, 2018, 2:04am UTC](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839/5 "2018-03-09T02:04:56Z")

</div>

Does JSON filter support for writing patterns??

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 9, 2018, 6:55am UTC](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839/6 "2018-03-09T06:55:43Z")

</div>

No, the JSON filter parses JSON. Use a grok filter to extract the timestamp, loglevel, and what else to separate fields, including the JSON string. Then use the json filter to parse the field with the JSON.

---

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [March 9, 2018, 7:18am UTC](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839/7 "2018-03-09T07:18:56Z")

</div>

i see thanks @magnusbaeck

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 6, 2018, 7:26am UTC](https://discuss.elastic.co/t/field-doesnt-create-when-field-value-is-empty/122839/8 "2018-04-06T07:26:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.