# Parse log line into JSON even when some tags from log can miss

**URL:** <https://discuss.elastic.co/t/parse-log-line-into-json-even-when-some-tags-from-log-can-miss/133559>\
**Category:** Logstash\
**Created:** [May 28, 2018, 2:14pm UTC](https://discuss.elastic.co/t/parse-log-line-into-json-even-when-some-tags-from-log-can-miss/133559 "2018-05-28T14:14:23Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vinitc](https://avatars.discourse-cdn.com/v4/letter/v/f07891/32.png) [@Vinitc](https://discuss.elastic.co/u/Vinitc)\
**Post date:** [May 28, 2018, 2:14pm UTC](https://discuss.elastic.co/t/parse-log-line-into-json-even-when-some-tags-from-log-can-miss/133559/1 "2018-05-28T14:14:23Z")

</div>

Hi,

My log line looks like below. In my log it is possible that few tags could miss (example: CITY or ZIPCODE or any other tag may not always be present). I want to parse this log and show output as JSON.

**Log Line:**  
20180524T104101.550 GMT, ID=A-124, FIRST\_NAME=ABC, LAST\_NAME=XYZ, ADDRESS1=123 STREET, CITY=NEW YORK, STATE=NEW YORK ZIPCODE=40001

I tried giving regex patterns but they work only when all tags are present if any of the random tags are not present then I get \_grokparsefailure

Can you please help me know if it is possible to parse log into JSON even when some tags can miss.

Thank you!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 28, 2018, 4:52pm UTC](https://discuss.elastic.co/t/parse-log-line-into-json-even-when-some-tags-from-log-can-miss/133559/2 "2018-05-28T16:52:24Z")

</div>

Use a grok filter to extract the timestamp into one field and the rest of the string into another field. Then feed the latter field to a kv filter.

---

<div class="post-metadata">

**Author:** ![Vinitc](https://avatars.discourse-cdn.com/v4/letter/v/f07891/32.png) [@Vinitc](https://discuss.elastic.co/u/Vinitc)\
**Post date:** [May 28, 2018, 6:01pm UTC](https://discuss.elastic.co/t/parse-log-line-into-json-even-when-some-tags-from-log-can-miss/133559/3 "2018-05-28T18:01:20Z")

</div>

Thanks. Let me try above suggestion.

---

<div class="post-metadata">

**Author:** ![Vinitc](https://avatars.discourse-cdn.com/v4/letter/v/f07891/32.png) [@Vinitc](https://discuss.elastic.co/u/Vinitc)\
**Post date:** [May 29, 2018, 2:44am UTC](https://discuss.elastic.co/t/parse-log-line-into-json-even-when-some-tags-from-log-can-miss/133559/4 "2018-05-29T02:44:48Z")

</div>

It worked. Thank you very much. I want to remove default fields like host, message, @timestamp, @version, etc. Is there a way to do it?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 29, 2018, 5:55am UTC](https://discuss.elastic.co/t/parse-log-line-into-json-even-when-some-tags-from-log-can-miss/133559/5 "2018-05-29T05:55:30Z")

</div>

You can't remove `@timestamp` but all other fields can be removed with a `remove_field` option in any existing filter or in a mutate filter of its own.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 26, 2018, 5:55am UTC](https://discuss.elastic.co/t/parse-log-line-into-json-even-when-some-tags-from-log-can-miss/133559/6 "2018-06-26T05:55:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.