# Fields in messages

**URL:** https://discuss.elastic.co/t/fields-in-messages/116785
**Category:** Logstash
**Created:** [January 24, 2018, 2:15am UTC](https://discuss.elastic.co/t/fields-in-messages/116785 "2018-01-24T02:15:54Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![kayaHL](https://avatars.discourse-cdn.com/v4/letter/k/bbce88/32.png) [@kayaHL](https://discuss.elastic.co/u/kayaHL)
#### Post date: [January 24, 2018, 2:15am UTC](https://discuss.elastic.co/t/fields-in-messages/116785/1 "2018-01-24T02:15:54Z")

</div>

Hi guys, I am new to Elasticsearch... I am using filebeat to ship the multiple line unstructured messages to logstash and use grok filter to structure the data. However, I noticed in the application logs, between messages, some messages have less field/value pair, some messages have more field/value pair, the grok  
patterns doesn't work for all messages... so of the messages can't be structured. Could someone help me and give me a right direction for the solution?

Example:

message 1:

04:45:01.633 - R (1,2) msglen:89 New  
Timestamp:109660 ItemCode:TEST1 ListingDate:20100102 **Expiry** :0 Term:

message 2:

04:45:01.633 - R (5,9) msglen:97 DELETE  
Timestamp:20988 ItemCode:TEST2 **ListingType** :CIN ListingDate:20160102 Term: **Processingdate** : 2017-0108

Many thanks!

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 24, 2018, 8:44am UTC](https://discuss.elastic.co/t/fields-in-messages/116785/2 "2018-01-24T08:44:04Z")

</div>

I believe it's more a #logstash question so I moved your post.

BTW please format your code using `</>` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and not the citation button. It will make your post more readable.

Or use markdown style like:

````
```
CODE
```

````

Please edit your post.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 26, 2018, 7:49am UTC](https://discuss.elastic.co/t/fields-in-messages/116785/3 "2018-01-26T07:49:12Z")

</div>

Use a kv filter to parse lists of key/value pairs, not grok.

---

<div class="post-metadata">

### Author: ![kayaHL](https://avatars.discourse-cdn.com/v4/letter/k/bbce88/32.png) [@kayaHL](https://discuss.elastic.co/u/kayaHL)
#### Post date: [January 26, 2018, 5:41pm UTC](https://discuss.elastic.co/t/fields-in-messages/116785/4 "2018-01-26T17:41:43Z")

</div>

Thank you! I will give it a try.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 23, 2018, 5:42pm UTC](https://discuss.elastic.co/t/fields-in-messages/116785/5 "2018-02-23T17:42:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.