# How to create new fields and documents based on variable length log?

**URL:** https://discuss.elastic.co/t/how-to-create-new-fields-and-documents-based-on-variable-length-log/232303
**Category:** Logstash
**Created:** [May 12, 2020, 7:09pm UTC](https://discuss.elastic.co/t/how-to-create-new-fields-and-documents-based-on-variable-length-log/232303 "2020-05-12T19:09:38Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![san\_sun](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@san\_sun](https://discuss.elastic.co/u/san_sun)
#### Post date: [May 12, 2020, 7:09pm UTC](https://discuss.elastic.co/t/how-to-create-new-fields-and-documents-based-on-variable-length-log/232303/1 "2020-05-12T19:09:38Z")

</div>

I have a log message with a format like:  
status : name message ; status: name message ; status: name message ; status: name message.....

I need to split based on the semicolon, and create separate Elasticsearch documents that have fields for each of status, name and message. For example, if my log message was like this:  
WARNING: A B ; CRITICAL: C D ; OK E F ; CRITICAL G H  
Then I would need four different documents. The first document would have three fields, where status is WARNING, name is A and message is B. The second document would have status CRITICAL, name C and message D, and so on.

How do I do this?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 12, 2020, 11:17pm UTC](https://discuss.elastic.co/t/how-to-create-new-fields-and-documents-based-on-variable-length-log/232303/2 "2020-05-12T23:17:50Z")

</div>

I would start by using mutate+split to separate the string into an array of four (or fewer?) strings. Then use a split filter to make each string a separate document. Then use dissect or grok to pull out the three fields.

---

<div class="post-metadata">

### Author: ![san\_sun](https://avatars.discourse-cdn.com/v4/letter/s/d6d6ee/32.png) [@san\_sun](https://discuss.elastic.co/u/san_sun)
#### Post date: [May 13, 2020, 5:03am UTC](https://discuss.elastic.co/t/how-to-create-new-fields-and-documents-based-on-variable-length-log/232303/3 "2020-05-13T05:03:10Z")

</div>

That worked! Thank you 🙂

For future reference, in case someone else has a similar problem, here is a sample code:

```auto
filter {
    mutate {
        split => { "message" => ";" }
    }
    split {
        field => "message"
    }
    grok {
        match => { "message" => "%{WORD:level}: %{WORD:name} %{GREEDYDATA:msg}" }
        remove_field => ["message"]
    }
}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 10, 2020, 5:03am UTC](https://discuss.elastic.co/t/how-to-create-new-fields-and-documents-based-on-variable-length-log/232303/4 "2020-06-10T05:03:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
