# Split document into multiple documents

**URL:** https://discuss.elastic.co/t/split-document-into-multiple-documents/303305
**Category:** Logstash
**Created:** [April 26, 2022, 3:47pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305 "2022-04-26T15:47:14Z")
**Posts on this page:** 14
**Page:** 1

<div class="post-metadata">

### Author: ![ciavam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ciavam/32/100766_2.png) [@ciavam](https://discuss.elastic.co/u/ciavam)
#### Post date: [April 26, 2022, 3:47pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/1 "2022-04-26T15:47:14Z")

</div>

Hello,

Through the logstash code I was able to get a result like:

{field0 = value  
field1 = [value1\_field1; value1\_field1]  
field2 = [value1\_field2; value2\_field2]  
}

Is it possible through logstash to go to what is written below?

{field0 = value  
field1 = value1\_field1  
field2 = value1\_field2  
}

{field0 = value  
field1 = value2\_field1  
field2 = value2\_field2  
}

Thanks for those who want to help me.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 26, 2022, 4:52pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/2 "2022-04-26T16:52:52Z")

</div>

Take a look at the [split](https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html) filter.

---

<div class="post-metadata">

### Author: ![ciavam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ciavam/32/100766_2.png) [@ciavam](https://discuss.elastic.co/u/ciavam)
#### Post date: [April 26, 2022, 9:04pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/4 "2022-04-26T21:04:46Z")

</div>

Thanks for your reply @Badger. I have already seen "split". In my configuration file I tried to write:

```auto
split {
        field => "field"
        target => "field"
    }

```

In this way, however, it is possible to divide the document only for a field. Is there a way to do this, but with multiple fields?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 26, 2022, 9:13pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/5 "2022-04-26T21:13:11Z")

</div>

> [@ciavam](#):
>
> Is there a way to do this, but with multiple fields?

Use multiple split filters.

---

<div class="post-metadata">

### Author: ![ciavam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ciavam/32/100766_2.png) [@ciavam](https://discuss.elastic.co/u/ciavam)
#### Post date: [April 26, 2022, 9:25pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/6 "2022-04-26T21:25:38Z")

</div>

My goal is to have a match between the i value of the first field and the i value of the second field (as seen in my first post). Is this goal achieved with multiple split filters?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 26, 2022, 9:59pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/7 "2022-04-26T21:59:59Z")

</div>

If you start with

```
{ "field0": "z", "field1": ["a", "b"], "field2": ["c", "d"] }

```

then

```
split { field => "field1" }
split { field => "field2" }

```

will produce four events, one for each pairing: a/c, a/d, b/c, b/d.

If you want the two events then you could do it in ruby. Your sample data has three value1 and only one value2, so this may be the wrong pairing (this results in a/c, b/d)

```
    ruby {
        code => '
            f1 = event.get("field1")
            f2 = event.get("field2")
            if f1.is_a? Array and f2.is_a? Array and f1.length == f2.length
                a = []
                f1.each_index { |x|
                    a << { "field1" => f1[x], "field2" => f2[x] }
                }
                event.set("[@metadata][stuff]", a)
            end
        '
    }
    if [@metadata][stuff] {
        split { field => "[@metadata][stuff]" }
        mutate {
            replace => {
                "field1" => "%{[@metadata][stuff][field1]}"
                "field2" => "%{[@metadata][stuff][field2]}"
            }
        }
    }
```

---

<div class="post-metadata">

### Author: ![ciavam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ciavam/32/100766_2.png) [@ciavam](https://discuss.elastic.co/u/ciavam)
#### Post date: [April 26, 2022, 10:35pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/8 "2022-04-26T22:35:28Z")

</div>

Thanks again for your help. Actually my document has more than two list fields (precisely 7), so starting with:

```auto
{ "field0": "z", "field1": ["a", "b"], "field2": ["c", "d"], "field3": ["e", "f"] ... }

```

I should get to:

```auto
{ "field0" : "z",
"field1": "a",
"field2": "c",
"field3": "e"
...
 }

{ "field0": "z",
"field1": "b",
"field2": "d",
"field3": "f"
...
 }

```

Is your ruby code easily adaptable to the case of more than two list fields?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 26, 2022, 11:10pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/9 "2022-04-26T23:10:27Z")

</div>

Sure you can do

```
        f1 = event.get("field1")
        f2 = event.get("field2")
        f3 = event.get("field3")
        f4 = event.get("field4")
        f5 = event.get("field5")
        f6 = event.get("field6")
        f7 = event.get("field7")

        if f1.is_a? Array and f2.is_a? Array and f3.is_a? Array and f4.is_a? Array and f5.is_a? Array and f6.is_a? Array and f7.is_a? Array and f1.length == f2.length and f1.length == f3.length and f1.length == f4.length ...
            a = []
            f1.each_index { |x|
                a << {
                    "field1" => f1[x], "field2" => f2[x], "field3" => f3[x], 
                    "field4" => f4[x], "field5" => f5[x], "field6" => f6[x], 
                    "field7" => f7[x] 
                }
            }
            event.set("[@metadata][stuff]", a)
        end
```

---

<div class="post-metadata">

### Author: ![ciavam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ciavam/32/100766_2.png) [@ciavam](https://discuss.elastic.co/u/ciavam)
#### Post date: [April 27, 2022, 11:42am UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/10 "2022-04-27T11:42:05Z")

</div>

Thank you so much @Badger , it works! The only problem is that:

"field" =\> "{ \ "field \ " : numberValue}"

rather than:

`"field" => numberValue`

Unfortunately I'm new to logstash. Sorry for too many questions, could you help me last time?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 27, 2022, 1:17pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/11 "2022-04-27T13:17:16Z")

</div>

You are still using the split and doing the mutate for all seven fields, right?

---

<div class="post-metadata">

### Author: ![ciavam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ciavam/32/100766_2.png) [@ciavam](https://discuss.elastic.co/u/ciavam)
#### Post date: [April 27, 2022, 1:30pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/12 "2022-04-27T13:30:19Z")

</div>

Yes, right.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 27, 2022, 1:48pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/13 "2022-04-27T13:48:56Z")

</div>

> [@ciavam](#):
>
> "field" =\> "{ \ "field \ " : numberValue}"

OK, so use another mutate to

```
mutate { replace => { "field" => "%{[field][field]}" } }

```

---

<div class="post-metadata">

### Author: ![ciavam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ciavam/32/100766_2.png) [@ciavam](https://discuss.elastic.co/u/ciavam)
#### Post date: [April 27, 2022, 3:47pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/14 "2022-04-27T15:47:16Z")

</div>

Thanks again

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 25, 2022, 3:47pm UTC](https://discuss.elastic.co/t/split-document-into-multiple-documents/303305/15 "2022-05-25T15:47:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
