# Splitting multiple arrays of objects from a single log

**URL:** <https://discuss.elastic.co/t/splitting-multiple-arrays-of-objects-from-a-single-log/195375>\
**Category:** Logstash\
**Created:** [August 15, 2019, 5:01pm UTC](https://discuss.elastic.co/t/splitting-multiple-arrays-of-objects-from-a-single-log/195375 "2019-08-15T17:01:27Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\
**Post date:** [August 15, 2019, 5:01pm UTC](https://discuss.elastic.co/t/splitting-multiple-arrays-of-objects-from-a-single-log/195375/1 "2019-08-15T17:01:27Z")

</div>

If I need to split multiple arrays of objects from a single log, let's say, I have 3 arrays in a log, let's say something like this:

"arrays1" : [{o1}, {o2}, {o3}, {o4}, {o5}, {o6}]  
"arrays2" : [{ob1}, {ob2}, {ob3}, {ob4}, {ob5}, {ob6}, {ob7}]  
"arrays3" : [{obj1}, {obj2}, {obj3}]

and, I want to split them all and create new documents for individual objects from the above arrays.

Since the arrays are not the same size, how Elasticsearch splits above arrays and create new documents?

Does it follow this pattern or some other completely different?

doc1 = o1, ob1, obj1  
doc2 = o2, ob2, obj2  
doc3 = o3, ob3, obj3  
doc4 = o4, ob4, -  
doc5 = o5, ob5, -  
doc6 = o6, ob6, -  
doc7 = -, ob7, -

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 15, 2019, 5:22pm UTC](https://discuss.elastic.co/t/splitting-multiple-arrays-of-objects-from-a-single-log/195375/2 "2019-08-15T17:22:21Z")

</div>

How do you want it to be split?

---

<div class="post-metadata">

**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\
**Post date:** [August 15, 2019, 5:38pm UTC](https://discuss.elastic.co/t/splitting-multiple-arrays-of-objects-from-a-single-log/195375/3 "2019-08-15T17:38:27Z")

</div>

I want to get rid of arrays of objects. What's the default split method? If I use the split module in Logstash, how the above arrays get split?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 15, 2019, 6:40pm UTC](https://discuss.elastic.co/t/splitting-multiple-arrays-of-objects-from-a-single-log/195375/4 "2019-08-15T18:40:59Z")

</div>

If you do

```
    split { field => "arrays1" }
    split { field => "arrays2" }
    split { field => "arrays3" }

```

You will get 126 events (3 \* 6 \* 7).

---

<div class="post-metadata">

**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\
**Post date:** [August 15, 2019, 7:17pm UTC](https://discuss.elastic.co/t/splitting-multiple-arrays-of-objects-from-a-single-log/195375/5 "2019-08-15T19:17:58Z")

</div>

@Badger great, this is what I was looking for, the final number. The default method seems to be doing permutation on all data sets. Is there any other methods I can use to split arrays and don't create that many documents?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 15, 2019, 7:54pm UTC](https://discuss.elastic.co/t/splitting-multiple-arrays-of-objects-from-a-single-log/195375/6 "2019-08-15T19:54:06Z")

</div>

> [@daniel\_a](#):
>
> Is there any other methods I can use to split arrays and don't create that many documents?

Sure. It really depends what you want.

---

<div class="post-metadata">

**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\
**Post date:** [August 15, 2019, 9:32pm UTC](https://discuss.elastic.co/t/splitting-multiple-arrays-of-objects-from-a-single-log/195375/7 "2019-08-15T21:32:54Z")

</div>

This is something I'd need to figure out by looking at the logs themselves at the source. How are the arrays even created at the first place? Are they created at the source or somewhere in transit?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 12, 2019, 9:33pm UTC](https://discuss.elastic.co/t/splitting-multiple-arrays-of-objects-from-a-single-log/195375/8 "2019-09-12T21:33:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.