Splitting multiple arrays of objects from a single log

Elastic Stack Logstash
1

If I need to split multiple arrays of objects from a single log, let's say, I have 3 arrays in a log, let's say something like this:

"arrays1" : [{o1}, {o2}, {o3}, {o4}, {o5}, {o6}]
"arrays2" : [{ob1}, {ob2}, {ob3}, {ob4}, {ob5}, {ob6}, {ob7}]
"arrays3" : [{obj1}, {obj2}, {obj3}]

and, I want to split them all and create new documents for individual objects from the above arrays.

Since the arrays are not the same size, how Elasticsearch splits above arrays and create new documents?

Does it follow this pattern or some other completely different?

doc1 = o1, ob1, obj1
doc2 = o2, ob2, obj2
doc3 = o3, ob3, obj3
doc4 = o4, ob4, -
doc5 = o5, ob5, -
doc6 = o6, ob6, -
doc7 = -, ob7, -

2

How do you want it to be split?

3

I want to get rid of arrays of objects. What's the default split method? If I use the split module in Logstash, how the above arrays get split?

4

If you do

    split { field => "arrays1" }
    split { field => "arrays2" }
    split { field => "arrays3" }

You will get 126 events (3 * 6 * 7).

5

@Badger great, this is what I was looking for, the final number. The default method seems to be doing permutation on all data sets. Is there any other methods I can use to split arrays and don't create that many documents?

6

Sure. It really depends what you want.

7

This is something I'd need to figure out by looking at the logs themselves at the source. How are the arrays even created at the first place? Are they created at the source or somewhere in transit?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.