Split a json array and then split on each object

Elastic Stack Logstash
1

I would like to configure Logstash so that given a JSON input of

[
  { "record": "a", "actions": [ { "id": 1 }, { "id": 2 }  ] },
  { "record": "b", "actions": [ { "id": 3 }, { "id": 4 }  ] }
]

it outputs

{record: "a", "action": { "id": 1 }}
{record: "a", "action": { "id": 2 }}
{record: "b", "action": { "id": 3 }}
{record: "b", "action": { "id": 4 }}

I have figured out how to split the array into two events, or one of the items into the correct output if passed by itself, but I'm not sure how to both split the input array and then also split each item by actions.

Thanks for your help!

2

What does your expected output look like?

3

Hi,

Thanks for your reply!

The expected output would be that each of the 4 outputs above would be passed on as a log event.

I'd like logstash to listen for an http request with the array indicated as the JSON input above and then pass on the 4 output documents indicated above to Elasticsearch and to the filesystem. I think I've figured out the input/output steps work, but I'm not sure how to split the input twice. First to get one object for each member of the initial input array and then to split again on the "actions" property of each of those events.

Does that answer your question? Thanks for your help!

Thanks,

Evan

4

Is the above what you want as an output? If not can you give a JSON example of what it would look like? I'm not following.

5

Yes, that is what I'd want to be output to be. So each one of those objects would be sent to Elasticsearch.

6

Thought you figured out the split already. The next part is to do the output to elasticsearch. Unless I don't understand your input correctly you should be almost done. Here is what I used to test.

input { generator { lines => ['[
  { "record": "a", "actions": [ { "id": 1 }, { "id": 2 }  ] },
  { "record": "b", "actions": [ { "id": 3 }, { "id": 4 }  ] }
]'] count => 1 codec => "json" } }
filter{
 split {
   field => "actions"
 }
}
output { stdout { codec => json } }
1 Like
7

Yeah, that works!

I was getting an error when I tried something similar, but I must have made a silly mistake. I was also testing the split using stdin rather than generator - maybe that tripped me up?

Regarding the count => 1, does that send each element to the split one by one? Would I need anything like that if that input was sent via http or from a file rather than using the generator?

How can we rename actions to action as part of the split?

Thanks so much for your help and patience.

8

The count for the generator is how many time to repeat the message. It's used for testing. You wouldn't need that for HTT input. I would really focus on your input first because it's possible the data could be structured different.

To rename fields you use the filter mutate rename function.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.