Creating a field of field

Elastic Stack Logstash
1

Hello,

I have this field in my JSON file which looks like this :

"activeAlarms":[{"id":*****, "description":"something", "repairAction": "do this"}, {"id":*****, "description":"somethingElse", "repairAction": "do that}, etc...]

What I'd like to do is to create the fields "id", "description", "repairAction".
I tried multiple things such as splitting activeAlarms and then splitting the others fields but it didn't quite work out yet.

2

What have you tried and what do you not like about the result?

3

Ok so right now the fields activeAlarms.id.keyword, etc.. seem to exist (although they appear in "missing fields" in Kibana, i don't know why). When I try to use them in a data table, the "id" of the first activeAlarms isn't related to the "description" and the "repairAction" of the first alarm. What it does is all the combinations of all the alarms: 1st "id" with 1st "description" and 1st "repairAction" then 1st "id" with 1st descritption and 2nd "repairAction" and so on, which is useless.
I'm pretty sure I can't solve the problem with Kibana,
in order to link these fields I think I have to recreate them.

I know it's very unclear sorry.

4

I would expect that using a split filter on activeAlarms to create two events, one containing

{"id":*****, "description":"something", "repairAction": "do this"}

and the other containing

{"id":*****, "description":"somethingElse", "repairAction": "do that"}

would be what you want.

5

Yes that would do it ! But the split doesn't work because it only does on "array and string" . I tried to convert activeAlarms into String first but my logstash crashed.

6

You showed activeAlarms as an array. If you use

output { stdout { codec => rubydebug } }

then what does the resulting output show?

7

It doesn't show anything
I just have this error every time I try to split this field.

java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid11645.hprof ...

Could it be because the field is too big ? I used a simple example in my post but in fact there is in average 40 "active alarms" and each one of them has like 20 fields whose some of them are quite big too

8

Try this.

9

Mmhh logstash didn't respond

10

it didn't create the hprof file though

11

I really think that will do it though, is there a reason you can find to logstash not responding ? I have to use SIGKILL to stop it

12

That is normal. logstash is waiting for the inputs to generate more events. It will wait forever on most inputs.

13

I know but usually a ^C works
I tried the code and the script you gave me in a new file conf and it works perfectly :slight_smile:
But when I add it in my original file conf it doesn't, maybe it is too dense, I do make a lot of splits etc...

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.