Need help on to extact JSON data from filed and send as multiple events

[ssuccess2003]

Hi Team,

I need help on the below requirement. I have developed a custome beat which stores powershell JSON output in a string field and send it to log stash. I want to extract this stringified JSON and send it as seperate events.

Powershell JSON output which i store it in a string field and publish the event(via beats)

[{
"Name": "__GENUS",
"Value": 2,
"Type": 3,
"IsLocal": true,
"IsArray": false,
"Origin": "___SYSTEM",
"Qualifiers": ""
},
{
"Name": "__CLASS",
"Value": "Win32_Processor",
"Type": 8,
"IsLocal": true,
"IsArray": false,
"Origin": "___SYSTEM",
"Qualifiers": ""
}]

Beats Output to logstash as below:

{
"_index": "logstash-2018.05.06",
"_type": "doc",
"_id": "lqcsNmMBKmiJGSDBbfkb",
"_version": 1,
"_score": null,
"_source": {
"JDATA": "[{\r\n "Name": "__GENUS",\r\n "Value": 2,\r\n "Type": 3,\r\n "IsLocal": true,\r\n "IsArray": false,\r\n "Origin": "___SYSTEM",\r\n "Qualifiers": ""\r\n},\r\n{\r\n "Name": "__CLASS",\r\n "Value": "Win32_Processor",\r\n "Type": 8,\r\n "IsLocal": true,\r\n "IsArray": false,\r\n "Origin": "___SYSTEM",\r\n "Qualifiers": ""\r\n}]",
"type": "DESKTOP-75FCJS8",
"@version": "1",
"counter": 41,
"beat": {
"name": "DESKTOP-75FCJS8",
"hostname": "DESKTOP-75FCJS8",
"version": "7.0.0-alpha1"
},
"tags": [
"beats_input_raw_event"
],
"@timestamp": "2018-05-06T15:57:50.104Z",
"host": "DESKTOP-75FCJS8"
},
"fields": {
"@timestamp": [
"2018-05-06T15:57:50.104Z"
]
},
"sort": [
1525622270104
]
}

My requirement is mentioned below:

i need extract json from JDATA field, split it and send it as seperate events(event1, event2)

i want output of logstash to send it to elastic search should be as below:
event 1:
{
"Name": "__GENUS",
"Value": 2,
"Type": 3,
"IsLocal": true,
"IsArray": false,
"Origin": "___SYSTEM",
"Qualifiers": ""
}

event 2:
{
"Name": "__CLASS",
"Value": "Win32_Processor",
"Type": 8,
"IsLocal": true,
"IsArray": false,
"Origin": "___SYSTEM",
"Qualifiers": ""
}

Please help how to achieve this requirement.

[counterf (Jack Hunter)]

You need to use mutate to do this, I think.
JDATA is an array. To copy the data from the array you need to do something like this:

  mutate {
             copy => { "[JDATA][0][Name]" => "JDATA_NAME"}
             copy => { "[JDATA][0][Value]" => "JDATA_VALUE"}
             copy => { "[JDATA][0][Type]" => "JDATA_TYPE"}
             copy => { "[JDATA][0]..... 
          }

I am not sure if you need to keep [0] or change accordingly. Just try.

[magnusbaeck (Magnus Bäck)]

Ues a json filter to parse the JSON string in the JDATA field, then use a split filter to split the resulting array field into multiple events.