Logstash filter split array of json into individual objects

Elastic Stack Logstash
1

Hi ,
iam trying to ingest each json object of array as a new entry/event in dynatrace using logstash. This is my json array

{"RequestEventList":[{"Instant":"2023-01-27T09:00:01.16141Z","RequestKey":"3fcbef69-9-10a6080eb08b","RequestEventName":"ExtensionExecuted","ModuleKey":"585df-8ed2-b2ebe2955167","ModuleName":"LEAP","ApplicationKey":"15-a75dedc51231","ApplicationName":"LEAP","RequestEventDetails":{"TK":"b4882d2f-7175-633c-afdb-6430c1f12b86","TN":"Users","EK":"ce1ac246-4f2e-4b04-b8ee-5dd4cdfa410c","EN":"Development","FN":"E3RG5-DV51OF","ON":"CallOracleDB_Object","OK":"aa2966fa-b59c-af670b9b251c","OEN":"FILE_REQUIREMENTS_CS","OEK":"256cde67-6d5f-4f07-8e13-61fb7b761b9e","NE":"3","D":"455","EC":"0","EPN":"FileRequirement","EPK":"671f034b-df12-42d7-ac2b-987494539e97","AN":"(PageRender)","AK":"671f034b-df12-42d7-ac2b-987494539e97.#(PageRender)"}},{"Instant":"2023-01-27T09:01:09.37939Z","RequestKey":"2b06e08a-5828-40dc-b77a-c1d10752dc35","RequestEventName":"ExtensionExecuted","ModuleKey":"5857d2-b2ebe2955167","ModuleName":"LEAP","ApplicationKey":"f411-a75dedc51231","ApplicationName":"LEAP","RequestEventDetails":{"TK":"b4882d2f-7175-633c-afdb-6430c1f12b86","TN":"Users","EK":"ce1ac246-4f2e-4b04-b8ee-5dd4cdfa410c","EN":"Development","FN":"E3RG5-DV51OF","ON":"CallOracleDB_Object","OK":"aa2966f8-2e3d-498a-b59c-af670b9b251c","OEN":"FILE_REQUIREMENTS_CS","OEK":"256cde67-6d5f-4f07-8e13-61fb7b761b9e","NE":"3","D":"485","EC":"0","EPN":"FileRequirement","EPK":"671f034b-df12-42d7-ac2b-987494539e97","AN":"(PageRender)","AK":"671f034b-df12-42d7-ac2b-987494539e97.#(PageRender)"}}],"ResultsTruncated":true}

The output format that am looking for each json object is

"Instant": "2023-01-27T09:00:01.16141Z",
"RequestKey": "3fcbef69-99a4-10a6080eb08b",
"RequestEventName": "ExtensionExecuted",
"ModuleKey": "5857db79-42ebe2955167",
"ModuleName": "LEAP",
"ApplicationKey": "f411e5-a75dedc51231",
"ApplicationName": "LEAP",
"RequestEventDetails": 
"RequestEventDetails.TK": "b4882d3c-afdb-6430c1f12b86",
"RequestEventDetails.TN": "Users",
"RequestEventDetails.EK": "ce1ac246-404-b8ee-5dd4cdfa410c",
"RequestEventDetails.EN": "Development",
"RequestEventDetails.FN": "E3RG5-DV51OF",
"RequestEventDetails.ON": "CallOracleDB_Object",
"RequestEventDetails.OK": "aa2966f8-2e3d-498a-b59c-af670b9b251c",
"RequestEventDetails.OEN": "FILE_REQUIREMENTS_CS",
"RequestEventDetails.OEK": "256cde67-6d5f-4f07-8e13-61fb7b761b9e",
"RequestEventDetails.NE": "3",
"RequestEventDetails.D": "455",
"RequestEventDetails.EC": "0",
"RequestEventDetails.EPN": "FileRequirement",
"RequestEventDetails.EPK": "671f034b7-ac2b-987494539e97",
"RequestEventDetails.AN": "(PageRender)",
"RequestEventDetails.AK": "671f034b-df12-987494539e97.#(PageRender)"

My conf file is

input { 
   stdin{}
}

filter {
    mutate { gsub => [ "message","\r\n","" ] }
    
    json { source => "message" }
    split { field => "message" }
    split { field => "[RequestEventList][RequestEventDetails]" }
}

output {
    dynatrace {
        id => "dynatrace_output"
        ingest_endpoint_url => ""
        api_key => ""
    }

    stdout {
        codec => "rubydebug"
    }
}

But am getting all the objects in every entry/event on dynatrace dashboard. sample output below and am also getting "tags" => [
[0] "_split_type_failure"
]
Would be helpful if anyone could help me to solve this.

{
  "RequestEventList": [
    {
      "Instant": "2023-01-27T09:00:01.16141Z",
      "RequestKey": "3fcbef69-99a4-4337-87c8-10a6080eb08b",
      "RequestEventName": "ExtensionExecuted",
      "ModuleKey": "5857db79-4e53-47df-8ed2-b2ebe2955167",
      "ModuleName": "LEAP",
      "ApplicationKey": "f411e8a3-e5d4-4ae4-a815-a75dedc51231",
      "ApplicationName": "LEAP",
      "RequestEventDetails": {
        "TK": "b4882d2f-7175-633c-afdb-6430c1f12b86",
        "TN": "Users",
        "EK": "ce1ac246-4f2e-4b04-b8ee-5dd4cdfa410c",
        "EN": "Development",
        "FN": "E3RG5-DV51OF",
        "ON": "CallOracleDB_Object",
        "OK": "aa2966f8-2e3d-498a-b59c-af670b9b251c",
        "OEN": "FILE_REQUIREMENTS_CS",
        "OEK": "256cde67-6d5f-4f07-8e13-61fb7b761b9e",
        "NE": "3",
        "D": "455",
        "EC": "0",
        "EPN": "FileRequirement",
        "EPK": "671f034b-df12-42d7-ac2b-987494539e97",
        "AN": "(PageRender)",
        "AK": "671f034b-df12-42d7-ac2b-987494539e97.#(PageRender)"
      }
    },
    {
      "Instant": "2023-01-27T09:01:09.37939Z",
      "RequestKey": "2b06e08a-5828-40dc-b77a-c1d10752dc35",
      "RequestEventName": "ExtensionExecuted",
      "ModuleKey": "5857db79-4e53-47df-8ed2-b2ebe2955167",
      "ModuleName": "LEAP",
      "ApplicationKey": "f411e8a3-e5d4-4ae4-a815-a75dedc51231",
      "ApplicationName": "LEAP",
      "RequestEventDetails": {
        "TK": "b4882d2f-7175-633c-afdb-6430c1f12b86",
        "TN": "Users",
        "EK": "ce1ac246-4f2e-4b04-b8ee-5dd4cdfa410c",
        "EN": "Development",
        "FN": "E3RG5-DV51OF",
        "ON": "CallOracleDB_Object",
        "OK": "aa2966f8-2e3d-498a-b59c-af670b9b251c",
        "OEN": "FILE_REQUIREMENTS_CS",
        "OEK": "256cde67-6d5f-4f07-8e13-61fb7b761b9e",
        "NE": "3",
        "D": "485",
        "EC": "0",
        "EPN": "FileRequirement",
        "EPK": "671f034b-df12-42d7-ac2b-987494539e97",
        "AN": "(PageRender)",
        "AK": "671f034b-df12-42d7-ac2b-987494539e97.#(PageRender)"
      }
    }
  ],
  "ResultsTruncated": true
}
2

Why are you the third person to ask this question in a day? (First, second.)

1 Like
3

no idea. could you please suggest where am i going wrong with my filters?

4

Please read the other responses. If you have additional questions then explain why those responses did not help. Ask a more specific question.

5

Badger is my hero :innocent:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.