How to separate objects inside an array of objects?

Elastic Stack Logstash
1

Hi,

I still new to ELK and I have been trying to process data that come to me as json file. The json file looks lie this.

[
	{
		"user": "Beta",
		"percent": 28,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987793,
		"Location": "locationB",
		"desk": "MAC"
	},
	{
		"user": "Alpha",
		"percent": 86,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987733,
		"Location": "locationA",
		"desk": "LIN"
	},
	{
		"user": "Charlie",
		"percent": 03,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987724,
		"Location": "locationA",
		"desk": "LIN"
	},

	{
		"user": "test",
		"percent": 15,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987778,
		"Location": "locationB",
		"desk": "MAC"
	},

	{
		"user": "Delta",
		"percent": 28,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987793,
		"Location": "location1",
		"desk": "MAC"
	},

	{
		"user": "Juliana",
		"percent": 28,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987793,
		"Location": "location1",
		"desk": "MAC"
	}
]

Each object inside of that array has to be index. Is there a way to do that? How can I separate or process each object separate?

2

If you want each entry in the array to be a separate event then use a split filter.

3

I added the split filter but I am not sure if I am using it right since my array doesn't have a field name/id before the array of objects starts. Below is what I am planning to try today and I added some comments with what I am trying to get but I feel like maybe I need a loop or so.


#This is my input:

input {
	http {
		port => 8287
		ssl => true
		ssl_certificate_authorities => ["xxxxxxxxxxxxxxxx.crt"]
		ssl_certificate => "path.crt"
		ssl_key => "xxxxxxxxxxxxxxxxxx"
		ssl_verify_mode => peer
	}
}


filter {

	split {
		field => "/n"
	}
#After split I want each object to be parse shoudl I use json filter?
	json {source => "message"}

#Then I want to use this grock for the parse of each object/event
	grok {
	match => { "desktop" => "(?<site>^.{2}%{DATA}-%{DATA}%{INT:pod}%{GREEDYDATA}" }
	}

#I want to add and remove fields in each object/event	
	mutate {
	add_field => { 'processed_at' => "%{@timestamp}" }
	remove_field => ["headers"]
	}

	date {
		match => ['startTime', "yyyy-MM-dd'T'HH:mm:ss", "ISO8601"]
		remove_field => ['startTime']
	}
}


#Then I want to send/index each event that was parse.
output {

elasticsearch {
	hosts=>["https:xxxxxx:9200, "https:xxxxxx:9200]
	index => "testCM"
	user => XXXX
	password => "xxxxxxxxx"
	keystore => "xxxx.jks"
	keystore_password => "xxxxxx"
	cacert => "xxxxxxxxxxxxx.crt"
	}
}
4

Does the below sounds right?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.