Json file not parsing getting error unexpected character (':'' (code 58))

Elastic Stack Logstash
1

I have json files coming in the following format:

[
	{
		"user": "Beta",
		"percent": 28,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987793,
		"Location": "locationB",
		"desk": "MAC"
	},
	{
		"user": "Alpha",
		"percent": 86,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987733,
		"Location": "locationA",
		"desk": "LIN"
	},

	{
		"user": "Charlie",
		"percent": 03,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987724,
		"Location": "locationA",
		"desk": "LIN"
	},

	{
		"user": "test",
		"percent": 15,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987778,
		"Location": "locationB",
		"desk": "MAC"
	},

	{
		"user": "Delta",
		"percent": 28,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987793,
		"Location": "location1",
		"desk": "MAC"
	},

	{
		"user": "Juliana",
		"percent": 28,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987793,
		"Location": "location1",
		"desk": "MAC"
	}
]

This is my input:

input {
	http {
		port => 8287
		ssl => true
		ssl_certificate_authorities => ["xxxxxxxxxxxxxxxx.crt"]
		ssl_certificate => "path.crt"
		ssl_key => "xxxxxxxxxxxxxxxxxx"
		ssl_verify_mode => peer
	}
}


filter {
	grok {
	match => { "desktop" => "(?<site>^.{2}%{DATA}-%{DATA}%{INT:pod}%{GREEDYDATA}" }
	}
	json {source => "message"}
	
	mutate {
	add_field => { 'processed_at' => "%{@timestamp}" }
	remove_field => ["headers"]
	}

	date {
		match => ['startTime', "yyyy-MM-dd'T'HH:mm:ss", "ISO8601"]
		remove_field => ['startTime']
	}
}

output {

elasticsearch {
	hosts=>["https:xxxxxx:9200, "https:xxxxxx:9200]
	index => "testCM"
	user => XXXX
	password => "xxxxxxxxx"
	keystore => "xxxx.jks"
	keystore_password => "xxxxxx"
	cacert => "xxxxxxxxxxxxx.crt"
	}
}
2

Quote at the end is missing by copy+paste or in the original configuration?

3

I had to re-typed the config here but the original has the double quotes. The file gets to Elastic and Kibana but it doesntparse right. When I query the index this one dont have the objects and/or values.

4

Few tips:
Check how does your json arrive, usually should be in the message field.
Are you sure that the field named desktop is OK?
grok { match => { "desktop" ?
As fair as I can remember, you should remove square brackets [ ] from JSON message.
Use only debug in output:

output {
   stdout { codec => rubydebug{} }
}
5 
type or paste code here

[
	{
		"user": "Beta",
		"percent": 28,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987793,
		"Location": "locationB",
		"desktopGR": "MAC"
	},
	{
		"user": "Alpha",
		"percent": 86,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987733,
		"Location": "locationA",
		"desktopGR": "LIN"
	},

	{
		"user": "Charlie",
		"percent": 03,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987724,
		"Location": "locationA",
		"desktopGR": "LIN"
	},

	{
		"user": "test",
		"percent": 15,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987778,
		"Location": "locationB",
		"desktopGR": "MAC"
	},

	{
		"user": "Delta",
		"percent": 28,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987793,
		"Location": "location1",
		"desktopGR": "MAC"
	},

	{
		"user": "Juliana",
		"percent": 28,
		"startTime": "2022-07-07T11:31:45",
		"type": "CPU",
		"total": 1072987793,
		"Location": "location1",
		"desktopGR": "MAC"
	}
]

I fixed the jsonfile I forgot to put the whole name.

I previously removed the brackets manually and try to test it using another input (file) but still the same. The message appears has it but it is not storing the objects.

6

The whole array is ending in the message field.
What exactly will the codec rubydebug will do for the output?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.