Nested json field mutate string to number

Elastic Stack Logstash
1

Hello, I'm trying to convert a nested json field from a string to an integer. I've tried everything I've found on the web. I'm hoping you can give me some direction as to what I'm doing wrong. I'm trying to convert the resp_time_ms field to integer. The server that is running the data and config below is a Windows Server 2012 and the server receiving the data is a RedHat LInux box. The data and config below work to put data into Elasticsearch but the resp_time_ms goes in as a string. Also, I've tried deleting the index and also tried creating a new index by changing the indexname below.

Here is my data :
{"test_results":{"test":{"test_id":"1a21db7d-21fb-4f2f-897f-1fde26531597","type":"webpage","application_id":"RestServices","component_id":"","env_name":"INT","test_date":"2017-11-1514:21:21:000","pass_fail":"passed","webpage_test":{"test_id":"1a21db7d-21fb-4f2f-897f-1fde26531597","dashboard":"DB_Trx_PageLoad","webpage_result":[{"test_id":"1a21db7d-21fb-4f2f-897f-1fde26531597","page_name":"TRX_Rest_1","time_stamp":"2017-11-1514:21:21:000","resp_time_ms":"31.000","status":"passed"},{"test_id":"1a21db7d-21fb-4f2f-897f-1fde26531597","page_name":"TRX_Rest_2","time_stamp":"2017-11-1514:21:21:000","resp_time_ms":"609.000","status":"passed"},{"test_id":"1a21db7d-21fb-4f2f-897f-1fde26531597","page_name":"TRX_Rest_3","time_stamp":"2017-11-1514:21:21:000","resp_time_ms":"47.000","status":"passed"},{"test_id":"1a21db7d-21fb-4f2f-897f-1fde26531597","page_name":"TRX_Rest_4","time_stamp":"2017-11-1514:21:21:000","resp_time_ms":"16.000","status":"passed"},{"test_id":"1a21db7d-21fb-4f2f-897f-1fde26531597","page_name":"TRX_Rest_5","time_stamp":"2017-11-1514:21:21:000","resp_time_ms":"31.000","status":"passed"}]}}}}

Here is my config :
input
{
file
{
type => "DeploymentScoring"
path => [ "D:/DeployTest/AutomatedDeployTestResults/INT/input/*.json" ]
sincedb_path => "D:/DeployTest/AutomatedDeployTestResults/INT/input/DplyTestSinceDB"
start_position => "beginning"
}
}
filter{
json{
source => "message"
}
date {
match => ["test_date", "yyyy-MM-dd HH:mm:ss.SSSZ"]
target => "@timestamp"
}
mutate {
convert => [ "%[test_results][test][webpage_test][webpage_result][resp_time_ms]", "integer" ]
}
}
output
{
elasticsearch {
hosts => "hostname:9200"
manage_template => false
index => "indexname"
document_type => "doctype"
}
}

Thanks in advance.

2

convert => [ "%[test_results][test][webpage_test][webpage_result][resp_time_ms]", "integer" ]

Drop the "%".

I think it still won't work since the webpage_result subfield contains an array. In that case you'll have to use a ruby filter.

3

Thanks Magnus. Ya I tried it with and without the %. I'll do some research on the Ruby filter. Thanks for the help.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.