Logstash - csv with a json column - Add a new nested field from the existing field if it is integer

Below are the details: - CSV File,

visitid,labreport,
11111,"[{""test"":""BUN"",""value"":""0.5""},{""test"":""Creatinie"",""value"":""0.8""},{""test"":""Potassium"",""value"":""3.5""}]",
22222,"[{""test"":""BUN"",""value"":""1""},{""test"":""Creatinie"",""value"":""0.8""},{""test"":""Potassium"",""value"":""3.5""}]",
33333,"[{""test"":""BUN"",""value"":""1""},{""test"":""Creatinie"",""value"":""0.8""},{""test"":""Potassium"",""value"":""3.5""}]",
44444,"[{""test"":""BUN"",""value"":""1""},{""test"":""Creatinie"",""value"":""0.8""},{""test"":""Potassium"",""value"":""3.5""}]",
55555,"[{""test"":""BUN"",""value"":""1""},{""test"":""Creatinie"",""value"":""0.8""},{""test"":""Potassium"",""value"":""3.5""}]",
66666,"[{""test"":""BUN"",""value"":""1""},{""test"":""Creatinie"",""value"":""0.8""},{""test"":""Potassium"",""value"":""3.5""}]",

Index Mapping

PUT /test-index
{
"mappings" : {
"properties" : {
"visitid" : { "type" : "integer"},
"labreport" : {
"type" : "nested",
"properties" : {
"test" : { "type" : "text"},
"value" : { "type" : "text" },
"numValue" : { "type" : "text" }
}
}

    }
}

}

Logstash conf
input {
beats {
host => "0.0.0.0"
port => "5044"
codec => "json"
}
}

filter {
csv {
skip_header => "true"
separator => ","
columns => ["visitid","labreport"]
}

json {
source => "labreport"
target => "labreport"
}
}

output {
stdout {
codec => json
}
elasticsearch {
hosts => "http://localhost:9200"
index => "test-index"
action => "update"
doc_as_upsert => true
document_id => "%{visitid}"
}
}

What i wanted is i wanted to push the value of value column in the json to numValue in my nested mappings if it is a numeric field. i.e add a new field in the nested json array and copy the value to the new variable numValue only if the value is numeric ? Support please

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.